5506 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 64, NO. 12, DECEMBER 2015
Fig. 1. Packet reception ratio of broadcast packets for different distances and
payload sizes (for the same setting, the packet reception ratio for unicast packets
is 100%).
Fig. 2. Packet reception ratio of broadcast packets for different numbers of
sender nodes and payload sizes (broadcast rate: two packets per second).
radio (2-dBi antenna gain and 16-dBm transmission power).
Fig. 1 shows the effect of transmission distance and packet
payload size on the probe packet reception ratio. Fig. 2 shows
the packet reception ratio for different numbers of sender nodes
and payload sizes. In addition to the transmission distance,
which directly affects the signal-to-noise ratio at the receiver,
the payload size is another important factor that affects the
probe packet reception ratio significantly. The number of sender
nodes also h as a nonnegligible effect on the probe packet
reception ratio. This means that the probe packet reception ratio
could be different for different node densities.
The use of hello message as an indicator of link quality could
be dangerous when the packet size is not considered in the
evaluation (wh ich is the case of most rou ting protocols). One
may consider using unicast transmissions for probe packets.
However, it is difficult to use unicast packet reception ratio
to estimate the link quality. This is because the MCSs used
for unicast packets could be different for different channel
conditions. Moreover, there could be retransmissions at the
MAC layer, which always result in the successful delivery
of a unicast frame, and therefore, the estimation should be
conducted by taking into account both the MCS and the number
of transmissions. Therefore, the use of unicast probe packet
for link quality estimation is not practical. Due to the dynamic
characteristic of VANETs, it is particularly difficult to de fine
the relationship between the hello packet reception ratio and
the link quality (mainly reflected by the tr ansmission rate).
Therefore, we require an intelligent algorithm th at can adapt
to various scenarios.
Table I shows the corresponding TCP throughput for differ-
ent distances and MCSs, where “AUTO” denotes the default
rate co ntrol algorithm “minstrel.” Minstrel [31] is a widely
implemented rate control algorithm and considered to be one
of the best if not the best. Minstrel defines the measure of
successfulness (of packet transmission) as the throughput. Min-
strel updates the statistics table for every 100 ms. However, all
rates are tried on a regular basis. This is inefficient particularly
when the rate update interval is small. Although the rate control
algorithm can find the best MCS, the throughput is lower than
the best fixed rate due to the inefficient rate switching.
IV. R
ATE ESTIMAT ION ALGORITHM
A. Rate Estimation Based on Q-Learning and
Knowledge Transfer
Our aim is to find the relationship between the best MCS
and the hello (beacon) packet reception ratio. The relationship
varies with the change in environment, which requires an online
mechanism to d educe the relationship. If the best MCS can
be acquired from the probe reception ratio, the sender can set
the best transmission rate without trying all possible MCSs.
This way, the throughput can be improved. Since the network
environment could be different for different road segments,
we use a Q-learning-based approach to learn the best MCS.
To improve the convergence speed of the learning, a transfer-
learning-based approach is used as a supplement.
B. Calculation of Hello Message Reception Ratio
The hello messages are sent with a predefined time interval
(1 s by default). To get an accurate estimation in dynamic sce-
narios where frequent topology changes and packet collisions
are possible, we use ten hello intervals as a sliding window
size (sampling interval). The hello reception r atio is updated
for each hello interval based on the number of received hello
messages in the last ten hello intervals (10 s) as
HRR(c, x)
=
CNT
r
(c,x)
CNT
s
(x)
,CNT
s
(x) >= 10
CNT
r
(c,x)
CNT
s
(x)
·
1 −
1
2
CNT
s
(x)
, otherwise
(1)
where CNT
r
(c, x) is the number of hello messages received at
c from x,andCNT
s
(x) is the number of hello message sent
from x. As shown in the equation, we discount those nodes that
are only neighbors for less than 10 s (in case of CNT
s
(x)<10).