
210 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 61, NO. 1, JANUARY 2012
2) All the available channels are positively correlated,
2
i.e.,
p
11
>p
01
.
3) n is an integer value.
3
First, we assume that when n = 1, the transition time from
one state to another in the Markov chain is equal to T . Hereby,
the transition matrix corresponding to transmission time T is
p
00
p
01
p
10
p
11
=
p
(1)
00
p
(1)
01
p
(1)
10
p
(1)
11
.
If n>1, the transition time changes from T to T/n. Based
on assumption 1, the transition matrix corresponding to trans-
mission time T is given by
p
00
p
01
p
10
p
11
=
p
(n)
00
p
(n)
01
p
(n)
10
p
(n)
11
n
.
Accordingly, the determinant of the two matrices is equal,
which leads to
p
00
+ p
11
− 1 =
p
(n)
00
+ p
(n)
11
− 1
n
. (1)
The Markov model shown in Fig. 1 goes to stability, and
therefore, the local balance equation holds:
1 − p
00
1 − p
11
=
1 − p
(n)
00
1 − p
(n)
11
. (2)
In the condition that p
11
>p
01
(assumption 2), a more
simplified expression can be written as follows:
p
(n)
11
=
p
01
+(1 − p
11
)
n
√
p
11
− p
01
1 + p
01
− p
11
(3)
p
(n)
01
=
p
01
(1 −
n
√
p
11
− p
01
)
1 + p
01
− p
11
. (4)
B. Device Mode Transition Circle
As the channel model is put forward, we then propose a
corresponding device activity model, i.e., the device mode
transition circle, as shown in Fig. 2.
We define S(t) as the mode of device within each time slot
T/n, where S(t) ∈{0, 1, 2,...,n}. S(t)=0 denotes that the
device is performing spectrum sensing within sensing interval.
S(t)=1 denotes that the device is performing the first slot
transmission within transmission period. S(t)=n denotes that
the device is performing the nth slot transmission within the
transmission period.
The exact model is expressed as follows.
An SU is supposed to have K devices for multichannel sens-
ing and transmitting. Each device is responsible for sensing and
transmitting on a subchannel within the entire spectrum, which
is equal to the spectrum supported by the hardware capability
2
When p
11
>p
01
, two consecutive slots tend to have the same “good” or
“bad” state, which is defined as positively correlated. Similarly, When p
11
<
p
01
, two consecutive slots tend to have the different “good” or “bad” state,
which is defined as negatively correlated. When p
11
= p
01
, two consecutive
slots tend to have independent state.
3
This will be explained later in this part.
Fig. 2. Device mode transition circle.
on an SU. The summation of spectrum on K subchannels is
equal to the entire spectrum. The criterion of mapping the
entire spectrum into K subchannels is not specified in this
paper. For simplicity, we use channels instead of subchannels
in the following sections. Each of these K devices may only
be in one of two modes: 1) sensing mode and 2) transmission
mode. During sensing mode, each device chooses one of N
channels to sense. It then immediately turns into transmission
mode, which lasts for a transmission period if the channel
is “good.” Otherwise, it rechooses a channel to sense after a
sensing interval.
The SU is supposed to infer the future channel states from
its history of decision and observation. Define ω(t) as a belief
state [2], which represents a sufficient statistic for optimal
decision making. The belief state is obtained as the conditional
probability that the channel state is “1,” given all past decisions
and observations. Thus, we can obtain the belief state in time
slot t + 1 recursively, i.e.,
ω(t + 1)=
p
(n)
01
,S(t)=0
p
(n)
11
,S(t)=n
T (ω(t)), not sensed
where
T (ω(t))
∆
= ω(t)p
(n)
11
+(1 − ω(t)) p
(n)
01
(5)
is the one-step belief update for unobserved channels. The
i-step belief update for unobserved channels can simply be
represented by the iterative form of (5), which is formulated
as T
i
(ω(t)).
There is a stationary point ω
0
of the Markov chain
ω
0
=
p
(n)
01
p
(n)
10
+ p
(n)
01
=
p
01
p
01
+ p
10
.
Based on this model, the long-run r eward based on the benefit
of data transmission and the penalty from sensing cost will be
discussed in t he following sections.
C. Discussion About n
From the description about the model, we can see that an
obvious limitation is that n is an integer value. This is due to
the consideration for model simplification as well as practi-
cal use.