![](https://csdnimg.cn/release/download_crawler_static/10762472/bg4.jpg)
6
IEEE TRANSACTIONS ON INFORMATION THEORY
January
P,, 5 4emNEcR)
where E(R) is given by (22).
(23)
Proof: Pick a code with M’ = 2M code words which satis-
fies Corollary 1 when the source uses the 2M code words
with equal probability. [The rate, R’ in (21) and (22) is
now (In 2&9/N.] Remove the M words in the code for
which P,, is largest. It is impossible for over half the words
in the code to have an error probability greater than
twice the average; therefore the remaining code words
must satisfy
P,, 5 2eeNEcR”
(24)
Since R’ = (In 2M)/N = R + (In 2)/N, and since O<p< 1,
(22) gives us
E(R’) > E(R) - F
(25)
Substituting (25) in (24) gives us (23), thereby com-
pleting the proof.
III.
PROPERTIES OF THE RELIABILITY CURVE,
E(R)
The maximization of (22) over p and p depends on the
behavior of the function E,,(p, p). Theorem 2 describes
Eo(p, p) as a function of p, and Theorem 3 describes
Eo(p, p) as a function of p. Both theorems are proved in
the Appendix.
Theorem 2
Consider a channel with K inputs, J outputs, and
transition probabilities
Pik, 1 < 7c 5 K
Let p = (pl, . . . ,
pl() be a probability vector on the
channel inputs, and assume that the average mutual
information
I(p) = 5 5 pr, P,,ln
“’
k=l ,=I
*J$l Pi pii
is nonzero. Then, for p 2. 0,
NP,
PI = 0
for p = 0
(26)
E~(P,
P) > 0
for p > 0
(27)
dEo(p,
P) > o
8P
for p > 0
(2%
~-UP,
P)
8P
= I(P)
(29)
p=o
a2E;;, P) I o
(30)
with equality in (30) if and only if both of the following
conditions are satisfied:
1) PiA is independent of Ic for j, k such that pkPik # 0
2) xk;Pili+, p, is independent of j.
Using this theorem, we can easily perform the maxi-
mization of (22) over p for a given p. Define
E(R, 1-4 = max [ -
PR + JWP,
PII
OSP<l
(31)
Setting the partial derivative of the bracketed part of (31)
equal to 0, we get
R = f?_Eo(~t P)
dP
(32)
From (30), if some p in the range 0 5 p 5 1 satisfies (32),
then that p must maximize (31). Furthermore, from (30)
dE,(p, p)/ap is nonincreasing with p, so that a solution
to (32) exists if R lies in the range
@o(~,
P)
dP
5 R L: I(P)
p=i
In this range it is most convenient to use (32) to relate
E(R, p) and R parametrically as functions of p. This
gives us
E’(R p) = &’ (p p) - p i!%‘(” p,
0 ,
dP
R = GLP, P)
dP
O<Pll
Figure 2 gives a graphical construction for the solution
of these parametric equations.
For R < dE,(p, p)/dp jpcl, the parametric equations
(34) and (35) are not valid. In this case, the function
-pR + E,,(p, p) increases with p in the range 0 5 p 5 1,
and therefore the maximum occurs at p = 1. Thus
E(R,p) = E,(l,p) -R for R <
dE’o(P,
P)
dP
(36)
+?=I
The behavior of E(R, p) as a function of R given by
(34)-(36) is shown in Fig. 3; this behavior depends upon
whether t?Eo(p, p)/dp2 is negative or 0. If it is negative,
then R as given by (35) is strictly decreasing with p. Differ-
entiating (34) with respect to p, we get -p d’E,,(p, p)/dp’;
thus E(R, p) is strictly increasing with p for p >_ 0, and
is equal to 0 for p = 0. Thus if R < Z(p), then E(R, p) > 0.
If p is chosen to achieve capacity, C, then for R < C,
E(R, p) > 0, and the error probability can be made to
vanish exponentially with the block length.
Taking the ratio of the derivatives of (34) and (35),
we see that
WR, P)
aR =-P
Thus the parameter p in (34) and (35) has significance
as the negative slope of the E, R curve.
From the conditions following (30), it is clear that if
d2Eo(p, p)/dp’ = 0 for one value of p > 0, it is 0 for all
p > 0. Under these circumstances, R and E(R, p) as
given by (34) and (35) simply specify the point at which
R = I(p), E(R, p) = 0. The rest of the curve, as shown
in Fig. 4, comes from (36).