![](https://csdnimg.cn/release/download_crawler_static/10691838/bg11.jpg)
2.5. LINEAR ALGEBRA
17
X
1
([0
;
0
:
2]) = [0
;
0
:
2]
=
2 F
. In particular, the probability that
X
0
:
2 is not dened. What
is going on here is that
X
is not
F
-measurable: the probability space
f
;
F
; P
g
is not `rich'
enough to `measure'
X
.
Let
f
;
F
; P
g
be [0
;
1] with the uniform probability. For
n
1, let
X
n
(
!
) =
n
1
f
!
1
=n
g
.
Note that
X
n
(
!
)
!
0 as
n
! 1
, for all
!
6
= 0. Thus,
P
(
X
n
!
0) = 1 and we write
X
n
a:s:
!
0
and we say that
X
n
converges to 0
almost surely
. Note however that
E
(
X
n
) = 1
9
E
(0) = 0.
See Lebesgue's theorem in Appendix A.
Recall Borel-Cantelli and assume that
E
(
X
2
n
)
=n
2
. Fix any
>
0 and note that
P
1
n
=1
P
(
j
X
n
j
>
)
P
1
n
=1
E
(
X
2
n
)
=
2
<
1
, so that
P
(
j
X
n
j
> ;
i.o.
g
= 0. This shows that
X
n
a:s
!
0.
Recall also that
E
[
X
j
Y
] is some function
g
(
Y
) with the property that
E
((
X
g
(
Y
))
h
(
Y
)) =
0 for all
h
(
). That is,
X
E
[
X
j
Y
]
? f
h
(
Y
)
; h
(
) is a function
g
. Thus,
E
[
X
j
Y
] is the projection
of
X
onto the subspace of function of
Y
. Also, the projection of a point onto an hyperplane
is the closest element of the hyperplane to that point. Thus,
E
[
X
j
Y
] is the function
g
(
Y
) of
Y
that minimizes
E
((
X
g
(
Y
))
2
). These facts are the
projection property
and the
minimum
mean squares estimator property
of conditional expectation.
2.5 Linear Algebra
Appendix B reviews the main ideas of Linear Algebra. Here are some points that we use
frequently.
The elementary row operations (eros) consist of either adding to a row of a matrix a multiple
of another row or of interchanging two rows. Consider then the homogeneous equations
Ax
= 0
where
A
2 <
m
n
and
x
2 <
n
. By performing eros, we can bring the matrix
A
to a row echelon
form where row
k
has the form [0
; : : : ;
0
;
; : : : ;
] and its rst nonzero term, called its pivot,
corresponds to column
n
(
k
) where
n
(
k
) is strictly increasing. The eros do not change the
solutions of
Ax
= 0. Moreover, if
A
has the row echelon form, then we can solve the equations
bottom up. Consider the last row
L
. Each term
m > n
(
L
) corresponds to an element of
x
that we can set arbitrarily. The pivot then corresponds to an element of
x
uniquely determined
from the previously selected values. Continuing in this way, we nd that the free variables of
x
are the non-pivot elements. This argument shows that if
n > m
, then there must be non-pivot
elements and there are innitely many solutions to
Ax
= 0. See Problem 2.6 for an illustration.
A
basis
of some linear subspace
V
is a collection of linearly independent vectors of that space
such that each element can be written as a linear combination of the basis vectors. Assume that
there is a basis
A
= [
a
1
j j
a
m
]. Then any collection
f
v
1
; : : : ; v
n
g V
with
n > m
must be
linearly dependent. Indeed,
v
i
=
Aw
i
for
i
= 1
; : : : ; n
. Hence, 0 =
P
n
i
=1
x
i
v
i
=
P
n
i
=1
x
i
Aw
i
=
A
(
P
n
i
=1
x
i
w
i
) is equivalent to 0 =
P
n
i
=1
x
i
w
i
, which is a system of
m
equations with
n > m
variables, and we know from the previous discussion that it has innitely many solutions. It
follows that all bases must have the same number of elements, which is called the
dimension
of the subspace.
The following result is important in the study of Gaussian random variables.
Theorem 2.1.
Diagonalization of Hermitian Matrix
Let
H
2
C
n
n
be a
Hermitian matrix
, i.e., such that
H
=
H
.
The eigenvalues
1
; : : : ;
n
of
H
are real (they are not necessarily distinct);
H
has
n
orthonormal eigenvectors
f
u
1
; : : : ; u
n
g
that form a basis for
C
n
. That is,
u
i
u
j
=
1
f
i
=
j
g
.