概率论与随机过程：随机信号处理教程

5星 · 超过95%的资源需积分: 44 116 浏览量更新于2024-07-30 4 收藏 1.26MB PDF 举报

“probability and random processes.pdf”是一本关于随机信号处理的英文教程，旨在为数学背景的工程研究生和访问学者提供一个深入的、自成一体的度量理论、概率论和离散时间随机过程的理论，特别是关注非ergodic和非stationary的随机过程的ergodic和stationary性质。这本书原本是作者更广泛的信息和ergodic理论书籍的第一部分。在“Probability, Random Processes, and Ergodic Properties”这一章中，作者罗伯特·M·格雷探讨了概率论的基础，随机过程以及它们的ergodic性质。以下是该书部分内容的详细说明： 1. **概率和随机过程**：这部分介绍了概率论的基本概念和随机过程的基本定义。概率论是研究随机事件和不确定性现象的数学理论，而随机过程则是一系列随机变量的时间序列，常用于描述物理系统、经济模型或通信系统中的随机行为。 2. **概率空间和随机变量**：概率空间是由样本空间、事件和概率测度构成的三元组，是概率论的基础。随机变量是定义在概率空间上的函数，它将每个样本点映射到实数值，表示某种可测量的量。 3. **随机过程和动力系统**：随机过程可以视为在时间上演变的动力系统，这里讨论了如何将随机性与动态行为相结合。动力系统理论研究系统的长期行为，而当引入随机性时，这种行为变得更加复杂。 4. **分布**：随机变量的分布描述了其可能取值的概率分布。常见的分布有正态分布、泊松分布、二项分布等，它们在统计和工程中有着广泛应用。 5. **扩展**：这一部分可能涉及如何将概率测度从一个较小的空间扩展到更大的空间，保持某些性质不变，这对于理解和分析更复杂的随机现象至关重要。 6. **同构**：两个概率空间如果在概率意义上具有相同的结构，那么它们是同构的。这允许我们通过一个等价的模型来理解不同的随机过程。 7. **标准字母表**：在概率论中，标准字母表通常指的是完备且分离的σ-代数，这是建立概率空间的基础。标准空间的概念有助于简化理论和证明。 8. **标准空间的性质**：这些空间具有一些特殊的属性，比如它们的完备性和分离性，使得在这些空间上进行概率测度的构造和分析更为方便。 9. **简单标准空间**：这些是相对简单的标准空间实例，如Borel代数在实数集上的应用，它们是理解和构建更复杂概率模型的基础。 10. **度量空间**：度量空间是具有距离概念的集合，是研究连续性和收敛性的基础。在概率论中，度量空间被用来定义随机变量的收敛性。 11. **标准空间中的扩展**：在标准空间中，可以方便地扩展概率测度，这在建立随机过程的理论时特别有用。 12. **Kolmogorov扩展定理**：这个定理允许从有限维分布推导出无限维（如时间序列）的概率分布，是随机过程理论的核心工具之一。 13. **无基扩展**：在没有特定基的情况下，如何扩展概率测度是一个挑战，这部分可能涉及如何在不依赖特定序列或时间点的情况下建立概率模型。这本书深入探讨了概率论和随机过程的高级主题，特别是对于那些没有接受过度量理论概率课程的读者，提供了宝贵的教育资源。通过学习这本书，读者能够掌握处理和分析随机信号的理论工具，为在工程、科学和金融等领域应用这些概念打下坚实基础。

4 CHAPTER 1. PROBABILITY AND RANDOM PROCESSES

Thus to prove that P is a probability measure one must show that the preceding inequality is in

fact an equality.

Random Variables

Given a measurable space (Ω, B), let (A, B

) denote another measurable space. The ﬁrst space can

be thought of as an input space and the second as an output space. A random variable or measurable

function deﬁned on (Ω, B) and taking values in (A, B

) is a mapping or function f : Ω → A with

the property that

if F ∈ B

, then f

−1

(F ) = {ω : f(ω) ∈ F } ∈ B. (1.12)

The name random variable is commonly associated with the case where A is the real line and B

the Borel ﬁeld (which we shall later deﬁne) and occasionally a more general sounding name such

as random object is used for a measurable function to include implicitly random variables (A the

real line), random vectors (A a Euclidean space), and random processes (A a sequence or waveform

space). We will use the term random variable in the general sense.

A random variable is just a function or mapping with the property that inverse images of input

events determined by the random variable are events in the original measurable space. This simple

property ensures that the output of the random variable will inherit its own probability measure.

For example, with the probability measure P

deﬁned by

(B) = P (f

−1

(B)) = P ({ω : f(ω) ∈ B}); B ∈ B

(A, B

, P

) becomes a probability space since measurability of f and elementary set theory ensure

that P

is indeed a probability measure. The induced probability measure P

is called the distri-

bution of the random variable f. The measurable space (A, B

) or, simply, the sample space A

is called the alphabet of the random variable f . We shall occasionally also use the notation P f

−1

which is a mnemonic for the relation P f

−1

(F ) = P (f

−1

(F )) and which is less awkward when f

itself is a function with a complicated name, e.g., Π

I→M

If the alphabet A of a random variable f is not clear from context, then we shall refer to f as

an A-valued random variable. If f is a measurable function from (Ω, B) to (A, B

), we will say that

f is B/B

-measurable if the σ-ﬁelds are not clear from context.

Exercises

1. Set theoretic diﬀerence is deﬁned by F − G = F ∩ G

and symmetric diﬀerence is deﬁned by

F ∆G = (F ∩ G

) ∪ (F

∩ G).

(Set diﬀerence is also denoted by a backslash, F \G.) Show that

F ∆G = (F ∪ G) − (F ∩ G)

and

F ∪ G = F ∪ (F − G).

Hint: When proving two sets F and G are equal, the straightforward approach is to show ﬁrst

that if ω ∈ F , then also ω ∈ G and hence F ⊂ G. Reversing the procedure proves the sets

equal.

6 CHAPTER 1. PROBABILITY AND RANDOM PROCESSES

19. Let f : Ω → A be a random variable. Prove the following properties of inverse images:

−1

) =



−1

(G)



−1

(

∞

[

k=1

) =

∞

[

k=1

−1

)

If G ∩ F = ∅, then f

−1

(G) ∩ f

−1

(F ) = ∅.

1.3 Random Processes and Dynamical Systems

We now consider two mathematical models for a random process. The ﬁrst is the familiar one in

elementary courses: a random process is just a sequence of random variables. The second model is

likely less familiar: a random process can also be constructed from an abstract dynamical system

consisting of a probability space together with a transformation on the space. The two models are

connected by considering a time shift to be a transformation, but an example from communica-

tion theory shows that other transformations can be useful. The formulation and proof of ergodic

theorems are more natural in the dynamical system context.

Random Processes

A discrete time random process, or for our purposes simply a random process, is a sequence of random

variables {X

}

n∈I

or {X

; n ∈ I}, where I is an index set, deﬁned on a common probability space

(Ω, B, P ). We usually assume that all of the random variables share a common alphabet, say A.

The two most common index sets of interest are the set of all integers Z = {. . . , −2, −1, 0, 1, 2, . . .},

in which case the random process is referred to as a two-sided random process, and the set of all

nonnegative integers Z

= {0, 1, 2, . . .}, in which case the random process is said to be one-sided.

One-sided random processes will often prove to be far more diﬃcult in theory, but they provide

better models for physical random processes that must be “turned on” at some time or that have

transient behavior.

Observe that since the alphabet A is general, we could also model continuous time random

processes in the preceding fashion by letting A consist of a family of waveforms deﬁned on an interval,

e.g., the random variable X

could be in fact a continuous time waveform X(t) for t ∈ [nT, (n+1)T),

where T is some ﬁxed positive real number.

The preceding deﬁnition does not specify any structural properties of the index set I. In partic-

ular, it does not exclude the possibility that I be a ﬁnite set, in which case random vector would be

a better name than random process. In fact, the two cases of I = Z and I = Z

will be the only

really important examples for our purposes. The general notation of I will be retained, however,

in order to avoid having to state separate results for these two cases. Most of the theory to be

considered in this chapter, however, will remain valid if we simply require that I be closed under

addition, that is, if n and k are in I , then so is n + k (where the “+” denotes a suitably deﬁned

addition in the index set). For this reason we henceforth will assume that if I is the index set for a

random process, then I is closed in this sense.

Dynamical Systems

An abstract dynamical system consists of a probability space (Ω, B, P ) together with a measurable

transformation T : Ω → Ω of Ω into itself. Measurability means that if F ∈ B, then also T

−1

F =

1.3. RANDOM PROCESSES AND DYNAMICAL SYSTEMS 7

{ω : T ω ∈ F } ∈ B. The quadruple (Ω, B, P, T ) is called a dynamical system in ergodic theory. The

interested reader can ﬁnd excellent introductions to classical ergodic theory and dynamical system

theory in the books of Halmos [30] and Sinai [66]. More complete treatments may be found in [2]

[63] [57] [14] [72] [51] [20] [41]. The name dynamical systems comes from the focus of the theory on

the long term dynamics or dynamical behavior of repeated applications of the transformation T on

the underlying measure space.

An alternative to modeling a random process as a sequence or family of random variables deﬁned

on a common probability space is to consider a single random variable together with a transformation

deﬁned on the underlying probability space. The outputs of the random process will then be values

of the random variable taken on transformed points in the original space. The transformation will

usually be related to shifting in time, and hence this viewpoint will focus on the action of time

itself. Suppose now that T is a measurable mapping of points of the sample space Ω into itself. It is

easy to see that the cascade or composition of measurable functions is also measurable. Hence the

transformation T

deﬁned as T

ω = T (T ω) and so on (T

ω = T (T

n−1

ω)) is a measurable function

for all positive integers n. If f is an A-valued random variable deﬁned on (Ω, B), then the functions

: Ω → A deﬁned by f T

(ω) = f (T

ω) for ω ∈ Ω will also be random variables for all n in

. Thus a dynamical system together with a random variable or measurable function f deﬁnes a

single-sided random process {X

}

n∈Z

by X

(ω) = f(T

ω). If it should be true that T is invertible,

that is, T is one-to-one and its inverse T

−1

is measurable, then one can deﬁne a double-sided random

process by X

(ω) = f(T

ω), all n in Z.

The most common dynamical system for modeling random processes is that consisting of a

sequence space Ω containing all one- or two-sided A-valued sequences together with the shift trans-

formation T , that is, the transformation that maps a sequence {x

} into the sequence {x

n+1

}

wherein each coordinate has been shifted to the left by one time unit. Thus, for example, let

Ω = A

= {all x = (x

, x

, . . .) with x

∈ A for all i} and deﬁne T : Ω → Ω by T (x

, x

, . . .) =

, x

, . . .). T is called the shift or left shift transformation on the one-sided sequence space.

The shift for two-sided spaces is deﬁned similarly. This representation is sometimes referred to as

the Kolmogorov representation of a random process.

Some interesting dynamical systems in communications applications do not, however, have this

structure. As an example, consider the mathematical model of a device called a sigma-delta modula-

tor, that is used for analog-to-digital conversion, that is, encoding a sequence of real numbers into a

binary sequence (analog-to-digital conversion), which is then decoded into a reproduction sequence

approximating the original sequence (digital-to-analog conversion) [35] [9] [22]. Given an input se-

quence {x

} and an initial state u

, the operation of the encoder is described by the diﬀerence

equations

= x

− q(u

= e

n−1

+ u

n−1

where q(u) is +b if its argument is nonnegative and −b otherwise (q is called a binary quantizer).

The decoder is described by the equation

ˆx

i=1

q(u

n−i

The basic idea of the code’s operation is this: An incoming continuous time, continuous amplitude

waveform is sampled at a rate that is so high that the incoming waveform stays fairly constant over

N sample times (in engineering parlance the original waveform is oversampled or sampled at many

times the Nyquist rate). The binary quantizer then produces outputs for which the average over

8 CHAPTER 1. PROBABILITY AND RANDOM PROCESSES

N samples is very near the input so that the decoder output ˆx

is a good approximation to the

input at the corresponding times. Since ˆx

has only a discrete number of possible values (N + 1 to

be exact), one has thereby accomplished analog-to-digital conversion. Because the system involves

only a binary quantizer used repeatedly, it is a popular one for microcircuit implementation.

As an approximation to a very slowly changing input sequence x

, it is of interest to analyze the

response of the system to the special case of a constant input x

= x ∈ [−b, b) for all n (called a

quiet input). This can be accomplished by recasting the system as a dynamical system as follows:

Given a ﬁxed input x, deﬁne the transformation T by

T u =

(

u + x − b; if u ≥ 0

u + x + b; if u < 0.

Given a constant input x

= x, n = 1, 2, . . . , N, and an initial condition u

(which may be ﬁxed or

random), the resulting U

sequence is given by

= T

If the initial condition u

is selected at random, then the preceding formula deﬁnes a dynamical

system which can be analyzed.

The example is provided simply to emphasize the fact that time shifts are not the only interesting

transformation when modeling communication systems.

The diﬀerent models provide equivalent models for a given process: one emphasizing the sequence

of outputs and the other emphasising the action of a transformation on the underlying space in

producing these outputs. In order to demonstrate in what sense the models are equivalent for given

random processes, we next turn to the notion of the distribution of a random process.

Exercises

1. Consider the sigma-delta example with a constant input in the case b = 1/2, u

= 0, and

x = 1/π. Find u

for n = 1, 2, 3, 4.

2. Show by induction in the constant input sigma-delta example that if u

= 0 and x ∈ [−b, b),

then u

∈ [−b, b) for all n = 1, 2, . . ..

3. Let Ω = [0, 1) and F = [0, 1/2) and ﬁx an α ∈ (0, 1). Deﬁne the transformation T x = αx,

where r ∈ [0, 1) denotes the fractional part of r; that is, every real number r has a unique

representation as r = K + r for some integer K. Show that if α is rational, then T

x is a

periodic sequence in n.

1.4 Distributions

Although in principle all probabilistic quantities of a random process can be determined from the

underlying probability space, it is often more convenient to deal with the induced probability mea-

sures or distributions on the space of possible outputs of the random process. In particular, this

allows us to compare diﬀerent random processes without regard to the underlying probability spaces

and thereby permits us to reasonably equate two random processes if their outputs have the same

probabilistic structure, even if the underlying probability spaces are quite diﬀerent.

We have already seen that each random variable X

of the random process {X

} inherits a

distribution because it is measurable. To describe a process, however, we need more than simply

剩余216页未读，继续阅读

greenhillberg

粉丝: 0
资源: 2

概率论与随机过程：随机信号处理教程

Probability and Random Processes

Probability, Random Variables and Stochastic Processes 4th edition

Probability and Random processes

Intuitive Probability and Random Processes using MATLAB

Intuitive Probability and Random Processes Using MatLab 答案

Probability and Random Processes for Electrical and Computer Engineers 入门的概率论教材

Probability-Theory——Stochastic-Processes.pdf

Probability, Statistics, and Random Processes for Engineers (4th Edition).pdf

Probability, Random Variables And Random Processes - Schaum's Outlines

Probability, Random Variables and Stochastic Processes 3ed pdf part3

最新资源