离散时间马尔可夫链的稳定性条件

需积分: 12 78 浏览量更新于2024-07-31 收藏 2.67MB PDF 举报

"这篇文章主要探讨了离散时间马尔科夫链的稳定性，包括拓扑稳定性和概率稳定性之间的关系，并提出了适用于广泛链类别的统一稳定性标准。通过动态规划公式的形式化表述，建立了第一进入时间的矩估计，从而扩展了Lyapunov-Foster准则，用于证明各种稳定性条件。即使在非不可约的情况下，这些结果也保证了遍历定理的有效性，为确定收敛速率提供了一种更普遍的测试函数方法。" 离散时间马尔科夫链（Discrete-Time Markov Chain, DTMC）是统计和概率论中的一个基本概念，用于描述系统状态随时间变化的随机过程，其中下一个状态只依赖于当前状态而与过去的历史无关。在【标题】"Stability of Markovian processes I_ Criteria for discrete-time chains"中，"Stability"指的是DTMC的稳定性，这是研究马尔科夫过程长期行为的重要方面。【描述】提到了几个关键的稳定性形式，包括紧性（Tightness）、哈里斯回转（Harris Recurrence）和遍历性（Ergodicity）。紧性是拓扑学的概念，对于概率分布序列来说，意味着它们会集中在一个相对有限的区域。哈里斯回转是马尔科夫链理论中的一个重要概念，表示链以非零概率无限次返回任意非空开集。遍历性则是马尔科夫链具有统计稳定性的标志，它保证了链在足够长时间后平均行为的唯一性。【标签】"Markovian"指明了讨论的主题是马尔科夫性质的随机过程。在这个过程中，马尔科夫性质意味着未来的状态概率分布仅依赖于当前状态，而与过去的历史无关。文章【部分内容】中提到，作者通过离散形式的 Dynkin 公式来建立统一的稳定性标准。Dynkin 公式是概率论中的一个工具，用于计算随机过程在特定时间达到或离开某个集合的概率。通过对进入“小集合”（Petite Set）的第一步时间的矩估计，他们扩展了Lyapunov-Foster准则。Lyapunov函数是证明马尔科夫链稳定性的一个常用工具，它可以用来刻画系统的负指数增长趋势。这些结果不仅适用于具有连续状态空间的马尔科夫链，而且在状态空间具有适当丰富的小集合时，各种稳定性的概念是等价的。此外，即使在链不一定是不可约（Irreducible）的情况下，这些稳定性条件也能保证遍历定理的适用性，即在足够长的时间尺度上，马尔科夫链的平均行为可以被描述为某个平稳分布。最后，这些研究还为确定马尔科夫链的收敛速率提供了一种更通用的方法，允许使用更广泛的测试函数来分析链的动态。这对于理解和预测复杂系统的行为，尤其是在系统建模、控制理论和数据分析等领域具有重要意义。

Stability of

Markovian

processes

The

chain

will be called

p-bounded

probability

if,

for some measure

this

condition

holds

for

a.e.

initial condition

The

chain

? is bounded

probability by

definition if

its distributions

{pk}

are

tight

for each

fixed initial condition: the

consequences

of this

are

well known

(see

[5]).

The nomenclature

of boundedness in

probability

is derived from

dynamical

systems

literature

(see [22]

for a detailed

discussion).

The

concepts

appear

relatively

unexplored

in discrete-time Markov chain

theory,

although

[16]

uses

the idea of

non-evanescence

and

actually

calls it

recurrence.

These

conditions,

the

continuous-time

context,

have also been called

'recurrence'

conditions:

see,

for

example, [14].

It is

this

sort

plurality

nomenclature that had led

us to

adopt

the term

'stability'

cover

the

various

concepts:

as a

justification

note

that non-evanescence

is related to the

concept

Lagrange-stability

the

dynamical

systems

literature

[4],

whilst a

Markov

chain

induces a

dynamical

system

whose state

space

consists

probability

measures,

and in this

context

tightness

equivalent

Lagrange 'stability'.

We show

in the

section

that

non-evanescence can be

closely

connected with

Harris recurrence

under suitable

conditions.

Clearly,

though,

since the

stochastic

stability

conditions

use

require

irreducibility,

they

are not in

general

equivalent

the

topological

conditions.

Section

define a

class of

chains

('T-chains')

for

which the

stability

conditions are

equivalent

the

irreducible

case,

and

show

that

for T-chains

which

are

reducible,

the

topological

stability

conditions

imply,

essentially,

decomposition

the

space

into a

countable number of

appropriately

stochastically

stable

subsets,

namely

Harris

sets.

In Section

give

different

approach.

define a class of

sets

('petite

sets',

extending

the

idea of small

sets

[26])

which we show

have the

property

that,

all

compact

sets are

petite,

then these

and

other

stability

conditions

are

closely

related.

We also indicate conditions

under which all

compact

sets

are

fact

petite.

2. Continuous

components,

T-chains and

stability

2.1. The

definition

and

existence

T-chains.

Here

show

that,

for a

class of

chains introduced

[33],

the

forms of

stability

above are

closely

related.

For a

probability

a on

Z+

we define the Markov

transition

function

(4)

Ka-E

a(i)Pi.

For two

probabilities

and b on

Z+

we have

KaKb

Ka*b,

where a

denotes the

convolution

a and b.

We let

(1-

e)-1(1,

E2,

)

for 0<

and

denote

the

probability

Z+

which is

supported

Z+.

let KE

denote

the

specific

case where

and

denote

a..

this

situation

always

assume

fixed

positive

number.

begin

with

definition.

547

SEAN P. MEYN

AND R.

TWEEDIE

A kernel

T is called

continuous

component

of a

function

(X, 13)-

(i)

For A

the

function

T(., A)

is lower

semi-continuous;

(ii)

For all

X and

X?,

the measure

T(x, .)

satisfies

K(x, A)

T(x, A).

The

continuous

component

T is

called

non-trivial at

T(x, X)

In this section

will be

particularly

interested

in continuous

components

of the

Markov

transition

function

Ka,

defined

(4).

A chain

will be called

a T-chain

if,

for

some

the

Markov

transition function

admits

a continuous

component

T which is non-trivial for

all

x e

Three

important

classes

Markov

chains are

those

evolving

on a countable state

space,

those

which are

essentially

random

walks,

and those

obtained

from

stochastic

systems:

from

Proposition

3.2 below we see

that

a Markov chain

a countable

state

space

always

a T-chain.

is shown

[33]

that

random walks are T-chains

and

only

if the

increment

distribution is

'spread

out'

with

respect

to the Haar

measure

on the

space.

If X

a smooth

(C?)

manifold

and the Markov chain is

generated by

the

Markovian

system

(5)

,k+l

F(Q)k,

Wk+l),

with

w an

i.i.d.

sequence,

and

F:X

P->X

smooth,

then

is a T-chain if

the

distribution

possesses

lower semi-continuous

density,

and an associated

control

system

is forward accessible

(combine

Theorem

2.1(iii)

[18]

with

Proposition

3.2).

Forward

accessibility

is a

generalization

of the

controllability

condition

from linear

systems

theory.

Hence

if X

Rd and the

function F

appearing

(5)

linear,

that

(k+I

AIk

Bwk+l

for

matrices

A and

B of

appropriate

dimension,

then

0 is a T-chain

the

controllability

matrix

[Ad-lBI..

ABI

has

rank d.

practice,

then,

many

chains of

interest are indeed

T-chains,

and the

results below

can

widely applied.

first show that

for

T-chain,

either

sample paths

converge

they

enter

'stochastically

recurrent'

part

of the

space.

For a set

let A?=

L(x, A)

0}.

easy

to show that the

set

A? is either

empty

absorbing.

Proposition

2.1.

Suppose

that

a T-chain. If A

not

properly

essential,

then

for

each x

EX,

{ {-->

enters

}

Proof.

Let

A=U

Bi,

with

each

inessential;

then from Lemma

1.1,

the sets

Bij

e X:

L(x, Bi)

>j-1}

are also

inessential,

for

any

Thus

where

each

is inessential.

Since

T is lower

semi-continuous,

the sets

Oj^

X:T(x,

Ai)>j-l}

are

open,

Uj-

T(x,

A?)>j-},

jZ+.

Since T is

everywhere

non-trivial we have

for all

T(x,

(UA)

T(x,X)

and

hence the

sets

{O,i,

Uj}

form an

open

cover

548

剩余32页未读，继续阅读

ManRock

粉丝: 117
资源: 6

离散时间马尔可夫链的稳定性条件

Stability of Markovian processes III_ Foster-Lyapunov criteria for continuous-time processes

Stability of Markovian processes II_ Continuous-time processes and sampled chains

Event-triggered H infinity state estimation for discrete-time stochastic genetic regulatory networks with Markovian jumping parameters and time-varying delays

On optimal fault detection for discrete-time Markovian jump linear systems

Reliable Control for Discrete-Time Markovian Jump Singular Systems with Partly Unknown Transition Probabilities

Stabilization for discrete-time LPV systems with Markovian switching under partially known transition probabilities

fault_detection_system_FAULTDETECTION_Markovian_stochastic_Jump!

Stochastic finite-time stability of nonlinear Markovian switching systems with impulsive effects

Study on the security of discrete-variable quantum key distribution over non-Markovian channels

Semi-Markov Toolbox.zip_hap_high_semi markov _semi- markovian_se

最新资源