Introduction
Markov Chains is a recurrent topic in systems analysis. The vast literature on the
subject and applicability attests its importance and timeliness. A host of authors
have delved into the formalism, proposing new modelling primitives, or enhancing
numerical tractability throughout the years. However, if one wants to learn about its
fundamental underpinnings, however simple, the overwhelming number of outcomes
prevents others from deepening the knowledge. Another problem is that sometimes
the wealth of previous results about Markov Chains focuses on specic audiences,
e.g., statisticians, mathematicians, and so on, seldom on computer programmers.
We aim to bridge this gap in this work, by presenting Markov Chains in simple
terms and then tackling the programming issues within the formalism. This book is
dedicated to programmers and people initiating studies in Markov Chains. It aims to
expose Markov Chains to programmers (at any level) having elementary mathemat-
ical background and willingness to delve into this modelling and analysis approach.
If you are looking for more formal aspects, please look elsewhere in the vast lit-
erature on this subject. This book is about understanding the basic principles sur-
rounding Markov Chains and how to implement basic numerical methods to derive
actionable indices for analysis. It has been written for programmers or engineers with
enthusiasm for computing (and coding), eager to get their hands dirty on Markov
Chains and willing to grasp results in short time.
In pursuing this objective, I shall:
• Employ as little mathematical notation as possible.
• Propose modelling with a corresponding spreadsheet or code (usually in C in
this edition).
• Simplify and convey notions with matching implementations.
The book does not address the mathematical foundations of Markov Chains and
intricate solution mechanisms such as Chapman-Kolmogorov equations, Perron-Fro-
benius theory, Krylov sub-spaces, Chebyshev methods, Arnoldi or GMRES solution
methods, and other similar concepts. Please, do submerse in this literature (else-
where) if it is relevant to you. There are a lot of interesting notions lurking in previous
research and, depending on your background, it might prove useful in your career.
For example, you could start looking at this book by Häggström [2] or Trivedi & Bob-
1
我的内容管理
展开
