FOREWORD
xviii
probability models in machine-readable form and to develop a single algorithm for
computing the consequences of any expressible probability model.
In the foregoing narrative, there is a fly in the ointment: the lack of expressible prob-
ability models. The formal languages of Bayesian and Markov networks are, in fact,
quite limited in their expressive power. They are, in a sense, probabilistic analogs of
Boolean circuits. To get a sense of what this limitation means, consider the problem
of writing payroll software for a large company. In a high-level programming lan-
guage such as Java, this might involve tens of thousands of lines of code. Now, imag-
ine trying to implement exactly the same functionality by wiring together Boolean
gates. Such a task seems utterly impossible. The circuit would be unimaginably
large, complicated, and opaque, because circuits lack the expressive power to capture
the structure of the problem.
In 1997, Avi Pfeffer, the author of this book and then still a student, along with his
advisor Daphne Koller and collaborator David McAllester, published the seminal
paper on probabilistic programming languages (
PPL
s), providing the key idea linking
probability theory with the expressive power of high-level programming languages.
The idea was to let programs be probability models by introducing stochastic ele-
ments and defining the meaning of a program to be the probability of each possible
execution trace. This idea tied together in a productive way two of the most important
branches of mathematics, and we are just beginning to explore the new possibilities
that have emerged from it.
This book takes the reader gently through these ideas using the Figaro language to
illustrate the concepts and their applications. It avoids unnecessary mathematics and
concentrates on real-world examples, which are laid out in detail and carefully
explained. It is suitable for someone with a typical programming background. As a by-
product of working through the book, the reader will, with less effort than usual,
acquire a strong grasp of the principles and techniques of Bayesian inference and sta-
tistical learning. Perhaps most importantly, the reader will learn the skill of modeling,
which is among the most critical skills any scientist or engineer can have. Figaro and
other
PPL
s allow one to express that skill directly, rapidly, and precisely.
The book is an important step in moving probabilistic programming from the
research laboratories where it has been developed, out into the real world. Undoubt-
edly the capabilities of
PPL
systems will, in some ways, fail to meet this challenge, and
those research laboratories will have their work cut out. On the other hand, readers of
this book will undoubtedly find creative ways to apply Figaro and its relatives to a wide
range of new problems never imagined by the author.
S
TUART
R
USSELL
P
ROFESSOR
OF
C
OMPUTER
S
CIENCE
U
NIVERSITY
OF
C
ALIFORNIA
, B
ERKELEY
www.it-ebooks.info