大卫·芒福德的模式理论：现实世界信号的随机分析

5星 · 超过95%的资源需积分: 9 154 浏览量更新于2024-07-23 1 收藏 6.48MB PDF 举报

《Mummford-Pattern-Theory: The Stochastic Analysis of Real-World Signals》是一本专著，由计算机视觉与图像处理领域的先驱David Mumford和生物数学家Agnès Desolneux合著。本书作为"Applying Mathematics"系列的一部分，探讨了在现实世界信号中发现和理解模式的理论——即Mummford模式理论。该理论主要聚焦于将概率论和统计学的方法应用于对复杂、随机的自然和人造信号进行分析，这些信号可能出现在诸如图像处理、生物信号分析、物理学、地理信息系统等多个领域。书中的核心内容涉及以下几个关键知识点： 1. **模式理论基础**：Mumford模式理论提供了一种数学框架，用于解析和解释自然界中复杂的模式，如纹理、形状和结构。它强调模式的形成并非简单的规则性，而是受到随机性和噪声的影响，这与传统形式的几何学和图论有所不同。 2. **随机分析方法**：书中详细介绍了如何运用随机过程和随机场的概念来研究真实世界的信号，如信号的统计特性、自相似性和分形结构。这包括对信号的统计建模，以及通过随机过程的生成函数和概率密度函数来刻画信号的行为。 3. **信号处理和特征提取**：作者讨论了如何通过滤波器、小波变换和其他信号处理技术，从实际数据中提取有用的模式信息。这些技术对于降噪、图像分割和识别至关重要。 4. **模式识别系统**：书中还涉及了模式识别算法的发展，特别是针对非结构化数据的机器学习方法，如支持向量机、神经网络等，它们在处理复杂模式时表现出强大的适应性和泛化能力。 5. **跨学科应用**：《Mummford-Pattern-Theory》不仅关注理论层面，还提供了多个实际案例，展示了该理论在图像处理（如医学成像）、遥感科学、生态学、生物学（例如细胞形态分析）以及其他工程领域的应用。 6. **版权信息**：请注意，该书享有2010年的版权，并且任何复制或利用材料的行为都必须获得版权所有者的书面许可。总结来说，《Mummford-Pattern-Theory: The Stochastic Analysis of Real-World Signals》是一本深入探讨如何用数学工具理解和分析复杂、现实世界信号的权威之作，对于从事信号处理、计算机视觉、机器学习等领域研究的学者和工程师具有很高的参考价值。

0.1. The Manifesto of Pattern Theory 3

p t kbdg m n N

sing

Continuant – – –––– – – –

Voiced

–––++++ + +

Nasal

– – –––– + + +

Labial

+––+––+ – –

Coronal

–+––+–– + –

++–++–+ + –

Strident

– – –––– – – –

f 8 s S v D z ZtSdZ

thing she this rosebeige chair joy

Continuant + + ++++ + + – –

Voiced

––––+++ + –+

Nasal

– – –––– – – – –

Labial

+–––+–– – – –

Coronal

–+++–++ + ++

+++–+++ – – –

Strident

+ – +++– + + + +

Ta b l e 1 . Features of English consonants [3, Table 4.8].

This deterministic rule-based approach failed, and the state of the art is to

use a family of precisely tuned stochastic models (hidden Markov models) and a

Bayesian maximum a posteriori (MAP) estimator as an improvement.

The identical story played out in vision, in parsingsentences from text, in ex-

pert systems, and so forth. In all cases, the initial hope was that the deterministic

laws of physics plus logical syllogisms for combining facts would give reliable

methods for decoding the signals of the world in all modalities. These simple

approaches seem to always fail because the signals are too variable and the hid-

den variables too subtly encoded. One reason for this is that there are always

so many extraneous factors affecting the signal: noise in the microphone and the

ear, other sounds in the room besides the speaker, variations in the geometry of

the speaker’s mouth, the speaker’s mood, and so on. Although, in principle, one

might hope to know these as well as the word being pronounced, inferring each

extra factor makes the task more difﬁcult. (In some cases, such as radioactive

decay in PET scans, it is even impossible by quantum theory.) Thus stochastic

models are required.

Now, the world is verycomplexand the signals it generates are likewise com-

plex, both in their patterns and in their variability. If pattern theory makes any

sense, some rules must exist whereby the patterns caused by the objects, pro-

cesses, and laws in the world are not like some arbitrary recursively enumerable

4 0. What Is Pattern Theory?

set. For instance, in the movie Contact, the main character discovers life in outer

space because it broadcastsa signal encodingthe primes from 2 to101. If infants

had to recognize this sort of pattern in order to learn to speak, they would never

succeed. By and large, the patterns in the signals received by our senses are cor-

rectly learned by infants, at least to the level required to reconstruct the various

typesofobjectsinourworldand theirproperties,and tocommunicatewithadults.

This is a marvelousfact,and pattern theoryattempts to understandwhythis isso.

Two further key ideas can be added to the manifesto to explain this:

4. The variousobjects, processes,and rules of the worldproducepatterns that

can be described as precise pure patterns distorted and transformed by a

limited family of deformations,similar across all modalities.

5. When all the stochastic factors affecting any given observationare suitably

identiﬁed, they show a large amountof conditional independence.

The last point is absolutely essential to creating reasonably simple stochastic

models. For example, supposeAliceand Bob are two suspects beinginterrogated

in two separate rooms. Then, given what they both know and have planned and

whattheyhavecommunicatedtoeachother,directlyorindirectly,theiractionsare

independent. Seethe manyfurtherexamplesinPearl’s veryimportantbook[174].

To apply pattern theory properly, it is essential to identify correctly the pat-

terns present in the signal. We often have an intuitive idea of the important pat-

terns, but the human brain does many things unconsciously and also takes many

shortcuts to get things done quickly. Thus, a careful analysis of the actual data

to see what they are telling us is preferable to slapping together an off-the-shelf

Gaussian or log-linear model based on our guesses. Here is a very stringent test

of whether a stochastic model is a good description of the world: sample from

it. This is so obvious that one would assume everyone does this, but in actual-

ity, this is not so. The samples from many models that are used in practice are

absurd oversimpliﬁcations of real signals, and, even worse, some theories do not

include the signal itself as one of its random variables (using only some derived

variables),so it is not evenpossible to sample signals from them.

In what ways is pattern theory different from the better-known ﬁeld of statis-

tical pattern recognition? Traditionally,the focus of statistical pattern recognition

wasthe study of oneor moredata sets {x

∈ R

}

α∈I

with the goals of (a) ﬁtting

This was, for instance, the way most traditional speech recognition systems worked: their ap-

proach was to throw away the raw speech signal in the preprocessing stage and replace it with codes

designed to ignore speaker variation. In contrast, when all humans listen to speech, they are clearly

aware of the idiosyncrasies of the individual speaker’s voice and of any departures from normal. The

idea of starting by extracting some hopefully informative features from a signal and only then classi-

fying it is via some statistical algorithm is enshrined in classic texts such as [64].

0.2. The Basic Types of Patterns 5

(parametricandnonparametric)probabilitydistributionsto eachdata set, (b)ﬁnd-

ing optimal decision rules for classifying new data into the correct data set, and

essential issue is the “bias-variance” trade-off: to model fully the complexities of

the data source but not the accidental variations of the speciﬁc data set. When

Grenander ﬁrst proposed pattern theory as a distinct enterprise, his approach had

several very novelaspects. He put forth the following ﬁve propositions:

1. To describe the patterns in typical data sets, one should always look for

appropriatehiddenvariables,intermsofwhichthepatternsaremoreclearly

described.

2. The set of variables, observed and hidden, typically forms the vertices of

a graph, as in Gibbs models, and one must formulate prior probability dis-

tributions for the hidden variables as well as models for the observed vari-

ables.

3. This graphitself mightbe randomand its variability must thenbe modeled.

4. One can list the different types of “deformations” patterns are subject to,

thus creating the basic classes of stochastic models that can be applied.

5. These models should be used for pattern synthesis as well asanalysis.

As thesubjectevolved,statistical patternrecognitionmergedwith theareaof neu-

ral nets, and the ﬁrst two ideas were absorbed into statistical pattern recognition.

Theseso-calledgraphicalmodelsare nowseenas thebreadandbutterof theﬁeld,

and discovering these hidden variables is a challenging new problem. The use of

prior models has become the mainstream approach in vision and expert systems,

as it has been in speech since the 1960s. The other aspects of pattern theory,

however, are still quite distinctive.

0.2 The Basic Types of Patterns

Let us be more precise about the kinds of patterns and deformations referred to

in point 4 above. Real-world signals show two very distinct types of patterns.

We call these (1) value patterns and (2)geometrical patterns. Signals, in general,

are some sort of function f : X → V . The domain may be continuous (e.g., a

part of space such as the retina or an interval of time) or discrete (e.g., the nodes

of a graph or a discrete sample in space or time), and the range may be a vector

space or binary {0, 1} or something in the middle. In the case of value patterns,

6 0. What Is Pattern Theory?

we mean that the features of this pattern are computed from the values of f or

from some linear combinations of them (e.g., power in some frequency band).

In the case of geometric patterns, the function f can be thought of as producing

geometricalpatterns inits domain(e.g., the set ofits pointsofdiscontinuity). The

distinction affects which extra random variables we need to describe the pattern.

For value patterns, we typically add coefﬁcients in some expansion to describe

the particular signal. For geometric patterns, we add certain points or subsets of

the domain or features of such subsets. Traditional statistical pattern recognition

and the traditional theory of stationary processes deals only with the values of

f, not the geometry of its domain. Let us be more speciﬁc and describe these

patterns more explicitlyby distinguishing two sorts of geometric patterns—those

involving deforming the geometry and those involving a hierarchical geometric

pattern:

1. Value patterns and linear superposition. The most standard value model

creates the observedsignalfromthe linearsuperposition ofﬁxed or learned

basis functions. We assume the observed signal has values in a real vector

space s : X −→ V and that it is expanded in terms of auxiliary functions

: X −→ V as

s =Σ

In vision, X is the set of pixels and, for a gray-levelimage, V is the set of

real numbers. Here the coefﬁcients {c

} are hidden randomvariables, and

either thefunctions{s

} maybe a universalbasis such as sinesandcosines

or wavelets or some learned templates, as in Karhunen-Loeveexpansions,

orthe {s

} maybe random. The simplestcase isto allowone of themto be

random and thinkof it as theresidual (e.g., an additivenoise term). An im-

portant case is that of expanding a function into its componentson various

scales (as in wavelet expansions), so the terms s

are simply the compo-

nents of s on scale α. See Figure 2 for an example of a face expanded into

three images representing its structure on ﬁne, medium, and coarse scales.

Other variants are (1)amplitudemodulation in which thelow and high fre-

quencies are combined by multiplication instead of addition, and (2) the

case in which s is just a discrete sample from the full function Σc

Quite often, the goal of such an expansion,from a statistical pointof view,

is to make the coefﬁcients {c

} as independent as possible. If the coefﬁ-

cients are Gaussian and independent, this is called a principal components

analysis (PCA) or a Karhunen-Loeve expansion; if they are non-Gaussian

but independent,it is an ICA. We shall see howsuch expansionsworkvery

well to express the effects of lighting variation on face signals.

剩余412页未读，继续阅读

willy8216

粉丝: 2
资源: 7

大卫·芒福德的模式理论：现实世界信号的随机分析

Pattern Theory The Stochastic Analysis of Real-World Signals

Pattern Theory: From Representation to Inference.pdf

Graph Theory with Algorithms and its Applications

stochastic control in continuous-time and application in finance

rank-based stochastic pooling

counting process theory

an introduction to stochastic differential equations version微盘

Derive the stochastic gradient descent algorithm

point-wise MLP

最新资源