深度学习机器的数学原理

需积分: 11 91 浏览量更新于2024-07-17 收藏 1.49MB PDF 举报

"《学习机器的数学基础》- Morgan Kaufmann (2005) 是一本深入探讨深度学习机器数学原理的书籍，适合计算机领域的工程师阅读。" 在本书中，作者Nils J. Nilsson详细介绍了机器学习的基础，旨在为读者提供关于这一领域的扎实理论基础。以下是书中部分内容的详细阐述： 1. **引言** - **机器学习的定义**：机器学习是让计算机系统通过经验改善其性能的一种方法。它涉及模式识别、数据挖掘和统计学，旨在使计算机能从数据中自我学习和适应。 - **机器学习的来源**：包括人工智能、神经科学、统计学和控制理论等多学科的交叉融合。 - **机器学习的多样性**：包括监督学习、无监督学习、强化学习等多种类型。 2. **学习输入-输出函数** - **学习类型**：如分类、回归、聚类等，它们的目标是根据输入数据预测相应的输出。 - **输入向量**：是机器学习中用于训练和预测的数据表示形式。 - **输出**：可以是连续值、离散值或者复杂的结构化数据。 - **训练环境**：分为在线学习、批量学习等，决定了数据的获取方式和处理方式。 - **噪声**：指数据中的不准确或错误，影响学习过程和结果。 - **性能评估**：通过各种度量标准（如准确率、召回率、F1分数）来衡量模型的性能。 3. **学习需要偏置** - 在学习过程中，算法必须具备一定的先验知识或假设（即偏置），以避免过度拟合或欠拟合，找到最佳的泛化能力。 4. **应用示例** - 机器学习广泛应用在图像识别、语音识别、自然语言处理、推荐系统等领域，展示了其广泛的实际价值。 5. **参考资料** - 书中列举了重要的参考资料和历史注解，为读者提供了进一步研究的路径。 6. **布尔函数** - **布尔代数**：是逻辑运算的基础，用于表示和简化二元变量之间的关系。 - **图示表示**：通过图形方式展示布尔函数的结构，便于理解和分析。 - **布尔函数类别**：包括项、子句、与非（DNF）和或非（CNF）函数，这些是布尔逻辑的核心组成部分，对于理解机器学习中的决策过程至关重要。本书不仅涵盖了机器学习的基本概念，还深入到布尔函数等具体计算模型，对于理解深度学习机器的运作机制及其背后的数学原理具有极高的价值。无论是初学者还是资深工程师，都能从中获益匪浅。

1.2. LEARNING INPUT-OUTPUT FUNCTIONS 7

-10

-5

-10

-5

500

1000

500

-5

-10

-5

sample f-value

Figure 1.3: A Surface that Fits Four Points

clusions than those that could be derived from the formulas that we previously

had. But with this new formula we can derive C more quickly, given A, than

we could have done before. We can contrast speed-up learning with methods

that create genuinely new functions—ones that might give diﬀerent results after

learning than they did before. We say that the latter methods involve inductive

learning. As opposed to deduction, there are no correct inductions—only useful

ones.

1.2.2 Input Vectors

Because machine learning methods derive from so many diﬀerent traditions, its

terminology is rife with synonyms, and we will be using most of them in this

book. For example, the input vector is called by a variety of names. Some

of these are: input vector, pattern vector, feature vector, sample, example, and

instance. The components, x

, of the input vector are variously called features,

attributes, input variables, and components.

The values of the components can be of three main types. They might

be real-valued numbers, discrete-valued numbers, or categorical values. As an

example illustrating categorical values, information about a student might be

represented by the values of the attributes class, major, sex, adviser. A par-

ticular student would then be represented by a vector such as: (sophomore,

history, male, higgins). Additionally, categorical values may be ordered (as in

{small, medium, large}) or unordered (as in the example just given). Of course,

mixtures of all these types of values are possible.

In all cases, it is possible to represent the input in unordered form by listing

the names of the attributes together with their values. The vector form assumes

that the attributes are ordered and given implicitly by a form. As an example

of an attribute-value representation, we might have: (major: history, sex: male,

8 CHAPTER 1. PRELIMINARIES

class: sophomore, adviser: higgins, age: 19). We will be using the vector form

exclusively.

An important specialization uses Boolean values, which can be regarded as

a special case of either discrete numbers (1,0) or of categorical variables (True,

False).

1.2.3 Outputs

The output may be a real number, in which case the process embodying the

function, h, is called a function estimator, and the output is called an output

value or estimate.

Alternatively, the output may be a categorical value, in which case the pro-

cess embodying h is variously called a classiﬁer, a recognizer, or a categorizer,

and the output itself is called a label, a class, a category, or a decision. Classi-

ﬁers have application in a number of recognition problems, for example in the

recognition of hand-printed characters. The input in that case is some suitable

representation of the printed character, and the classiﬁer maps this input into

one of, say, 64 categories.

Vector-valued outputs are also possible with components being real numbers

or categorical values.

An important special case is that of Boolean output values. In that case,

a training pattern having value 1 is called a positive instance, and a training

sample having value 0 is called a negative instance. When the input is also

Boolean, the classiﬁer implements a Boolean function. We study the Boolean

case in some detail because it allows us to make important general points in

a simpliﬁed setting. Learning a Boolean function is sometimes called concept

learning, and the function is called a concept.

1.2.4 Training Regimes

There are several ways in which the training set, Ξ, can be used to produce a

hypothesized function. In the batch method, the entire training set is available

and used all at once to compute the function, h. A variation of this method

uses the entire training set to modify a current hypothesis iteratively until an

acceptable hypothesis is obtained. By contrast, in the incremental method, we

select one member at a time from the training set and use this instance alone

to modify a current hypothesis. Then another member of the training set is

selected, and so on. The selection method can be random (with replacement)

or it can cycle through the training set iteratively. If the entire training set

becomes available one member at a time, then we might also use an incremental

method—selecting and using training set members as they arrive. (Alterna-

tively, at any stage all training set members so far available could be used in a

“batch” process.) Using the training set members as they become available is

called an online method. Online methods might be used, for example, when the

1.3. LEARNING REQUIRES BIAS 9

next training instance is some function of the current hypothesis and the previ-

ous instance—as it would be when a classiﬁer is used to decide on a robot’s next

action given its current set of sensory inputs. The next set of sensory inputs

will depend on which action was selected.

1.2.5 Noise

Sometimes the vectors in the training set are corrupted by noise. There are two

kinds of noise. Class noise randomly alters the value of the function; attribute

noise randomly alters the values of the components of the input vector. In either

case, it would be inappropriate to insist that the hypothesized function agree

precisely with the values of the samples in the training set.

1.2.6 Performance Evaluation

Even though there is no correct answer in inductive learning, it is important

to have methods to evaluate the result of learning. We will discuss this matter

in more detail later, but, brieﬂy, in supervised learning the induced function is

usually evaluated on a separate set of inputs and function values for them called

the testing set . A hypothesized function is said to generalize when it guesses

well on the testing set. Both mean-squared-error and the total number of errors

are common measures.

1.3 Learning Requires Bias

Long before now the reader has undoubtedly asked why is learning a function

possible at all? Certainly, for example, there are an uncountable number of

diﬀerent functions having values that agree with the four samples shown in Fig.

1.3. Why would a learning procedure happen to select the quadratic one shown

in that ﬁgure? In order to make that selection we had at least to limit a priori

the set of hypotheses to quadratic functions and then to insist that the one we

chose passed through all four sample points. This kind of a priori information

is called bias, and useful learning without bias is impossible.

We can gain more insight into the role of bias by considering the special case

of learning a Boolean function of n dimensions. There are 2

diﬀerent Boolean

inputs possible. Suppose we had no bias; that is H is the set of all 2

Boolean

functions, and we have no preference among those that ﬁt the samples in the

training set. In this case, after being presented with one member of the training

set and its value we can rule out precisely one-half of the members of H—those

Boolean functions that would misclassify this labeled sample. The remaining

functions constitute what is called a “version space;” we’ll explore that concept

in more detail later. As we present more members of the training set, the graph

of the number of hypotheses not yet ruled out as a function of the number of

diﬀerent patterns presented is as shown in Fig. 1.4. At any stage of the process,

10 CHAPTER 1. PRELIMINARIES

half of the remaining Boolean functions have value 1 and half have value 0 for

any training pattern not yet seen. No generalization is possible in this case

because the training patterns give no clue about the value of a pattern not yet

seen. Only memorization is possible here, which is a trivial sort of learning.

log

j = no. of labeled

patterns already seen

< j

(generalization is not possible)

| = no. of functions not ruled out

Figure 1.4: Hypotheses Remaining as a Function of Labeled Patterns Presented

But suppose we limited H to some subset, H

, of all Boolean functions.

Depending on the subset and on the order of presentation of training patterns,

a curve of hypotheses not yet ruled out might look something like the one

shown in Fig. 1.5. In this case it is even possible that after seeing fewer than

all 2

labeled samples, there might be only one hypothesis that agrees with

the training set. Certainly, even if there is more than one hypothesis remaining,

most of them may have the same value for most of the patterns not yet seen! The

theory of Probably Approximately Correct (PAC) learning makes this intuitive

idea precise. We’ll examine that theory later.

Let’s look at a speciﬁc example of how bias aids learning. A Boolean function

can be represented by a hypercube each of whose vertices represents a diﬀerent

input pattern. We show a 3-dimensional version in Fig. 1.6. There, we show a

training set of six sample patterns and have marked those having a value of 1 by

a small square and those having a value of 0 by a small circle. If the hypothesis

set consists of just the linearly separable functions—those for which the positive

and negative instances can be separated by a linear surface, then there is only

one function remaining in this hypothsis set that is consistent with the training

set. So, in this case, even though the training set does not contain all possible

patterns, we can already pin down what the function must be—given the bias.

剩余187页未读，继续阅读

平头哥-dny

粉丝: 2

深度学习机器的数学原理

Mathematical Logic and the Foundations of Mathematics - An Introductory Survey

The Role of Transpose Matrices in Control Theory: Understanding the Mathematical Foundations of ...

The Role of Transpose Matrix in Computer Graphics: Understanding the Mathematical Foundations of 3D ...

Understanding the Mathematical Foundations of Rigid Body Motion and Stress Analysis

Mathematical-structure-of-ARIMA-models-Robert-Nau.pdf

Handbook-of-Mathematical-Methods-in-Imaging

Data-driven-mathematical-model-of-FOLFIRI-treatment-for-colon-cancer

Mathematical-Modelling-of-Behavior-MATH-463:课程MATH-463

Mathematical Foundations of Computer

Mathematical-modeling-of-Huawei-cup:2020年第17届华为杯数学建模一等奖解决方案

最新资源