二十年来机器学习：数据驱动的科技进步关键

需积分: 10 68 浏览量更新于2024-07-19 收藏 10.31MB PDF 举报

"《机器学习入门》是一本由Alex Smola和S.V.N. Vishwanathan撰写的专业著作，两位作者分别来自Yahoo! Labs、Purdue University和澳大利亚国立大学。这本书在过去的二十年间随着信息技术的发展，特别是在大数据时代的背景下，将机器学习提升到了核心地位。机器学习不仅成为了信息技术的核心组成部分，而且悄然渗透到日常生活的各个领域，成为推动科技进步的关键因素。本书详细介绍了机器学习的基本概念和原理，涵盖了从数据驱动的学习方法到算法设计，包括监督学习、无监督学习、半监督学习和强化学习等核心技术。它阐述了如何通过智能数据分析处理海量数据，以及如何利用这些分析结果来优化决策和预测，如推荐系统、图像识别、自然语言处理等领域应用。在内容上，读者可以期待深入理解机器学习的工作原理，例如模型训练、特征选择、评估指标等核心环节，以及如何通过实际案例来理解和应用机器学习理论。书中还可能涉及机器学习的历史背景、发展趋势和未来挑战，帮助读者认识到这个领域的广阔前景和潜在影响。版权方面，这本书受到法律保护，未经许可不得复制。首次出版于2008年，由剑桥大学出版社发行。印刷版本采用Monotype Times 10/13pt字体，并使用了LaTeX2ε排版系统。对于想要深入探索机器学习理论与实践的读者来说，《机器学习入门》是一本不可或缺的参考书，它将引领你进入这个充满无限可能的数据智能世界。"

8 1 Introduction

with those issues in an automatic fashion is to normalize the data. We will

discuss means of doing so in an automatic fashion.

Lists: In some cases the vectors we obtain may contain a variable number

of features. For instance, a physician might not necessarily decide to perform

a full battery of diagnostic tests if the patient appears to be healthy.

Sets may appear in learning problems whenever there is a large number of

potential causes of an eﬀect, which are not well determined. For instance, it is

relatively easy to obtain data concerning the toxicity of mushrooms. It would

be desirable to use such data to infer the toxicity of a new mushroom given

information about its chemical compounds. However, mushrooms contain a

cocktail of compounds out of which one or more may be toxic. Consequently

we need to infer the properties of an object given a set of features, whose

composition and number may vary considerably.

Matrices are a convenient means of representing pairwise relationships.

For instance, in collaborative ﬁltering applications the rows of the matrix

may represent users whereas the columns correspond to products. Only in

some cases we will have knowledge about a given (user, product) combina-

tion, such as the rating of the product by a user.

A related situation occurs whenever we only have similarity information

between observations, as implemented by a semi-empirical distance mea-

sure. Some homology searches in bioinformatics, e.g. variants of BLAST

[AGML90], only return a similarity score which does not necessarily satisfy

the requirements of a metric.

Images could be thought of as two dimensional arrays of numbers, that is,

matrices. This representation is very crude, though, since they exhibit spa-

tial coherence (lines, shapes) and (natural images exhibit) a multiresolution

structure. That is, downsampling an image leads to an object which has very

similar statistics to the original image. Computer vision and psychooptics

have created a raft of tools for describing these phenomena.

Video adds a temporal dimension to images. Again, we could represent

them as a three dimensional array. Good algorithms, however, take the tem-

poral coherence of the image sequence into account.

Trees and Graphs are often used to describe relations between collec-

tions of objects. For instance the ontology of webpages of the DMOZ project

(www.dmoz.org) has the form of a tree with topics becoming increasingly

reﬁned as we traverse from the root to one of the leaves (Arts → Animation

→ Anime → General Fan Pages → Oﬃcial Sites). In the case of gene ontol-

ogy the relationships form a directed acyclic graph, also referred to as the

GO-DAG [ABB

00].

Both examples above describe estimation problems where our observations

1.1 A Taste of Machine Learning 9

are vertices of a tree or graph. However, graphs themselves may be the

observations. For instance, the DOM-tree of a webpage, the call-graph of

a computer program, or the protein-protein interaction networks may form

the basis upon which we may want to perform inference.

Strings occur frequently, mainly in the area of bioinformatics and natural

language processing. They may be the input to our estimation problems, e.g.

when classifying an e-mail as spam, when attempting to locate all names of

persons and organizations in a text, or when modeling the topic structure

of a document. Equally well they may constitute the output of a system.

For instance, we may want to perform document summarization, automatic

translation, or attempt to answer natural language queries.

Compound structures are the most commonly occurring object. That

is, in most situations we will have a structured mix of diﬀerent data types.

For instance, a webpage might contain images, text, tables, which in turn

contain numbers, and lists, all of which might constitute nodes on a graph of

webpages linked among each other. Good statistical modelling takes such de-

pendencies and structures into account in order to tailor suﬃciently ﬂexible

models.

1.1.3 Problems

The range of learning problems is clearly large, as we saw when discussing

applications. That said, researchers have identiﬁed an ever growing number

of templates which can be used to address a large set of situations. It is those

templates which make deployment of machine learning in practice easy and

our discussion will largely focus on a choice set of such problems. We now

give a by no means complete list of templates.

Binary Classiﬁcation is probably the most frequently studied problem

in machine learning and it has led to a large number of important algorithmic

and theoretic developments over the past century. In its simplest form it

reduces to the question: given a pattern x drawn from a domain X, estimate

which value an associated binary random variable y ∈ {±1} will assume.

For instance, given pictures of apples and oranges, we might want to state

whether the object in question is an apple or an orange. Equally well, we

might want to predict whether a home owner might default on his loan,

given income data, his credit history, or whether a given e-mail is spam or

ham. The ability to solve this basic problem already allows us to address a

large variety of practical settings.

There are many variants exist with regard to the protocol in which we are

required to make our estimation:

10 1 Introduction

Fig. 1.6. Left: binary classiﬁcation. Right: 3-class classiﬁcation. Note that in the

latter case we have much more degree for ambiguity. For instance, being able to

distinguish stars from diamonds may not suﬃce to identify either of them correctly,

since we also need to distinguish both of them from triangles.

• We might see a sequence of (x

, y

) pairs for which y

needs to be estimated

in an instantaneous online fashion. This is commonly referred to as online

learning.

• We might observe a collection X := {x

, . . . x

} and Y := {y

, . . . y

} of

pairs (x

, y

) which are then used to estimate y for a (set of) so-far unseen



, . . . , x



. This is commonly referred to as batch learning.

• We might be allowed to know X

already at the time of constructing the

model. This is commonly referred to as transduction.

• We might be allowed to choose X for the purpose of model building. This

is known as active learning.

• We might not have full information about X, e.g. some of the coordinates

of the x

might be missing, leading to the problem of estimation with

missing variables.

• The sets X and X

might come from diﬀerent data sources, leading to the

problem of covariate shift correction.

• We might be given observations stemming from two problems at the same

time with the side information that both problems are somehow related.

This is known as co-training.

• Mistakes of estimation might be penalized diﬀerently depending on the

type of error, e.g. when trying to distinguish diamonds from rocks a very

asymmetric loss applies.

Multiclass Classiﬁcation is the logical extension of binary classiﬁca-

tion. The main diﬀerence is that now y ∈ {1, . . . , n} may assume a range

of diﬀerent values. For instance, we might want to classify a document ac-

cording to the language it was written in (English, French, German, Spanish,

Hindi, Japanese, Chinese, . . . ). See Figure 1.6 for an example. The main dif-

ference to before is that the cost of error may heavily depend on the type of

1.1 A Taste of Machine Learning 11

Fig. 1.7. Regression estimation. We are given a number of instances (indicated by

black dots) and would like to ﬁnd some function f mapping the observations X to

R such that f(x) is close to the observed values.

error we make. For instance, in the problem of assessing the risk of cancer, it

makes a signiﬁcant diﬀerence whether we mis-classify an early stage of can-

cer as healthy (in which case the patient is likely to die) or as an advanced

stage of cancer (in which case the patient is likely to be inconvenienced from

overly aggressive treatment).

Structured Estimation goes beyond simple multiclass estimation by

assuming that the labels y have some additional structure which can be used

in the estimation process. For instance, y might be a path in an ontology,

when attempting to classify webpages, y might be a permutation, when

attempting to match objects, to perform collaborative ﬁltering, or to rank

documents in a retrieval setting. Equally well, y might be an annotation of

a text, when performing named entity recognition. Each of those problems

has its own properties in terms of the set of y which we might consider

admissible, or how to search this space. We will discuss a number of those

problems in Chapter ??.

Regression is another prototypical application. Here the goal is to esti-

mate a real-valued variable y ∈ R given a pattern x (see e.g. Figure 1.7). For

instance, we might want to estimate the value of a stock the next day, the

yield of a semiconductor fab given the current process, the iron content of

ore given mass spectroscopy measurements, or the heart rate of an athlete,

given accelerometer data. One of the key issues in which regression problems

diﬀer from each other is the choice of a loss. For instance, when estimating

stock values our loss for a put option will be decidedly one-sided. On the

other hand, a hobby athlete might only care that our estimate of the heart

rate matches the actual on average.

Novelty Detection is a rather ill-deﬁned problem. It describes the issue

of determining “unusual” observations given a set of past measurements.

Clearly, the choice of what is to be considered unusual is very subjective.

A commonly accepted notion is that unusual events occur rarely. Hence a

possible goal is to design a system which assigns to each observation a rating

剩余233页未读，继续阅读

qq_40124410

粉丝: 0
资源: 1

二十年来机器学习：数据驱动的科技进步关键

A_Brief_Introduction_to_Machine_Learning_for_Engineers.pdf

An_Introduction_to_MCMC_for_Machine_Learning

introduction_to_machine_learning

CS480-680_introduction_to_MachineLearning

01_Introduction_tensorflow_machinelearning_源码

Intoduction_to_Machine_Learning_with_Python.2016

MACHINE LAB REPORT 1_latex_machinehsp_machinelearning_

Machine Learning_图像增强_machinelearning_

Introduction to Machine Learning _机器学习_

introduction_to_ml_with_spark:Spark机器学习入门的代码和设置信息

最新资源