深度学习基础与应用探索

需积分: 9 24 浏览量更新于2024-07-20 1 收藏 20.67MB PDF 举报

"Deep Learning" 是一本由领域内的专家编写的深度学习专著，得到了特斯拉和SpaceX的CEO埃隆·马斯克的高度评价。本书深入介绍了深度学习这一机器学习分支，它允许计算机通过经验学习并理解世界的概念层级。书中涵盖了从线性代数、概率论、信息理论到数值计算和机器学习的基础知识，详细讨论了深度前馈网络、正则化、优化算法、卷积网络、序列建模等实际应用，以及自然语言处理、语音识别、计算机视觉等多个领域的案例。标签: "Deep Machine Neural 深度学习" 在内容方面，本书首先从介绍读者群体和深度学习历史趋势开始，然后进入应用数学和机器学习基础部分：第2章“线性代数”中，讲解了标量、向量、矩阵和张量的基本概念，包括矩阵乘法、身份和逆矩阵、线性相关性、范数、特殊矩阵和向量、特征分解、奇异值分解、伪逆矩阵、迹运算和主成分分析等关键概念。第3章“概率和信息理论”探讨了为什么需要概率，随机变量、概率分布、边缘概率、条件概率、链式规则、独立性和条件独立性、期望、方差和协方差，以及信息论的基本原理，如熵和互信息等，同时列举了一些常见的概率分布。这些章节为理解深度学习模型的构建和优化提供了必要的数学基础。书中还涵盖了数值计算和优化算法，这对于训练神经网络至关重要。此外，卷积网络和序列模型的讲解让读者能够掌握处理图像和时间序列数据的深度学习技术。书中的实践方法指导读者如何在真实世界的应用中实施这些技术，例如在线推荐系统、生物信息学和电子游戏。最后，书中提供了研究视角，讨论了线性因子模型、自动编码器、表示学习、结构化概率模型、蒙特卡洛方法、配分函数、近似推理以及深度生成模型等理论主题，为深入研究深度学习的理论和未来方向提供了指导。这本书适合希望在工业界或研究领域从事深度学习工作的本科生和研究生阅读，也适用于希望在产品或平台上应用深度学习的软件工程师。附带的网站提供了补充材料，支持读者和教师的进一步学习。

CHAPTER 1. INTRODUCTION

Figure 1.1: Example of diﬀerent representations: suppose we want to separate two cate-

gories of data by drawing a line between them in a scatterplot. In the plot on the left, we

represent some data using Cartesian coordinates, and the task is impossible. In the plot

on the right, we represent the data with polar coordinates and the task becomes simple

to solve with a vertical line. (Figure credit: David Warde-Farley)

ten result in much better performance than can be obtained with hand-designed

representations. They also allow AI systems to rapidly adapt to new tasks, with

minimal human intervention. A representation learning algorithm can discover a

good set of features for a simple task in minutes, or a complex task in hours to

months. Manually designing features for a complex task requires a great deal of

human time and eﬀort; it can take decades for an entire community of researchers.

The quintessential example of a representation learning algorithm is the au-

toencoder. An autoencoder is the combination of an encoder function that converts

the input data into a diﬀerent representation, and a decoderfunction that converts

the new representation back into the original format. Autoencoders are trained

to preserve as much information as possible when an input is run through the

encoder and then the decoder, but are also trained to make the new representa-

tion have various nice properties. Diﬀerent kinds of autoencoders aim to achieve

diﬀerent kinds of properties.

When designing features or algorithms for learning features, our goal is usually

to separate the factorsofvariationthat explain the observed data. In this context,

we use the word “factors” simply to refer to separate sources of inﬂuence; the

factors are usually not combined by multiplication. Such factors are often not

quantities that are directly observed but they may exist either as unobserved

objects or forces in the physical world that aﬀect observable quantities, or they

are constructs in the human mind that provide useful simplifying explanations

CHAPTER 1. INTRODUCTION

or inferred causes of the observed data. They can be thought of as concepts or

abstractions that help us make sense of the rich variability in the data. When

analyzing a speech recording, the factors of variation include the speaker’s age,

their sex, their accent and the words that they are speaking. When analyzing an

image of a car, the factors of variation include the position of the car, its color,

and the angle and brightness of the sun.

A major source of diﬃculty in many real-world artiﬁcial intelligence applica-

tions is that many of the factors of variation inﬂuence every single piece of data

we are able to observe. The individual pixels in an image of a red car might be

very close to black at night. The shape of the car’s silhouette depends on the

viewing angle. Most applications require us to disentangle the factors of variation

and discard the ones that we do not care about.

Of course, it can be very diﬃcult to extract such high-level, abstract features

from raw data. Many of these factors of variation, such as a speaker’s accent,

can only be identiﬁed using sophisticated, nearly human-level understanding of

the data. When it is nearly as diﬃcult to obtain a representation as to solve the

original problem, representation learning does not, at ﬁrst glance, seem to help

us.

Deep learning solves this central problem in representation learning by intro-

ducing representations that are expressed in terms of other, simpler represen-

tations. Deep learning allows the computer to build complex concepts out of

simpler concepts. Fig. 1.2 shows how a deep learning system can represent the

concept of an image of a person by combining simpler concepts, such as corners

and contours, which are in turn deﬁned in terms of edges.

The quintessential example of a deep learning model is the feedforward deep

network or multilayerperceptron(MLP). A multilayer perceptron is just a mathe-

matical function mapping some set of input values to output values. The function

is formed by composing many simpler functions. We can think of each applica-

tion of a diﬀerent mathematical function as providing a new representation of the

input.

The idea of learning the right representation for the data provides one per-

spective on deep learning. Another perspective on deep learning is that it allows

the computer to learn a multi-step computer program. Each layer of the repre-

sentation can be thought of as the state of the computer’s memory after executing

another set of instructions in parallel. Networks with greater depth can execute

more instructions in sequence. Being able to execute instructions sequentially of-

fers great power because later instructions can refer back to the results of earlier

instructions. According to this view of deep learning, not all of the information

in a layer’s representation of the input necessarily encodes factors of variation

that explain the input. The representation is also used to store state information

CHAPTER 1. INTRODUCTION



⇥

Elementset

⇥



Elementset

Logistic

Regression

Logistic

Regression

Figure 1.3: Illustration of computational ﬂow graphs mapping an input to an output

where each node performs an operation. Depth is the length of the longest path from input

to output but depends on the deﬁnition of what constitutes a possible computational step.

The computation depicted in these graphs is the output of a logistic regression model,

σ(w

x), where σ is the logistic sigmoid function. If we use addition, multiplication and

logistic sigmoids as the elements of our computer language, then this model has depth

three. If we view logistic regression as an element itself, then this model has depth one.

that helps to execute a program that can make sense of the input. This state

information could be analogous to a counter or pointer in a traditional computer

program. It has nothing to do with the content of the input speciﬁcally, but it

helps the model to organize its processing.

There are two main ways of measuring the depth of a model.

The ﬁrst view is based on the number of sequential instructions that must

be executed to evaluate the architecture. We can think of this as the length of

the longest path through a ﬂow chart that describes how to compute each of the

model’s outputs given its inputs. Just as two equivalent computer programs will

have diﬀerent lengths depending on which language the program is written in, the

same function may be drawn as a ﬂow chart with diﬀerent depths depending on

which functions we allow to be used as individual steps in the ﬂow chart. Fig. 1.3

illustrates how this choice of language can give two diﬀerent measurements for

the same architecture.

Another approach, used by deep probabilistic models, illustrates not the depth

of the computational graph but the depth of the graph describing how concepts are

related to each other. In this case, the depth of the ﬂow-chart of the computations

needed to compute the representation of each concept may be much deeper than

the graph of the concepts themselves. This is because the system’s understanding

of the simpler concepts can be reﬁned given information about the more complex

concepts. For example, an AI system observing an image of a face with one eye in

CHAPTER 1. INTRODUCTION

shadow may initially only see one eye. After detecting that a face is present, it can

then infer that a second eye is probably present as well. In this case, the graph of

concepts only includes two layers—a layer for eyes and a layer for faces—but the

graph of computations includes 2n layers if we reﬁne our estimate of each concept

given the other n times.

Because it is not always clear which of these two views—the depth of the

computational graph, or the depth of the probabilistic modeling graph—is most

relevant, and because diﬀerent people choose diﬀerent sets of smallest elements

from which to construct their graphs, there is no single correct value for the depth

of an architecture, just as there is no single correct value for length of a computer

program. Nor is there a consensus about how much depth a model requires to

qualify as “deep.” However, deep learning can safely be regarded as the study of

models that either involve a greater amount of composition of learned functions

or learned concepts than traditional machine learning does.

To summarize, deep learning, the subject of this book, is an approach to AI.

Speciﬁcally, it is a type of machine learning, a technique that allows computer

systems to improve with experience and data. According to the authors of this

book, machine learning is the only viable approach to building AI systems that can

operate in complicated, real-world environments. Deep learning is a particular

kind of machine learning that achieves great power and ﬂexibility by learning

to represent the world as a nested hierarchy of concepts and representations,

with each concept deﬁned in relation to simpler concepts, and more abstract

representations computed in terms of less abstract ones. Fig. 1.4 illustrates the

relationship between these diﬀerent AI disciplines. Fig. 1.5 gives a high-level

schematic of how each works.

1.1 Who Should Read This Book?

This book can be useful for a variety of readers, but we wrote it with two main

target audiences in mind. One of these target audiences is university students (un-

dergraduate or graduate) learning about machine learning, including those who

are beginning a career in deep learning and artiﬁcial intelligence research. The

other target audience is software engineers who do not have a machine learning or

statistics background, but want to rapidly acquire one and begin using deep learn-

ing in their product or platform. Software engineers working in a wide variety of

industries are likely to ﬁnd deep learning to be useful, as it has already proven

successful in many areas including computer vision, speech and audio processing,

natural language processing, robotics, bioinformatics and chemistry, video games,

search engines, online advertising and ﬁnance.

This book has been organized into three parts in order to best accommodate

剩余642页未读，继续阅读

whomwhomwhom

粉丝: 5
资源: 50

深度学习基础与应用探索

Deep Learning深度学习(Bengio等著)中英文

Deep Learning电子书

Deep.Learning

deep learning toolbox 安装

Deeplearning4j 视频教程有吗？

how to learn deep learning

下载deep learning toolbox

怎么安装deep learning toolbox

window安装deeplearning4j

Java 安装deeplearning4j

最新资源