终身学习机器系列第二版：策略投票与智能决策探讨

需积分: 10 37 浏览量更新于2024-07-18 收藏 2.69MB PDF 举报

《终身机器学习系列第二版》是由Zhiyuan Chen和Bing Liu共同编著，Morgan & Claypool出版社出版的一本专业书籍，版权由该出版社于2018年持有。本书属于"合成讲座系列：人工智能与机器学习"的一部分，系列编辑包括Ronald J. Brachman（来自Jacobs Technion-Cornell Institute at Cornell Tech）和Peter Stone（来自University of Texas at Austin）。此书的国际标准书号(ISBN)分别为纸质版9781681733029、电子版9781681733036以及不同格式的版本。该系列旨在探讨和推进人工智能和机器学习领域的深度知识，强调的是"终身学习"的概念，即机器如何在不断变化的环境中持续学习和适应。《终身机器学习》第二版可能涵盖了多种子主题，如战略投票的理论（Reshef Meir, 2018）、预测人类决策行为从预测到行动的方法（Ariel Rosenfeld和Sarit Kraus, 2018）、数据科学中的博弈论应用（Boi Faltings和Goran Radanovic, 2017）、多目标决策制定技术（Diederik M. Roijers和Shimon Whiteson, 2017），以及统计关系人工智能中的逻辑、概率和计算（Luc De Raedt等, 2016）。书中还可能讨论了如何处理和推理定性偏好（Ganesh Ram Santhanam等人, 2016），以及metric learning（度量学习）这一关键的机器学习技术，由Aurélien Bellet等人深入阐述，这涉及到如何通过数学模型和算法来理解和优化数据中的距离或相似度度量，这对于许多机器学习任务，如图像识别、自然语言处理和推荐系统至关重要。《终身机器学习系列第二版》不仅提供理论基础，还结合了最新的研究成果，将理论与实践相结合，为读者展示了如何在不断发展的AI领域中实现持续的学习和进步。无论是研究者还是从业者，都能从中受益匪浅，获取关于如何构建智能系统以适应复杂环境的宝贵洞见。

xiv

Lifelong Information Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

7.1 NELL: A Never-Ending Language Learner . . . . . . . . . . . . . . . . . . . . . . . . . 111

7.1.1 NELL Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

7.1.2 Extractors and Learning in NELL . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

7.1.3 Coupling Constraints in NELL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

7.2 Lifelong Opinion Target Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

7.2.1 Lifelong Learning through Recommendation . . . . . . . . . . . . . . . . . . . 118

7.2.2 AER Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

7.2.3 Knowledge Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

7.2.4 Recommendation using Past Knowledge . . . . . . . . . . . . . . . . . . . . . . . 121

7.3 Learning on the Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

7.3.1 Conditional Random Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

7.3.2 General Dependency Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

7.3.3 e L-CRF Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

7.4 Lifelong-RL: Lifelong Relaxation Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . 127

7.4.1 Relaxation Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

7.4.2 Lifelong Relaxation Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

7.5 Summary and Evaluation Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Continuous Knowledge Learning in Chatbots . . . . . . . . . . . . . . . . . . . . . . . . . 131

8.1 LiLi: Lifelong Interactive Learning and Inference . . . . . . . . . . . . . . . . . . . . . 132

8.2 Basic Ideas of LiLi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

8.3 Components of LiLi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

8.4 A Running Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

8.5 Summary and Evaluation Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Lifelong Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

9.1 Lifelong Reinforcement Learning through Multiple Environments . . . . . . . 141

9.1.1 Acquiring and Incorporating Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

9.2 Hierarchical Bayesian Lifelong Reinforcement Learning . . . . . . . . . . . . . . . . 142

9.2.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

9.2.2 Hierarchical Bayesian Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

9.2.3 MTRL Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

9.2.4 Updating Hierarchical Model Parameters . . . . . . . . . . . . . . . . . . . . . . 144

9.2.5 Sampling an MDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

9.3 PG-ELLA: Lifelong Policy Gradient Reinforcement Learning . . . . . . . . . . 146

xvii

Preface

e purpose of writing this second edition is to extend the deﬁnition of lifelong learning, to

update the content of several chapters, and to add a new chapter about continual learning in

deep neural networks, which has been actively researched for the past two to three years. A few

chapters are also reorganized to make each of them more coherent.

e project of writing this book started with a tutorial on lifelong machine learning that

we gave at the 24th International Joint Conference on Artiﬁcial Intelligence (IJCAI) in 2015.

At that time, we had worked on the topic for a while and published several papers in ICML,

KDD, and ACL. When Morgan & Claypool Publishers contacted us about the possibility of

developing a book on the topic, we were excited. We strongly believe that lifelong machine

learning (or simply lifelong learning) is very important for the future of machine learning and

artiﬁcial intelligence (AI). Note that lifelong learning is sometimes also called continual learning

or continuous learning in the literature. Our original research interest in the topic stemmed from

extensive application experiences in sentiment analysis (SA) in a start-up company several years

ago. A typical SA project starts with a client who is interested in consumer opinions expressed

in social media about their products or services and those of their competitors. ere are two

main analysis tasks that an SA system needs to do: (1) discover the entities (e.g., iPhone) and

entity attributes/features (e.g., battery life) that people talked about in opinion documents such

as online reviews and (2) determine whether the opinion about each entity or entity attribute is

positive, negative, or neutral [Liu, 2012, 2015]. For example, from the sentence “iPhone is really

cool, but its battery life sucks,

” an SA system should discover that the author is (1) positive about

iPhone and (2) negative about iPhone’s battery life.

After working on many projects in many domains (which are types of products or services)

for clients, we realized that there is a great deal of sharing of information across domains and

projects. As we see more and more, new things get fewer and fewer. It is easy to see that sentiment

words and expressions (such as good, bad, poor, terrible, and cost an arm and a leg) are shared

across domains. ere is also a great deal of sharing of entities and attributes. For example,

every product has the attribute of price, most electronic products have battery, and many of them

also have screen. It is silly not to exploit such sharing to signiﬁcantly improve SA to make it much

more accurate than without using such sharing but only working on each project and its data

in isolation. e classic machine learning paradigm learns exactly in isolation. Given a dataset,

a learning algorithm runs on the data to produce a model. e algorithm has no memory and

thus is unable to use the previously learned knowledge. In order to exploit knowledge sharing,

an SA system has to retain and accumulate the knowledge learned in the past and use it to help

future learning and problem solving, which is exactly what lifelong learning aims to do.

xviii PREFACE

It is not hard to imagine that this sharing of information or knowledge across domains

and tasks is generally true in every ﬁeld. It is particularly obvious in natural language processing

because the meanings of words and phrases are basically the same across domains and tasks and

so is the sentence syntax. No matter what subject matter we talk about, we use the same language,

although each subject may use only a small subset of the words and phrases in a language. If that

is not the case, it is doubtful that a natural language would have ever been developed by humans.

us, lifelong learning is generally applicable, not just restricted to sentiment analysis.

e goal of this book is to introduce this emerging machine learning paradigm and to

present a comprehensive survey and review of the important research results and latest ideas in

the area. We also want to propose a uniﬁed framework for the research area. Currently, there

are several research topics in machine learning that are closely related to lifelong learning, most

notably, multi-task learning and transfer learning, because they also employ the idea of knowl-

edge sharing and transfer. is book brings all these topics under one roof and discusses their

similarities and diﬀerences. We see lifelong learning as an extension to these related paradigms.

rough this book, we would also like to motivate and encourage researchers to work on life-

long learning. We believe it represents a major research direction for both machine learning and

artiﬁcial intelligence for years to come. Without the capability of retaining and accumulating

knowledge learned in the past, making inferences about it, and using the knowledge to help

future learning and problem solving, achieving artiﬁcial general intelligence (AGI) is unlikely.

Two main principles have guided the writing of this book. First, it should contain strong

motivations for conducting research in lifelong learning in order to encourage graduate students

and researchers to work on lifelong learning problems. Second, the writing should be accessible

to practitioners and upper-level undergraduate students who have basic knowledge of machine

learning and data mining. Yet there should be suﬃcient in-depth materials for graduate students

who plan to pursue Ph.D. degrees in the machine learning and/or data mining ﬁelds.

is book is thus suitable for students, researchers, and practitioners who are interested in

machine learning, data mining, natural language processing, or pattern recognition. Lecturers

can readily use the book in class for courses in any of these related ﬁelds.

Zhiyuan Chen and Bing Liu

August 2018

剩余208页未读，继续阅读

Jesse-Jiang

粉丝: 1
资源: 30

终身学习机器系列第二版：策略投票与智能决策探讨

Python Machine Learning (2ed) 原版电子书+配套代码

04-图片篇.pptx

Lifelong Disk Failure Prediction via GAN-based Anomaly Detection.pdf

推荐几篇增量学习领域比较优秀的文献

给出自主机器学习的参考文献

Write an essay commenting on the saying “Learning is a daily experience and a lifetime mission.”

SwimTransformer的原文

多任务学习有什么新算法吗

The OpenLORIS-scene datasets for lifelong SLAM讲的什么

Which industries will see big growth in the next 10 years

最新资源