机器学习：贝叶斯与优化视角

5星 · 超过95%的资源需积分: 50 90 浏览量更新于2024-07-18 2 收藏 33.65MB PDF 举报

"Machine Learning: A Bayesian and Optimization Perspective" 是一本深入探讨贝叶斯机器学习及其优化方法的专业书籍，适合对这个领域有浓厚兴趣的读者，尤其是英语水平较高的学习者。贝叶斯机器学习是统计学与人工智能的一个重要分支，它以贝叶斯定理为基础，用于构建和理解复杂的概率模型。在本书中，作者Sergios Theodoridis将带领读者深入了解这一领域的核心概念。贝叶斯方法特别适用于处理不确定性问题，通过先验知识和观测数据相结合，形成后验概率分布，使得模型能够不断更新和改进。这种方法在许多领域如模式识别、数据挖掘、推荐系统、自然语言处理等有着广泛应用。优化方法是机器学习中的另一关键主题，特别是在训练模型时寻找最佳参数配置。书中可能涵盖了梯度下降、牛顿法、随机梯度下降等优化算法，以及如何解决局部最小值和全局最优解的问题。优化技术不仅用于监督学习中的参数调整，还涉及到无监督学习和半监督学习中的复杂结构发现。此外，书中可能会讨论贝叶斯网络、马尔科夫随机场、高斯过程等概率图模型，这些模型提供了表示和推理不确定性的有效工具。同时，贝叶斯优化则是一种用于全局优化的策略，尤其适用于黑盒函数，可能也会在书中有所涉及。对于初学者，本书可能从基础理论开始，逐步引入高级主题，提供清晰的解释和实例；对于经验丰富的从业者，书中的深入分析和最新研究成果可能为他们的研究提供新的视角和灵感。无论你是学生还是研究人员，这本书都将帮助你深化对贝叶斯方法和优化技术的理解，从而更好地应用于实际的机器学习项目。 "Machine Learning: A Bayesian and Optimization Perspective" 是一本全面而深入的教程，它将帮助读者掌握贝叶斯统计和机器学习优化的核心思想，提升在数据分析和预测模型构建方面的能力。通过阅读这本书，你可以期待获得关于贝叶斯推断、模型选择、模型验证和复杂系统优化的宝贵知识。

Contents xv

18.8 Learning Deep Networks........................................................... 903

18.8.1 The Need for Deep Architectures......................................... 904

18.8.2 Training Deep Networks .................................................. 905

18.8.3 Training Restricted Boltzmann Machines ................................ 908

18.8.4 Training Deep Feed-Forward Networks .................................. 914

18.9 Deep Belief Networks .............................................................. 916

18.10 Variations on the Deep Learning Theme .......................................... 918

18.10.1Gaussian Units ............................................................. 918

18.10.2Stacked Autoencoders ..................................................... 919

18.10.3The Conditional RBM..................................................... 920

18.11 Case Study: A Deep Network for Optical Character

Recognition ......................................................................... 923

18.12 Case Study: A Deep Autoencoder ................................................. 925

18.13 Example: Generating Data via aDBN............................................. 928

Problems .................................................................................... 929

References................................................................................... 932

CHAPTER 19 Dimensionality Reduction .................................................... 937

19.1 Introduction ......................................................................... 938

19.2 Intrinsic Dimensionality ............................................................ 939

19.3 Principle Component Analysis ..................................................... 939

19.4 Canonical Correlation Analysis .................................................... 950

19.4.1 Relatives of CCA .......................................................... 953

19.5 Independent Component Analysis ................................................. 955

19.5.1 ICA and Gaussianity....................................................... 956

19.5.2 ICA and Higher Order Cumulants ........................................ 957

19.5.3 Non-Gaussianity and Independent

Components ................................................................ 958

19.5.4 ICA Based on Mutual Information........................................ 959

19.5.5 Alternative Paths to ICA................................................... 962

19.6 Dictionary Learning: The k-SVD Algorithm...................................... 966

19.7 Nonnegative Matrix Factorization ................................................. 971

19.8 Learning Low-Dimensional Models: A Probabilistic

Perspective .......................................................................... 972

19.8.1 Factor Analysis ............................................................ 972

19.8.2 Probabilistic PCA .......................................................... 974

19.8.3 Mixture of Factors Analyzers: A Bayesian View

to Compressed Sensing.................................................... 977

19.9 Nonlinear Dimensionality Reduction .............................................. 980

19.9.1 Kernel PCA ................................................................ 980

19.9.2 Graph-Based Methods ..................................................... 982

xvi Contents

19.10 Low-Rank Matrix Factorization: ASparse Modeling Path....................... 991

19.10.1Matrix Completion......................................................... 991

19.10.2RobustPCA................................................................ 995

19.10.3Applications of Matrix Completionand ROBUST PCA ................ 996

19.11 A Case Study: fMRI Data Analysis ................................................ 998

Problems .................................................................................... 1002

References................................................................................... 1003

APPENDIX A Linear Algebra ................................................................... 1013

A.1 Propertiesof Matrices .............................................................. 1013

A.2 Positive Denite and Symmetric Matrices ........................................ 1015

A.3 Wirtinger Calculus.................................................................. 1016

References................................................................................ 1017

APPENDIX B Probability Theory and Statistics .......................................... 1019

B.1 Cramér-Rao Bound ................................................................. 1019

B.2 Characteristic Functions............................................................ 1020

B.3 Moments and Cumulants ........................................................... 1020

B.4 Edgeworth Expansion of a Pdf..................................................... 1021

Reference................................................................................. 1022

APPENDIX C Hints on Constrained Optimization ........................................ 1023

C.1 Equality Constraints ................................................................ 1023

C.2 Inequality Constraints .............................................................. 1025

References................................................................................ 1029

Index ..................................................................................................... 1031

Preface

Machine Learning is a name that is gaining popularity as an umbrella for methods that have been studied

and developed for many decades in different scientic communities and under different names, such as

Statistical Learning, Statistical Signal Processing, Pattern Recognition, Adaptive Signal Processing,

Image Processing and Analysis, System Identication and Control, Data Mining and Information

Retrieval, Computer Vision, and Computational Learning. The name “Machine Learning” indicates

what all these disciplines have in common, that is, to learn from data,andthenmake predictions.

What one tries to learn from data is their underlying structure and regularities, via the development of

a model, which can then be used to provide predictions.

To this end, a number of diverse approaches have been developed, ranging from optimization of cost

functions, whose goal is to optimize the deviation between what one observes from data and what the

model predicts, to probabilistic models that attempt to model the statistical properties of the observed

data.

The goal of this book is to approach the machine learning discipline in a unifying context,

by presenting the major paths and approaches that have been followed over the years, without giving

preference to a specic one. It is the author’s belief that all of them are valuable to the newcomer who

wants to learn the secrets of this topic, from the applications as well as from the pedagogic point of

view. As the title of the book indicates, the emphasis is on the processing and analysis front of machine

learning and not on topics concerning the theory of learning itself and related performance bounds.

In other words, the focus is on methods and algorithms closer to the application level.

The book is the outgrowth of more than three decades of the author’s experience on research and

teaching various related courses. The book is written in such a way that individual (or pairs of) chapters

are as self-contained as possible. So, one can select and combine chapters according to the focus he/she

wants to give to the course he/she teaches, or to the topics he/she wants to grasp in a rst reading. Some

guidelines on how one can use the book for different courses are provided in the introductory chapter.

Each chapter grows by starting from the basics and evolving to embrace the more recent advances.

Some of the topics had to be split into two chapters, such as sparsity-aware learning, Bayesian learning,

probabilistic graphical models, and Monte Carlo methods. The book addresses the needs of advanced

graduate, postgraduate, and research students as well as of practicing scientists and engineers whose

interests lie beyond black-box solutions. Also, the book can serve the needs of short courses on specic

topics, e.g., sparse modeling, Bayesian learning, probabilistic graphical models, neural networks and

deep learning.

Most of the chapters include Matlab exercises, and the related code is available from the book’s

website. The solutions manual as well as PowerPoint lectures are also available from the book’s website.

xvii

Acknowledgments

Writing a book is an effort on top of everything else that must keep running in parallel. Thus, writing

is basically an early morning, after ve, and over the weekends and holidays activity. It is a big effort

that requires dedication and persistence. This would not be possible without the support of a number of

people—people who helped in the simulations, in the making of the gures, in reading chapters, and

in discussing various issues concerning all aspects, from proofs to the structure and the layout of the

book.

First, I would like to express my gratitude to my mentor, friend, and colleague Nicholas Kalouptsidis,

for this long-lasting and fruitful collaboration.

The cooperation with Kostas Slavakis over the last six years has been a major source of inspiration

and learning and has played a decisive role for me in writing this book.

I am indebted to the members of my group, and in particular to Yannis Kopsinis, Pantelis Bouboulis,

Simos Chouvardas, Kostas Themelis, George Papageorgiou, and Charis Georgiou. They were beside

me the whole time, especially during the difcult nal stages of the completion of the manuscript.

My colleagues Aggelos Pikrakis, Kostas Koutroumbas, Dimitris Kosmopoulos, George Giannakopou-

los, and Spyros Evaggelatos gave a lot of their time for discussions, helping in the simulations, and

reading chapters.

Without my two sabbaticals during the spring semesters of 2011 and 2012, I doubt I would have

ever nished this book. Special thanks to all my colleagues in the Department of Informatics and

Telecommunications of the National and Kapodistrian University of Athens.

During my sabbatical in 2011, I was honored to be a holder of an Excellence Chair in Carlos III

University of Madrid and spent the time with the group of Anibal Figuieras-Vidal. I am indebted to

Anibal for his invitation and all the fruitful discussions and the bottles of excellent red Spanish wine

we had together. Special thanks to Jerónimo Arenas-García and Antonio Artés-Rodríguez, who have

also introduced me to aspects of traditional Spanish culture.

During my sabbatical in 2012, I was also honored to be an Otto Mønsted Guest Professor at

the Technical University of Denmark with the group of Lars Kai Hansen. I am indebted to him for

the invitation and our enjoyable and insightful discussions, as well as his constructive comments on

reviewing chapters of the book and for the visits to the Danish museums on weekends. Also, special

thanks to Jan Larsen and Morten Mørup for the fruitful discussions.

A number of colleagues were kind enough to read and review chapters and parts of the book and

come back with valuable comments and criticisms. My sincere thanks to Tulay Adali, Kostas Berberidis,

Jim Bezdek, Gustavo Camps-Valls, Taylan Cemgil and his students, Petar Djuric, Paulo Diniz, Yannis

Emiris, Georgios Giannakis, Mark Girolami, Dimitris Gunopoulos, Alexandros Katsioris, Evaggelos

Karkaletsis, Dimitris Katselis, Athanasios Liavas, Eleftherios Kodis, Elias Koutsoupias, Alexandros

Makris, Dimitirs Manatakis, Elias Manolakos, Francisco Palmieri, Jean-Christophe Pesquet, Bhaskar

Rao, Ali Sayed, Nicolas Sidiropoulos, Paris Smaragdis, Isao Yamada, and Zhilin Zhang.

Finally, I would like to thank Tim Pitts, the Editor in Academic Press, for all his help.

xix

剩余1071页未读，继续阅读

酷钉

粉丝: 288
资源: 40

机器学习：贝叶斯与优化视角

机器学习 贝叶斯机器学习 贝叶斯

《贝叶斯深度学习原理 》

《大规模机器学习与凸优化》

机器学习贝叶斯方法的应用场景

贝叶斯网络和生成对抗网络有哪些共同点

机器学习朴素贝叶斯调参

机器学习 朴素贝叶斯 numpy

优化算法与机器学习算法的融合

贝叶斯优化算法在生物医学中的应用

机器学习c++贝叶斯文本情绪分析

最新资源

机器学习贝叶斯机器学习贝叶斯

《贝叶斯深度学习原理》

机器学习朴素贝叶斯 numpy