Sergios Theodoridis的机器学习入门经典：贝叶斯与优化视角

需积分: 10 104 浏览量更新于2024-07-20 收藏 27.46MB PDF 举报

《塞尔吉奥·西格里奥斯的机器学习：贝叶斯与优化视角》是一本由Sergios Theodoridis编著的高质量机器学习教材，专为初学者和进阶者设计。本书以Bayesian（贝叶斯）方法为核心，结合优化理论，为读者提供了一个全面而深入的理解机器学习的框架。在书中，作者从基础概念出发，逐步引导读者探索机器学习的基本原理，如概率论、统计推断和模型构建。Bayesian方法是本书的重要组成部分，它强调利用先验知识和观测数据更新信念，为学习过程赋予了更深层次的解释。通过贝叶斯学习，读者将学会如何处理不确定性，并在决策过程中应用概率模型。此外，本书还涵盖了优化技术在机器学习中的关键作用。优化问题在许多机器学习算法中扮演核心角色，包括参数估计、模型选择和特征工程。作者讲解了各种优化算法，如梯度下降、牛顿法以及现代优化方法，帮助读者掌握如何在复杂函数空间中寻找最优解。书中还穿插了许多实际案例和实验，让读者能够在实践中巩固理论知识。这些案例涉及图像识别、自然语言处理、推荐系统等多个领域，使读者能够了解贝叶斯和优化在解决实际问题中的应用效果。版权信息表明，这本书受到严格保护，未经许可，不得以任何形式复制或传播。对于那些对机器学习有兴趣的读者来说，《机器学习：贝叶斯与优化视角》不仅是一本理论教材，也是一本实用工具书，它提供了从入门到进阶的学习路径，适合不同背景的学习者进行深入研究或教学使用。通过阅读这本书，读者将建立起坚实的理论基础，同时掌握在现代信息技术行业中至关重要的技能。

Contents

18.8 Learning Deep Networks........................................................... 903

18.8.1 The Need for Deep Architectures......................................... 904

18.8.2 Training Deep Networks .................................................. 905

18.8.3 Training Restricted Boltzmann Machines ................................ 908

18.8.4 Training Deep Feed-Forward Networks .................................. 914

18.9 Deep Belief Networks .............................................................. 916

18.10 Variations on the Deep Learning Theme .......................................... 918

18.10.1Gaussian Units ............................................................. 918

18.10.2Stacked Autoencoders ..................................................... 919

18.10.3The Conditional RBM..................................................... 920

18.11 Case Study: A Deep Network for Optical Character

Recognition ......................................................................... 923

18.12 Case Study: A Deep Autoencoder ................................................. 925

18.13 Example: Generating Data via aDBN............................................. 928

Problems .................................................................................... 929

References................................................................................... 932

CHAPTER 19 Dimensionality Reduction .................................................... 937

19.1 Introduction ......................................................................... 938

19.2 Intrinsic Dimensionality ............................................................ 939

19.3 Principle Component Analysis ..................................................... 939

19.4 Canonical Correlation Analysis .................................................... 950

19.4.1 Relatives of CCA .......................................................... 953

19.5 Independent Component Analysis ................................................. 955

19.5.1 ICA and Gaussianity....................................................... 956

19.5.2 ICA and Higher Order Cumulants ........................................ 957

19.5.3 Non-Gaussianity and Independent

Components ................................................................ 958

19.5.4 ICA Based on Mutual Information........................................ 959

19.5.5 Alternative Paths to ICA................................................... 962

19.6 Dictionary Learning: The k-SVD Algorithm...................................... 966

19.7 Nonnegative Matrix Factorization ................................................. 971

19.8 Learning Low-Dimensional Models: A Probabilistic

Perspective .......................................................................... 972

19.8.1 Factor Analysis ............................................................ 972

19.8.2 Probabilistic PCA .......................................................... 974

19.8.3 Mixture of Factors Analyzers: A Bayesian View

to Compressed Sensing.................................................... 977

19.9 Nonlinear Dimensionality Reduction .............................................. 980

19.9.1 Kernel PCA ................................................................ 980

19.9.2 Graph-Based Methods ..................................................... 982

xvi

Contents

19.10 Low-Rank Matrix Factorization: ASparse Modeling Path....................... 991

19.10.1 Matrix Completion......................................................... 991

19.10.2 Robust PCA................................................................ 995

19.10.3Applications of Matrix Completion and ROBUST PCA ................ 996

19.11 A Case Study: fMRI Data Analysis ................................................ 998

Problems .................................................................................... 1002

References................................................................................... 1003

APPENDIX A Linear Algebra

................................................................... 1013

A.1 Properties of Matrices .............................................................. 1013

A.2 Positive Denite and Symmetric Matrices ........................................ 1015

A.3 Wirtinger Calculus.................................................................. 1016

References................................................................................ 1017

APPENDIX B Probability Theory and Statistics .......................................... 1019

B.1 Cramér-Rao Bound ................................................................. 1019

B.2 Characteristic Functions............................................................ 1020

B.3 Moments and Cumulants ........................................................... 1020

B.4 Edgeworth Expansion of aPdf..................................................... 1021

Reference................................................................................. 1022

APPENDIX C Hints on Constrained Optimization

........................................ 1023

C.1 Equality Constraints ................................................................ 1023

C.2 Inequality Constraints .............................................................. 1025

References................................................................................ 1029

Index ..................................................................................................... 1031

Preface

Machine Learning is a name that is gaining popularity as an umbrella for methods that have been studied

and developed for many decades in different scientic communities and under different names, such as

Statistical Learning, Statistical Signal Processing, Pattern Recognition, Adaptive Signal Processing,

Image Processing and Analysis, System Identication and Control, Data Mining and Information

Retrieval, Computer Vision, and Computational Learning. The name “Machine Learning” indicates

what all these disciplines have in common, that is, to learn from data,andthenmake predictions.

What one tries to learn from data is their underlying structure and regularities, via the development of

a model, which can then be used to provide predictions.

To this end, a number of diverse approaches have been developed, ranging from optimization of cost

functions, whose goal is to optimize the deviation between what one observes from data and what the

model predicts, to probabilistic models that attempt to model the statistical properties of the observed

data.

The goal of this book is to approach the machine learning discipline in a unifying context,

by presenting the major paths and approaches that have been followed over the years, without giving

preference to a specic one. It is the author’s belief that all of them are valuable to the newcomer who

wants to learn the secrets of this topic, from the applications as well as from the pedagogic point of

view. As the title of the book indicates, the emphasis is on the processing and analysis front of machine

learning and not on topics concerning the theory of learning itself and related performance bounds.

In other words, the focus is on methods and algorithms closer to the application level.

The book is the outgrowth of more than three decades of the author’s experience on research and

teaching various related courses. The book is written in such a way that individual (or pairs of) chapters

are as self-contained as possible. So, one can select and combine chapters according to the focus he/she

wants to give to the course he/she teaches, or to the topics he/she wants to grasp in a rst reading. Some

guidelines on how one can use the book for different courses are provided in the introductory chapter.

Each chapter grows by starting from the basics and evolving to embrace the more recent advances.

Some of the topics had to be split into two chapters, such as sparsity-aware learning, Bayesian learning,

probabilistic graphical models, and Monte Carlo methods. The book addresses the needs of advanced

graduate, postgraduate, and research students as well as of practicing scientists and engineers whose

interests lie beyond black-box solutions. Also, the book can serve the needs of short courses on specic

topics, e.g., sparse modeling, Bayesian learning, probabilistic graphical models, neural networks and

deep learning.

Most of the chapters include Matlab exercises, and the related code is available from the book’s

website. The solutions manual as well as PowerPoint lectures are also available from the book’s website.

xvii

Acknowledgments

Writing a book is an effort on top of everything else that must keep running in parallel. Thus, writing

is basically an early morning, after ve, and over the weekends and holidays activity. It is a big effort

that requires dedication and persistence. This would not be possible without the support of a number of

people—people who helped in the simulations, in the making of the gures, in reading chapters, and

in discussing various issues concerning all aspects, from proofs to the structure and the layout of the

book.

First, I would like to express my gratitude to my mentor, friend, and colleague Nicholas Kalouptsidis,

for this long-lasting and fruitful collaboration.

The cooperation with Kostas Slavakis over the last six years has been a major source of inspiration

and learning and has played a decisive role for me in writing this book.

I am indebted to the members of my group, and in particular to Yannis Kopsinis, Pantelis Bouboulis,

Simos Chouvardas, Kostas Themelis, George Papageorgiou, and Charis Georgiou. They were beside

me the whole time, especially during the difcult nal stages of the completion of the manuscript.

My colleagues Aggelos Pikrakis, Kostas Koutroumbas, Dimitris Kosmopoulos, George Giannakopou-

los, and Spyros Evaggelatos gave a lot of their time for discussions, helping in the simulations, and

reading chapters.

Without my two sabbaticals during the spring semesters of 2011 and 2012, I doubt I would have

ever nished this book. Special thanks to all my colleagues in the Department of Informatics and

Telecommunications of the National and Kapodistrian University of Athens.

During my sabbatical in 2011, I was honored to be a holder of an Excellence Chair in Carlos III

University of Madrid and spent the time with the group of Anibal Figuieras-Vidal. I am indebted to

Anibal for his invitation and all the fruitful discussions and the bottles of excellent red Spanish wine

we had together. Special thanks to Jerónimo Arenas-García and Antonio Artés-Rodríguez, who have

also introduced me to aspects of traditional Spanish culture.

During my sabbatical in 2012, I was also honored to be an Otto Mønsted Guest Professor at

the Technical University of Denmark with the group of Lars Kai Hansen. I am indebted to him for

the invitation and our enjoyable and insightful discussions, as well as his constructive comments on

reviewing chapters of the book and for the visits to the Danish museums on weekends. Also, special

thanks to Jan Larsen and Morten Mørup for the fruitful discussions.

A number of colleagues were kind enough to read and review chapters and parts of the book and

come back with valuable comments and criticisms. My sincere thanks to Tulay Adali, Kostas Berberidis,

Jim Bezdek, Gustavo Camps-Valls, Taylan Cemgil and his students, Petar Djuric, Paulo Diniz, Yannis

Emiris, Georgios Giannakis, Mark Girolami, Dimitris Gunopoulos, Alexandros Katsioris, Evaggelos

Karkaletsis, Dimitris Katselis, Athanasios Liavas, Eleftherios Kodis, Elias Koutsoupias, Alexandros

Makris, Dimitirs Manatakis, Elias Manolakos, Francisco Palmieri, Jean-Christophe Pesquet, Bhaskar

Rao, Ali Sayed, Nicolas Sidiropoulos, Paris Smaragdis, Isao Yamada, and Zhilin Zhang.

Finally, I would like to thank Tim Pitts, the Editor in Academic Press, for all his help.

xix

剩余1071页未读，继续阅读

dolphincc

粉丝: 14
资源: 8

Sergios Theodoridis的机器学习入门经典：贝叶斯与优化视角

《模式识别》第四版源码 Sergios Theodoridis Konstantinos Koutroumbas

模式识别 by Sergios Theodoridis

Machine Learning-A Bayesian and Optimization Perspective.pdf

pattern recognition 4th edition matlab

模式识别第四版英文版-Sergios Theodoridis

笔记本盖子状态检测GPD Pocket 3自动禁用触摸屏

基于Java和C语言的2023年Arduino重拾项目设计源码

OCR识别-支持识别80+种语言的算法项目-包括Latin-Chinese-Arabic等等-附详细完整流程教程.zip

MyBatis 27道面试题和答案（2025最新版）.docx

实验九-MATLAB实验SIMULINK仿真.zip

最新资源