视觉注意力的计算视角：从历史到模型

图像处理

视觉注意

需积分: 9 69 浏览量更新于2024-07-18 收藏 3.65MB PDF 举报

身份认证购VIP最低享 7 折!

30元优惠券

《视觉注意力的计算视角》是一本由约翰·K·托索斯所著的专业书籍，出版于麻省理工学院出版社，探讨了视觉注意在计算机视觉、生物视觉以及人工智能领域的关键作用。作者在本科时期被视觉注意力这一主题深深吸引，尤其是1971年两篇论文，一篇关于计算机模式识别，特别是光学字符识别的算法设计，展示了如何通过模拟人类视觉过程来让计算机“看”；另一篇则关注眼动在视觉感知中的角色及其对内部视觉表征构建的影响。书中，作者将视觉注意力视为连接机器与生物视觉的关键桥梁，从计算的角度出发，深入剖析了视觉注意在视觉感知和计算机视觉中的数学模型。书中涵盖了以下几个核心知识点： 1. **视觉显著度**：视觉显著性是衡量一个物体在复杂场景中吸引我们注意的能力，它是理解视觉注意机制的基础。作者探讨了如何通过算法设计来模拟人类对不同特征（如颜色、形状、纹理等）的敏感性，以及这些特征如何影响视觉系统的搜索策略。 2. **视觉模型**：书中详述了基于数学的视觉模型，包括可能的神经网络架构和信息处理模型，这些模型试图模仿人脑中处理视觉信息的方式，以解决诸如目标检测、物体识别和场景理解等问题。 3. **计算机视觉中的视觉注意力**：作者讨论了如何将视觉注意应用于计算机视觉任务，如实时图像分析、视频监控和自动驾驶等领域，通过动态分配计算资源，提高系统在复杂环境中的性能和效率。 4. **生物视觉与计算视觉的比较**：书中的内容还涉及了生物视觉科学的最新进展，对比生物眼动和大脑视觉皮层的工作原理，以此为灵感改进计算视觉算法，推动人工智能的发展。 5. **理论与应用的结合**：作者不仅限于理论探讨，还分享了实际应用案例，展示如何将计算视角的视觉注意力理论转化为可行的技术解决方案，尤其是在人工智能和机器人技术中的应用。 6. **献词意义**：这本书是对作者孩子的一份特别礼物，反映了他对视觉注意研究的持久热情，以及希望通过研究成果传递给下一代对这个领域的深入理解和探索。《视觉注意力的计算视角》提供了深入理解视觉注意的计算框架，对于那些在计算机视觉、人工智能和认知科学领域工作的研究人员和工程师来说，是一本极具价值的参考文献。它揭示了视觉注意在跨学科研究中的核心地位，并为未来的研究和发展指明了方向。

资源详情

资源推荐

The theory presented in this volume was only possible because of the terriﬁ cally

good fortune of having many talented people around me all of who contributed to

the overall body of work (listed alphabetically). I thank each and every one:

•

superb postdoctoral fellows — Neil Bruce, Xin Chen, Florin Cutzu, Daniel Loach,

Julio Martinez-Trujillo, Pietro Parodi, Marc Pomplun, and Michael Tombu;

•

gifted graduate students and research assistants — Neil Bruce, Sean Culhane,

David Dolson, Brian Down, Yuhzong Lai, Randi Lanciwicki, Yueju Liu, Fernando

Nuﬂ o, Antonio Rodriguez-Sanchez, Albert Rothenstein, Xun Shi, Ksenia Shubina,

Eugene Simine, Ankur Vijh, Winky Wai, David Wilkes, Yiming Ye, Andrei

Zaharescu, and Kunhao Zhou;

•

talented and generous colleagues and collaborators, some are coauthors, others

may have provided a key conversation or pointer at the right time — Ruzena Bajcsy,

Nico Boehler, Jochen Braun, Doug Cheyne, Sven Dickinson, Gregory Dudek, Mazyar

Fallah, Olivier Faugeras, Fred Hamker, Hans-Joachim Heinze, Jens-Max Hopf,

Laurent Itti, Michael Jenkin, Allan Jepson, Pierre Kornprobst, Evangelos Milios, John

Mylopoulos (my ﬁ rst and continuing mentor), Zenon Pylyshyn, John Reynolds, Ariel

Schoenfeld, Iannis Tourlakis, Anne Treisman, Stefan Treue, Jeremy Wolfe, Richard

Wildes, Hugh Wilson, and the one who by far contributed the most, Steven Zucker.

I especially thank Eugene Simine for his programming assistance and for preparing

many of the ﬁ gures.

I also need to thank those who took the time to read drafts of this book and

give me feedback and suggestions — Alexander Andreopoulos, Nico Boehler, Neil

Bruce, Konstantinos Derpanis, Jens-Max Hopf, Marc Pomplun, Albert Rothenstein,

Ehsan Fazl-Ersi, Lia Tsotsos, and Konstantine Tsotsos — and three anonymous ref-

erees arranged by The MIT Press. The presentation is much better as a result, but

I take full ownership of any remaining errors or problems. Yes, my children did in

fact read this and provided terriﬁ c feedback. Lia, being a PhD candidate in visual

neuroscience, and Konstantine, being a senior engineering undergraduate who has

Acknowledgments

xvi Acknowledgments

worked on visually guided robotics, are exactly the kinds of people for whom the

book is intended and thus were ideal reviewers.

The staff of the MIT Press has been surprisingly easy to work with. I especially

thank Susan Buckley, Katherine Almeida, and freelancer Chris Curioli. Finally, I

thank Robert Prior for his encouragement, easygoing manner, and, above all, for

his patience while I took my snail ’ s pace toward completion. I also thank Sandy

Pentland for starting me on this project long ago when he suggested that my 1990

paper in The Behavioral and Brain Sciences could form the basis for a good book.

I am grateful to the following main sources of funding that have made my work

possible: the Natural Sciences and Engineering Research Council of Canada

(NSERC), the Canadian Institute for Advanced Research, the Ontario Centres of

Excellence (OCE), and the Canada Research Chairs Program, where I hold the

Canada Research Chair in Computational Vision. I also thank my ﬁ rst home base

where this research was initiated, the University of Toronto, for their support and

the wonderful research environment in the Department of Computer Science. I

thank Olivier Faugeras and the France-Canada Research Foundation for providing

me with a few quiet weeks in a lovely setting in Provence, France, to focus on my

writing. Finally, my current home base, York University, has provided the key ele-

ments of research environment and support for the past 10 years while the research

program matured and enabled me to write this book. My department, Computer

Science and Engineering, the Centre for Vision Research, and the ofﬁ ce of the Vice-

President of Research and Innovation (special thanks to V.P. Stan Shapson) have

provided me with terriﬁ c infrastructure, institutional support, funding, and the intel-

lectual playground that allowed me to pursue my research dreams.

As this is my ﬁ rst authored book, I believe the acknowledgments would be incom-

plete if I did not take the opportunity to thank my parents for the sacriﬁ ces of so

many kinds, their love, unwavering support, and constant encouragement that

formed the foundation for my life. My father taught me the meaning of idealism by

his teachings of the ancient Greek ideals and with the romantic poetry he wrote.

He always looked through the way the world really was to the way it should be,

what he hoped it could become. And as John Polanyi said, idealism is the highest

form of reasoning. My mother taught me how to take that idealism and put it into

practice. Hard work, perseverance, single-mindedness, focus, and then when you

think you have worked hard enough, more hard work. Thomas Edison said that

genius is 1% inspiration and 99% perspiration — my mother knew this because I

really perspired!

My family has been very supportive while I worked on this book. They realized

that while I was focused on this writing, I was at my happiest scientiﬁ cally! Espe-

cially to my wife Patty — the one I celebrate with, the one whose shoulder I cry on,

the one with whom I share the trials and joys of raising our beautiful children, my

soul mate for over 30 years now — thank you!

2 Chapter 1

Four decades later, it seemed that little had changed. Spearman (1937) commented

on the diversity of meanings associated with the word:

For the word attention quickly came to be associated . . . with a diversity of meanings that

have the appearance of being more chaotic even than those of the term ‘ intelligence. ’

Almost eleven decades after James, Sutherland (1998) suggested that:

[A]fter many thousands of experiments, we know only marginally more about attention than

about the interior of a black hole.

Taken together, these quotes make the situation seem bleak! The ﬁ eld is full of

controversy, and it seems that a bit more care is required before moving on. A brief

tour through some of the early thinking on the topic helps reveal sources of debate

and key issues. A more detailed treatment can be found in Tsotsos, Itti, and Rees

(2005). The ﬁ rst scientiﬁ c reference to attention, even though its etymology is traced

to ancient Rome, seems to be due to Descartes (1649), who connected attention to

movements of the pineal body that acted on the animal spirit:

Thus when one wishes to arrest one ’ s attention so as to consider one object for a certain

length of time, this volition keeps the gland tilted towards one side during that time.

Keeping with the idea that body organs are involved, Hobbes (1655) believed:

While the sense organs are occupied with one object, they cannot simultaneously be moved

by another so that an image of both arises. There cannot therefore be two images of two

objects but one put together from the action of both.

Leibnitz (1765) ﬁ rst linked attention to consciousness, a possibility that has received

much debate recently, and attributed this to inhibition from competing ideas:

In order for the mind to become conscious of perceived objects, and therefore for the act of

apperception, attention is required.

Hebart (1824) was the ﬁ rst to develop an elaborate algebraic model of attention

using differential calculus and may be considered the ﬁ rst attention modeler. His

general view on attention, however, was still rather simple:

He is said to be attentive, whose mind is so disposed that it can receive an addition to its

ideas: those who do not perceive obvious things are, on the other hand, lacking in

attention.

Since the 1800s, much genius has gone into experimental methods that were hoped

to shed some light on the phenomenon of attention. Helmholtz (1860) believed that

nervous stimulations are perceived directly, never the objects themselves, and there

are mental activities that enable us to form an idea as to the possible causes of the

observed actions on the senses. These activities are instantaneous, unconscious, and

Attention—We All Know What It Is 3

cannot be corrected by the perceiver by better knowledge — he called this uncon-

scious inference , and thus he believed that attention is an unconscious phenomenon.

On the other hand, Panum (1858) believed that attention is an activity entirely

subservient to an observer ’ s conscious will. Attention becomes difﬁ cult to hold once

interest in an object fades. The greater the disparities between the intensities of two

impressions, the harder it is to keep attention on the weaker one. Panum studied

this in the speciﬁ c context of binocular rivalry; but more generally, he observed that

we are able to ‘ see ’ only a certain number of objects simultaneously. He therefore

concluded that it makes sense that the ﬁ eld of view is ﬁ rst ﬁ lled with the strongest

objects. In studying an object, ﬁ rst attention, and then the eye, is directed to those

contours that are seen by indirect vision.

Hamilton (1859) wondered about the span of attention:

The doctrine that the mind can attend to, or be conscious of, only a single object at a time

would in fact involve the conclusion that all comparison and discrimination are impossible.

. . . Suppose that the mind is not limited to the simultaneous consideration of a single object,

a question arises — how many objects can it embrace at once?

His last question is important even today. Brentano (1874) developed act psychol-

ogy, where an act is a mental activity that affects percepts and images rather than

objects. Examples include attending, picking out, laying stress on something, and

similar actions. This was the ﬁ rst discussion of the possibility that a subject ’ s actions

play a dominant role in perception. Metzger (1974) lists aspects of action that con-

tribute to perception: bringing stimuli to receptors, enlarging the ‘ accessible area, ’

foveation (the act of centering the central, highest-resolution part of the retina onto

an object), optimization of the state of receptors, slowing down of fading and local

adaptation, exploratory movement, and ﬁ nally the search for principles of organiza-

tion within visual stimuli.

Wundt (1874) further linked attention and consciousness, suggesting that atten-

tion, as an inner activity, causes ideas to be present in consciousness to differing

degrees. The focus of attention can narrow or widen, reﬂ ecting these degrees of

consciousness. For Titchener (1908), attention was an intensive attribute of a con-

scious experience equated with ‘ sensible clearness. ’ He compared attention to a

wave, but with only one peak (corresponding with one ’ s focus). He argued that the

effect of attention is to increase clarity, whereas Kulpe (1902) suggested that atten-

tion enhanced not clarity but discriminability. Petermann (1929) argued against the

subject being a passive perceiver of stimuli. He proposed an attention-direction

theor y , based on actions, as the mechanism for an active attentive process. As will

become apparent, this theme keeps reappearing. These and other ideas were never

formalized in any way and remained conceptual, yet interesting, viewpoints on the

issue.

剩余332页未读，继续阅读

haz2yy

粉丝: 5
资源: 24

视觉注意力的计算视角：从历史到模型

Prime Numbers A Computational Perspective 2rd

Cyber-Physical Systems A Computational Perspective

ProbSparse self-attention

Local-to-Global Self-Attention in Vision Transformers

window attention blocks

LSTM-Attention

Efficient Video Recommendation with Multi-Head Self-Attention and Hybrid Sampling

a computational investigation into the human representation and processing o

Computational Inelasticity (J.C. Simo, T.J.R. Hughes)

注意力机制参考文献引用

the early second paper about Optical computational imaging

matlab fvm,The Finite Volume Method in Computational Fluid Dynamics: An Advanced Introduction with O...

history key frames (history submap for loop closure)

Visual Molecular Dynamics

an iterative solver was used for this model. however, a direct solver may en

:Computational Linguistics参考文献overleaf模板

computational science and engineering pdf

Bayesian Color Constancy Revisited

android transfer learning

最新资源