局部自相似性在图像与视频配准中的应用

5星 · 超过95%的资源需积分: 50 114 浏览量更新于2024-09-07 1 收藏 2.73MB PDF 举报

“Local Self-Similarities 图像配准 - 使用局部自相似描述子进行图像匹配与视频分析的先进方法” 在计算机视觉领域，图像配准是一项关键任务，它涉及到将两个或多个图像对齐，以便比较、融合或分析它们的内容。局部自相似（LSS）是一种强大的理论框架，用于描述图像的局部特性，并在图像配准中发挥重要作用。以下是关于“Local Self-Similarities 图像配准”的详细解释： 1. **局部自相似描述子**：局部自相似描述子关注的是图像的局部区域而不是整体，这样可以处理具有复杂背景、光照变化或局部变形的图像。这种描述子能够捕获图像中重复出现的模式或结构，即使这些模式在不同位置或不同图像中存在差异。 2. **对数极坐标系统与局部仿射变形**：使用对数极坐标系统来表示图像，可以有效地处理局部的仿射变形。这种方法允许描述子在保持鲁棒性的同时，适应图像的局部形状变化。 3. **选择最大值策略**：在描述子的构建过程中，通常选取每个bin中的最大值作为bin的代表值。这样做可以使得描述子对匹配位置的小幅度偏移具有一定的鲁棒性，从而提高配准的准确性。 4. **使用patch而非单一像素**：相较于仅考虑单个像素，使用patch（小区域）可以获取图像的更多上下文信息，包括颜色、纹理和边缘等特征。这有助于更好地识别和理解图像内容，提高配准的精度。 5. **丰富的信息内容**： LSS描述子不仅包含了图像的结构骨架，还包含了大量的颜色、边缘和其他视觉特征。这使得它能够处理无明显边界纹理物体的配准问题，以及在复杂背景中检测对象。 6. **多尺度分析**：通过在不同尺度上测量局部自相似性，算法能够处理不同大小的物体和不同级别的细节，适应不同场景的变化。 7. **复杂视觉数据处理**：基于LSS的方法能够处理复杂的视觉数据，如在实际杂乱图像中使用粗略手绘草图检测对象，处理无明确边界的纹理物体，以及在没有先验学习的情况下检测复杂动作的视频序列。 8. **与其他度量比较**：文献中提到将LSS度量与现有的图像相似性度量进行比较，展示了其在处理各种挑战性任务时的优越性能。总结来说，“Local Self-Similarities 图像配准”是一种创新的图像分析技术，利用图像的局部自相似性来进行精确的配准，适用于处理各种复杂情况下的图像和视频数据。这种方法不仅提升了图像配准的准确性和鲁棒性，还在处理现实世界中的视觉挑战方面展现出巨大的潜力。

Matching Local Self-Similarities across Images and Videos

Eli Shechtman Michal Irani

Dept. of Computer Science and Applied Math

The Weizmann Institute of Science

76100 Rehovot, Israel

Abstract

We present an approach for measuring similarity be-

tween visual entities (images or videos) based on match-

ing internal self-similarities. What is correlated across

images (or across video sequences) is the internal lay-

out of local self-similarities (up to some distortions), even

though the patterns generating those local self-similarities

are quite different in each of the images/videos. These in-

ternal self-similarities are efﬁciently captured by a com-

pact local “self-similarity descriptor”, measured densely

throughout the image/video, at multiple scales, while ac-

counting for local and global geometric distortions. This

gives rise to matching capabilities of complex visual data,

including detection of objects in real cluttered images using

only rough hand-sketches, handling textured objects with

no clear boundaries, and detecting complex actions in clut-

tered video data with no prior learning. We compare our

measure to commonly used image-based and video-based

similarity measures, and demonstrate its applicability to ob-

ject detection, retrieval, and action detection.

1. Introduction

Determining similarity between visual data is necessary

in many computer vision tasks, including object detection

and recognition, action recognition, texture classiﬁcation,

data retrieval, tracking, image alignment, etc. Methods for

performing these tasks are usually based on representing

an image using some global or local image properties, and

comparing them using some similarity measure.

The relevant representations and the corresponding sim-

ilarity measures can vary signiﬁcantly. Images are of-

ten represented using dense photometric pixel-based prop-

erties or by compact region descriptors (features) often

used with interest point detectors. Dense properties in-

clude raw pixel intensity or color values (of the entire im-

age, of small patches [25, 3] or fragments [22]), texture

ﬁlters [15] or other ﬁlter responses [18]. Common com-

pact region descriptors include distribution based descrip-

tors (e.g., SIFT [13]), differential descriptors (e.g., local

derivatives [12]), shape-based descriptors using extracted

edges (e.g. Shape Context [1]), and others. For a compre-

hensive comparison of many region descriptors for image

matching see [16].

Figure 1. These images of the same object (a heart) do NOT share

common image properties (colors, textures, edges), but DO share

a similar geometric layout of local internal self-similarities.

Although these representations and their corresponding

measures vary signiﬁcantly, they all share the same basic

assumption – that there exists a common underlying visual

property (i.e., pixels colors, intensities, edges, gradients or

other ﬁlter responses) which is shared by the two images (or

sequences), and can therefore be extracted and compared

across images/sequences. This assumption, however, may

be too restrictive, as illustrated in Fig. 1. There is no ob-

vious image property shared between those images. Nev-

ertheless, we can clearly notice that these are instances of

the same object (a heart). What makes these images similar

is the fact that their local intensity patterns (in each image)

are repeated in nearby image locations in a similar relative

geometric layout. In other words, the local internal lay-

outs of self-similarities are shared by these images, even

though the patterns generating those self-similarities are

not shared by those images. The notion of self similarity

in video sequences is even stronger than in images. E.g.,

people wear the same clothes in consecutive frames and

backgrounds tend to change gradually, resulting in strong

self-similar patterns in local space-time video regions.

In this paper we present a “local self-similarity descrip-

tor” which captures internal geometric layouts of local

self-similarities within images/videos, while accounting for

small local afﬁne deformations. It captures self-similarity

of color, edges, repetitive patterns (e.g., the right image in

Fig. 1) and complex textures in a single uniﬁed way. A tex-

tured region in one image can be matched with a uniformly

colored region in the other image as long as they have a

similar spatial layout. These self-similarity descriptors are

estimated on a dense grid of points in image/video data, at

multiple scales. A good match between a pair of images (or

a pair of video sequences), corresponds to ﬁnding a match-

ing ensemble of such descriptors – with similar descriptor

values at similar relative geometric positions, up to small

non-rigid deformations. This allows to match a wide vari-

下载后可阅读完整内容，剩余7页未读，立即下载

李伯爵的指间沙

粉丝: 154
资源: 3

局部自相似性在图像与视频配准中的应用

Local-Self-Similarities

local self-similarity descriptor matlab code

Exploiting self-similarities for single frame superresolution

Free Viewpoint Action Recognition based on Self-similarities

java-similarities:从 code.google.compjava-similarities 自动导出

Algorithm-laravel-string-similarities.zip

laravel-string-similarities：比较两个字符串并获得相似百分比

RS-422 vs RS-485 Similarities and Key Differences-综合文档

Structural-Context Similarities for Uncertain Graphs

Images similarities searcher:首先按内容对图像进行索引，然后在索引中搜索相似性-开源

最新资源