单幅偏振图像复杂场景法向估计

需积分: 0 45 浏览量更新于2024-08-03 1 收藏 9.71MB PDF 举报

"Lei等人在CVPR 2022上发表的论文《Shape From Polarization for Complex Scenes in the Wild》介绍了如何从单张偏振图像中估计复杂场景的表面法线，克服了现有技术在处理复杂野外场景时的局限性。他们创建了首个包含真实世界复杂场景偏振数据的基准数据集，并提出了一种基于学习的框架，该框架利用多头自注意力模块和视角编码来解决由复杂材质和非正交投影引起的偏振模糊问题。实验结果显示，他们的方法能够显著提高场景级偏振形状估计的准确性和鲁棒性，且适用于远距离户外场景。" 这篇论文的核心知识点包括： 1. **形状从偏振（Shape from Polarization, SfP）**：传统的SfP方法主要关注单一对象的法线估计，而不是复杂的自然场景。SfP是一种利用光的偏振特性推断物体表面几何形状的技术。 2. **复杂场景的挑战**：在复杂的野外环境中，由于存在多种材质、光照条件和非正交投影，高精度的场景级SfP面临困难，特别是缺乏真实世界的复杂场景偏振数据。 3. **新数据集**：论文贡献了第一个真实世界的场景级SfP数据集，包含了输入偏振图像和对应的地面实况法线图，为研究提供了基础。 4. **学习框架**：论文提出了一个基于深度学习的方法，其中包含多头自注意力模块和视角编码。这种方法设计用于处理因复杂材料和非正交投影导致的偏振模糊问题。 5. **多头自注意力模块**：自注意力机制是Transformer架构的关键部分，它允许模型同时考虑输入序列的不同位置，以捕捉长程依赖。在SfP任务中，多头注意力可能有助于识别和处理不同材质和角度的信息。 6. **视角编码**：这一技术可能用于表示不同的观察角度，帮助模型理解不同视角下偏振光与表面法线的关系，从而提高估计的准确性。 7. **泛化能力**：经过训练的模型能够应用于远距离户外场景，表明偏振光与表面法线之间的关系不受距离影响，提高了方法的适用范围。 8. **实验结果**：通过实验，作者证明了他们的方法在提高复杂场景的SfP性能方面具有显著优势，增强了对不同光照和环境条件的鲁棒性。这篇研究不仅提供了一个新的数据集，还提出了一个新的处理复杂场景偏振信息的框架，对于理解和利用偏振光在计算机视觉领域的应用具有重要意义，特别是在3D重建和环境理解等方面。

Shape from Polarization for Complex Scenes in the Wild

Chenyang Lei

Chenyang Qi

Jiaxin Xie

Na Fan

Vladlen Koltun

Qifeng Chen

HKUST

Apple

Abstract

We present a new data-driven approach with physics-

based priors to scene-level normal estimation from a single

polarization image. Existing shape from polarization (SfP)

works mainly focus on estimating the normal of a single ob-

ject rather than complex scenes in the wild. A key barrier

to high-quality scene-level SfP is the lack of real-world SfP

data in complex scenes. Hence, we contribute the ﬁrst real-

world scene-level SfP dataset with paired input polarization

images and ground-truth normal maps. Then we propose a

learning-based framework with a multi-head self-attention

module and viewing encoding, which is designed to handle

increasing polarization ambiguities caused by complex ma-

terials and non-orthographic projection in scene-level SfP.

Our trained model can be generalized to far-ﬁeld outdoor

scenes as the relationship between polarized light and sur-

face normals is not affected by distance. Experimental re-

sults demonstrate that our approach signiﬁcantly outper-

forms existing SfP models on two datasets. Our dataset

and source code will be publicly available at https:

//github.com/ChenyangLEI/sfp-wild.

1. Introduction

Accurate surface normal estimation in the wild can pro-

vide valuable information about a scene’s geometry and

can be used in various computer vision tasks, including

segmentation [19], 3D reconstruction [26], and many oth-

ers [22, 33]. Therefore, normal estimation is an important

task studied for a long time. However, estimating high-

quality normals in the wild is still an open problem. Var-

ious techniques such as photometric stereo [9, 10] can pro-

duce high-frequency normals, but most of them only pro-

vide short-range object-level normal maps. Active depth

sensors can be another approach to obtaining normals from

depth maps, but the corresponding depth maps are often

sparse (LiDAR) or noisy (time-of-ﬂight, structured light)

so they can not estimate normals reliably. Also, the depth

range of active sensors is limited.

Joint ﬁrst authors

Input I

Input ϕ Without pol. With pol.

Figure 1. Our method can estimate dense scene-level surface nor-

mals from a single polarization image. Polarization can provide

effective cues for obtaining more accurate results. In the ﬁrst row,

polarization provides geometry cues for our model so that it is not

fooled by objects in the printed image on a wall. In the second

and third rows, polarization provides guidance for planes with dif-

ferent surface normals even when their materials are quite similar.

: unpolarized image; ϕ: angle of polarization.

In this work, we are interested in estimating surface nor-

mal from a single polarization image for complex scenes in

the wild. Since the polarization of light changes differently

when the light interacts with the surfaces of different shapes

and materials (governed by the Fresnel equations [12]), the

polarization images can provide dense surface orientation

cues from the polarized light perceived at each pixel. Also,

compared with the active sensors and object-level normal

estimation techniques (e.g., photometric stereo), the polar-

ization camera is a passive sensor that is not constrained to a

speciﬁc depth range. Thus polarization images are promis-

ing data sources for accurate normal estimation in the wild.

However, estimating normals from a polarization image

for complex scenes (scene-level SfP) is challenging. To

the best of our knowledge, no existing SfP work focuses

on complex scenes, and several challenges are yet to be

solved. Firstly, polarization contains ambiguities from un-

known information such as object materials and reﬂection

12632

下载后可阅读完整内容，剩余9页未读，立即下载

STDHSQN

粉丝: 0
资源: 1

单幅偏振图像复杂场景法向估计

The definition of cross polarization_Ludwig.pdf

Hed_ell.zip_Polarization_electromagnetic_hed

Polar_BP_code.zip_BP改进_Polarization_polar码 PB译码_thoselja_极化码BP译码

FLat panel Vladimir_G._Chigrinov,_Vladimir_M._Kozenkov,_Hoi-(BookFi.org)

polarization.zip_Polarization_electromagnetic_极化_电磁

matlab.rar_MATLAB极化滤波_Polarization_matlab snrz.m_极化_极化滤波 MATLAB

tx_Q.zip_Polarization_pmd

polarization.zip_Polarization_polar coding_极化码_极化码 matlab_极化码；

Antenna_polarization_3D.m_极化方向图_极化_极化阵列_三维方向图_antenna_

about matlab.m.rar_Polarization_basickrk_matlab偏振度图_tabled1i_whe

最新资源