LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image
Chuhang Zou
†
Alex Colburn
‡
Qi Shan
‡
Derek Hoiem
†
†
University of Illinois at Urbana-Champaign
{czou4, dhoiem}@illinois.edu
‡
Zillow Group
{alexco, qis}@zillow.com
Abstract
We propose an algorithm to predict room layout from a
single image that generalizes across panoramas and per-
spective images, cuboid layouts and more general layouts
(e.g. “L”-shape room). Our method operates directly on the
panoramic image, rather than decomposing into perspec-
tive images as do recent works. Our network architecture is
similar to that of RoomNet [
15], but we show improvements
due to aligning the image based on vanishing points, pre-
dicting multiple layout elements (corners, boundaries, size
and translation), and fitting a constrained Manhattan lay-
out to the resulting predictions. Our method compares well
in speed and accuracy to other existing work on panora-
mas, achieves among the best accuracy for perspective im-
ages, and can handle both cuboid-shaped and more general
Manhattan layouts.
1. Introduction
Estimating the 3D layout of a room from one image is an
important goal, with applications such as robotics and vir-
tual/augmented reality. The room layout specifies the posi-
tions, orientations, and heights of the walls, relative to the
camera center. The layout can be represented as a set of
projected corner positions or boundaries, or as a 3D mesh.
Existing works apply to special cases of the problem, such
as predicting cuboid-shaped layouts from perspective im-
ages or from panoramic images.
We present LayoutNet, a deep convolution neural net-
work (CNN) that estimates the 3D layout of an indoor
scene from a single perspective or panoramic image (Fig-
ure.
1). Our method compares well in speed and accu-
racy on panoramas and is among the best on perspec-
tive images. Our method also generalizes to non-cuboid
Manhattan layouts, such as “L”-shaped rooms. Code is
available at:
https://github.com/zouchuhang/
LayoutNet
.
Our LayoutNet approach operates in three steps ( Fig-
ure.
2). First, our system analyzes the vanishing points
LayoutNet
Figure 1. Illustration. Our LayoutNet predicts a non-cuboid room
layout from a single panorama under equirectangular projection.
and aligns the image to be level with the floor (Sec. 3.1).
This alignment ensures that wall-wall boundaries are ver-
tical lines and substantially reduces error according to our
experiments. In the second step, corner (layout junctions)
and boundary probability maps are predicted directly on the
image using a CNN with an encoder-decoder structure and
skip connections (Sec.
3.2). Corners and boundaries each
provide a complete representation of room layout. We find
that jointly predicting them in a single network leads to bet-
ter estimation. Finally, the 3D layout parameters are opti-
mized to fit the predicted corners and boundaries (Sec.
3.4).
The final 3D layout loss from our optimization process is
difficult to back-propagate through the network, but direct
regression of the 3D parameters during training serves as an
effective substitute, encouraging predictions that maximize
accuracy of the end result.
Our contributions are:
• We propose a more general RGB image to layout al-
gorithm that is suitable for perspective and panoramic
2051
下载后可阅读完整内容，剩余8页未读，立即下载
wangcheng510
- 粉丝: 0
- 资源: 7
我的内容管理
展开
最新资源
- Google Test 1.8.x版本压缩包快速下载指南
- Java实现二叉搜索树的插入与查找功能
- Python库丰富性与数据可视化工具Matplotlib
- MATLAB通信仿真设计源代码与应用解析
- 响应式环保设备网站模板源码下载
- 微信小程序答疑平台完整设计源码案例
- 全元素DFT计算所需赝势UPF文件集合
- Object-C实现的Flutter组件开发详解
- 响应式环境设备网站模板下载 - 恒温恒湿机营销平台
- MATLAB绘图示例与知识点深入探讨
- DzzOffice平台新插件：excalidraw白板功能介绍与使用指南
- Java基础实训教程：电子商城项目开发与实践
- 物业集团管理系统数据库设计项目完整复刻包
- 三五族半导体能带参数计算器：精准模拟与应用
- 毕业论文：基于SSM框架的毕业生跟踪调查反馈系统设计与实现
- 国产化数据库适配：人大金仓与达梦实践教程
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
