深度学习检测与识别人类对象交互：InteractNet模型

需积分: 14 191 浏览量更新于2024-09-07 收藏 7.31MB PDF 举报

"检测和识别人类-物体交互是理解视觉场景的关键。本文提出了一种名为InteractNet的新模型，该模型采用以人为核心的策略，通过人的外观（如姿势、服装和动作）来预测与之交互的物体的位置。" 在深度学习和计算机视觉领域，"Detecting and Recognizing Human-Object Interactions"是一个重要的研究课题。它涉及到的目标识别和照片检测是图像理解和人工智能的核心部分。人类-物体交互检测是指识别图像中人物与周围环境或物体的互动行为，例如“人正在拿起手机”或“人正在骑自行车”。这对于视频监控、自动驾驶、智能安防等实际应用具有重大意义。 InteractNet模型由Georgia Gkioxari、Ross Girshick、Piotr Dollár和Kaiming He等人提出，他们均来自Facebook AI Research (FAIR)。这个模型的独特之处在于它采用了以人为核心的策略。模型首先通过检测到的人物外观特征，比如其姿势、着装和动作，来预测可能的交互对象的位置。这依赖于模型对特定动作相关的物体位置密度的预测能力。通过这种方式，InteractNet能够利用人物的视觉信息来引导物体检测，提高了交互识别的准确性。此外，InteractNet模型还同时学习检测人物和物体，并通过融合这些预测结果来高效地推断出交互三元组。这种端到端的联合训练系统可以确保整个流程的协同优化，从而提高整体性能。模型的验证是在最近引入的基准数据集上进行的，这进一步证明了其在复杂日常场景中检测人类-物体交互的能力。 InteractNet的贡献在于提供了一个强大的工具，不仅能够识别单独的物体和人物，还能理解他们之间的关系。这种理解有助于提升计算机视觉系统的上下文感知和智能水平，对于未来智能家居、虚拟现实、增强现实以及机器人技术等领域的发展具有深远的影响。通过不断的研究和改进，我们可以期待更精确的人类-物体交互检测技术，进一步推动人工智能的边界。

Detecting and Recognizing Human-Object Interactions

Georgia Gkioxari Ross Girshick Piotr Doll

ar Kaiming He

Facebook AI Research (FAIR)

Abstract

To understand the visual world, a machine must not only

recognize individual object instances but also how they in-

teract. Humans are often at the center of such interac-

tions and detecting human-object interactions is an impor-

tant practical and scientiﬁc problem. In this paper, we ad-

dress the task of detecting hhuman, verb, objecti triplets

in challenging everyday photos. We propose a novel model

that is driven by a human-centric approach. Our hypothesis

is that the appearance of a person – their pose, clothing,

action – is a powerful cue for localizing the objects they

are interacting with. To exploit this cue, our model learns

to predict an action-speciﬁc density over target object loca-

tions based on the appearance of a detected person. Our

model also jointly learns to detect people and objects, and

by fusing these predictions it efﬁciently infers interaction

triplets in a clean, jointly trained end-to-end system we call

InteractNet. We validate our approach on the recently intro-

duced Verbs in COCO (V-COCO) and HICO-DET datasets,

where we show quantitatively compelling results.

1. Introduction

Visual recognition of individual instances, e.g., detect-

ing objects [10, 9, 26] and estimating human actions/poses

[11, 31, 2], has witnessed signiﬁcant improvements thanks

to deep learning visual representations [17, 29, 30, 16].

However, recognizing individual objects is just a ﬁrst step

for machines to comprehend the visual world. To under-

stand what is happening in images, it is necessary to also

recognize relationships between individual instances. In

this work, we focus on human-object interactions.

The task of recognizing human-object interactions [12,

32, 6, 13, 5] can be represented as detecting hhuman, verb,

objecti triplets and is of particular interest in applica-

tions and in research. From a practical perspective, pho-

tos containing people contribute a considerable portion of

daily uploads to internet and social networking sites, and

thus human-centric understanding has signiﬁcant demand

in practice. From a research perspective, the person cate-

gory involves a rich set of actions/verbs, most of which are

(a) (b)

cut

knife

(c)

stand

(d)

target

prob density

for cut

Figure 1. Detecting and recognizing human-object interactions.

(a) There can be many possible objects (green boxes) interacting

with a detected person (blue box). (b) Our method estimates an

action-type speciﬁc density over target object locations from the

person’s appearance, which is represented by features extracted

from the detected person’s box. (c) A hhuman, verb, objecti

triplet detected by our method, showing the person box, action

(cut), and target object box and category (knife). (d) Another pre-

dicted action (stand), noting that a person can simultaneously take

multiple actions and an action may not involve any objects.

rarely taken by other subjects (e.g., to talk, throw, work).

The ﬁne granularity of human actions and their interactions

with a wide array of object types presents a new challenge

compared to recognition of entry-level object categories.

In this paper, we present a human-centric model for rec-

ognizing human-object interaction. Our central observation

is that a person’s appearance, which reveals their action and

pose, is highly informative for inferring where the target

object of the interaction may be located (Figure 1(b)). The

search space for the target object can thus be narrowed by

conditioning on this estimation. Although there are often

many objects detected (Figure 1(a)), the inferred target lo-

cation can help the model to quickly pick the correct object

associated with a speciﬁc action (Figure 1(c)).

We implement this idea as a human-centric recognition

arXiv:1704.07333v3 [cs.CV] 27 Mar 2018

下载后可阅读完整内容，剩余9页未读，立即下载

huyiqun6

粉丝: 1
资源: 40

深度学习检测与识别人类对象交互：InteractNet模型

Detecing and Recognizing Human-Object Interactions

Spatially Conditioned Graphs for Detecting Human-Object.pdf

Detecting CD20-Rituximab specific interactions on lymphoma cells using atomic force microscopy

detecting-PAN-no-and-DOB-from-pan-card-image

Radar-Detecting-and-Tracking-master_radardetection_detection_rad

human-detection-and-motion-detection.rar_AVI_human motion matlab

Detecting peer-to-peer botnets

ÐArcher Detecting On-Chain-Off-Chain Synchronization Bugs in De

Detecting faces in images-a survey

最新资源