ASurveyofZero-ShotLearning(零样本学习综述)_零样本学习

深度学习

零样本学习

需积分: 47 1.2k 浏览量更新于2023-05-25 评论收藏 858KB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源评论

资源推荐

A Survey of Zero-Shot Learning: Seings, Methods,

and Applications

WEI WANG,

Nanyang Technological University, Singapore

VINCENT W. ZHENG,

WeBank, China

HAN YU and CHUNYAN MIAO,

Nanyang Technological University, Singapore

Most machine-learning methods focus on classifying instances whose classes have already been seen in train-

ing. In practice, many applications require classifying instances whose classes have not been seen previously.

Zero-shot learning is a powerful and promising learning paradigm, in which the classes covered by training

instances and the classes we aim to classify are disjoint. In this paper, we provide a comprehensive survey of

zero-shot learning. First of all, we provide an overview of zero-shot learning. According to the data utilized in

model optimization, we classify zero-shot learning into three learning settings. Second, we describe dierent

semantic spaces adopted in existing zero-shot learning works. Third, we categorize existing zero-shot learning

methods and introduce representative methods under each category. Fourth, we discuss dierent applications

of zero-shot learning. Finally, we highlight promising future research directions of zero-shot learning.

CCS Concepts: • Computing methodologies → Transfer learning;

Additional Key Words and Phrases: Zero-shot learning survey

ACM Reference format:

Wei Wang, Vincent W. Zheng, Han Yu, and Chunyan Miao. 2019. A Survey of Zero-Shot Learning: Settings,

Methods, and Applications. ACM Trans. Intell. Syst. Technol. 10, 2, Article 13 (January 2019), 37 pages.

https://doi.org/10.1145/3293318

1 INTRODUCTION

Supervised classication methods have achieved signicant success in research and have been

applied in many areas. Especially in recent years, beneting from the fast development of deep

This research is supported, in part, by the National Research Foundation, Prime Minister’s Oce, Singapore under its

IDM Futures Funding Initiative; the Interdisciplinary Graduate School, Nanyang Technological University, Singapore; and

the Nanyang Assistant Professorship (NAP). It is also partially supported by the NTU-PKU Joint Research Institute, a

collaboration between Nanyang Technological University and Peking University that is sponsored by a donation from the

Ng Teng Fong Charitable Foundation.

Authors’ addresses: W. Wang, Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY),

Interdisciplinary Graduate School, and School of Computer Science and Engineering, Nanyang Technological University,

50 Nanyang Avenue, Singapore; email: wwang008@e.ntu.edu.sg; V. W. Zheng, WeBank, Shahexilu 1819, Nanshan,

Shenzhen, China; email: wenchenzheng@gmail.com; H. Yu, School of Computer Science and Engineering, and Joint

NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY), Nanyang Technological University,

50 Nanyang Avenue, Singapore; email: han.yu@ntu.edu.sg; C. Miao, School of Computer Science and Engineering, and

Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY), Nanyang Technological University,

50 Nanyang Avenue, Singapore; email: ascymiao@ntu.edu.sg.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee

provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and

the full citation on the rst page. Copyrights for components of this work owned by others than ACM must be honored.

Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires

prior specic permission and/or a fee. Request permissions from permissions@acm.org.

2157-6904/2019/01-ART13 $15.00

https://doi.org/10.1145/3293318

ACM Transactions on Intelligent Systems and Technology, Vol. 10, No. 2, Article 13. Publication date: January 2019.

13:2 W. Wang et al.

learning techniques, they have made much progress. However, there are some restrictions for

methods under this learning paradigm. In supervised classication, sucient labeled training in-

stances are needed for each class. In addition, the learned classier can only classify the instances

belonging to classes covered by the training data, and it lacks the ability to deal with previously

unseen classes. However, in practical applications, there may not be sucient training instances

for each class. There could also be situations in which the classes not covered by the training

instances appear in the testing instances.

To deal with these problems, methods under dierent learning paradigms have been proposed.

To tackle the problem of learning classiers for classes with few training instances, few-shot

learning/one-shot learning methods [35, 122] have been proposed. In these methods, while learning

classiers for the classes with few instances, knowledge contained in instances of other classes is

utilized. To deal with previously unseen classes, a range of learning methods have been proposed.

In open set recognition methods [64, 132], when learning the classier with the training data, the fact

of unseen classes existing is taken in consideration. The learned classier can determine whether a

testing instance belongs to the unseen classes, but it cannot determine which specic unseen class

the instance belongs to. The cumulative learning [34]andclass-incremental learning [123] methods

have been proposed for problems in which labeled instances belonging to some previously un-

seen classes progressively appear after model learning. The learned classier can be adapted with

these newly available labeled instances to be able to classify classes covered by them. The open

world recognition [9] methods follow the process of “unseen classes detection, labeled instances

of unseen classes acquisition, and model adaption” and adapt the classier to be able to classify

previously unseen classes with the acquired labeled instances belonging to them.

For methods under the above learning paradigms, if the testing instances belong to unseen

classes that have no available labeled instances during model learning (or adaption), the learned

classier cannot determine the class labels of them. However, in many practical applications, we

need the classier to have the ability to determine the class labels for the instances belonging to

these classes. The following are some popular application scenarios:

• The number of target classes is large. An example is object recognition in computer vision.

Generally, human beings can recognize at least 30,000 object classes [10]. However, col-

lecting sucient labeled instances for such a large number of classes is challenging. Thus,

existing image datasets can only cover a small subset of these classes. There are many ob-

ject classes having no labeled instances in existing datasets. A similar example is activity

recognition [178], where the number of human activities is large, in contrast with the lim-

ited number of activity classes covered by existing datasets. Many activity classes have no

labeled instances in existing datasets.

• Target classes are rare. An example is ne-grained object classication. Suppose we want to

recognize owers of dierent breeds [8, 31]. It is hard to collect sucient image instances

for each specic ower breed. For many rare breeds, we cannot nd the corresponding

labeled instances.

• Target classes change over time. An example is recognizing images of products belonging to

a certain style and brand. As products of new styles and new brands appear frequently, for

some new products, it is dicult to nd corresponding labeled instances [98].

• In some particular tasks, it is expensive to obtain labeled instances. In some learning tasks

related with classication, the instance labeling process is expensive and time consuming.

Thus, the number of classes covered by existing datasets is limited, and many classes have no

labeled instances. For example, in the image semantic segmentation problem [93, 104], the

images used as training data should be labeled at the pixel level. This problem can be seen

ACM Transactions on Intelligent Systems and Technology, Vol. 10, No. 2, Article 13. Publication date: January 2019.

A Survey of Zero-Shot Learning: Seings, Methods, and Applications 13:3

as a pixel-level classication problem for the images. The number of object classes covered

by existing datasets is limited, with many object classes having no labeled instances. In

another example, in the image captioning problem [140], each image in the training data

should have a corresponding caption. This problem can be seen as a sequential classication

problem. The number of object classes covered by the existing image-text corpora is limited,

with many object classes not being covered.

In these applications, there are many classes having no labeled instances. For a classier, it is

important for it to have the ability to determine the class label of instances belonging to these

classes. To solve this problem, zero-shot learning (also known as zero-data learning [81]) is pro-

posed. The aim of zero-shot learning is to classify instances belonging to the classes that have no

labeled instances. Since its inception [80, 81, 113], zero-shot learning has become a fast-developing

eld in machine learning, with a wide range of applications in computer vision, natural language

processing, and ubiquitous computing.

1.1 Overview of Zero-Shot Learning

In zero-shot learning, there are some labeled training instances in the feature space. The classes

covered by these training instances are referred to as the seen classes. In the feature space, there

are also some unlabeled testing instances, which belong to another set of classes. These classes are

referred to as the unseen classes. The feature space is usually a real number space, and each instance

is represented as a vector within it. Each instance is usually assumed to belong to one class.

Now, we give the denition of zero-shot learning. Denote S = {c

|i = 1,...,N

} as the set of seen

classes, where each c

is a seen class. Denote U = {c

|i = 1,...,N

} as the set of unseen classes,

where each c

is an unseen class. Note that S∩U = ∅.DenoteX as the feature space, which is D-

dimensional; usually it is a real number space R

.DenoteD

= {(x

, y

) ∈X×S}

i=1

as the set

of labeled training instances belonging to seen classes; for each labeled instance (x

, y

), x

is the

instance in the feature space, and y

is the corresponding class label. Denote X

= {x

∈X}

i=1

as the set of testing instances, where each x

is a testing instance in the feature space. Denote

= {y

∈U}

i=1

as the corresponding class labels for X

, which are to be predicted.

Denition 1.1 (Zero-Shot Learning). Given labeled training instances D

belonging to the seen

classes S, zero-shot learning aims to learn a classier f

(·) : X→U that can classify testing

instances X

(i.e., to predict Y

) belonging to the unseen classes U .

From the denition, we can see that the general idea of zero-shot learning is to transfer the

knowledge contained in the training instances D

to the task of testing instance classication.

The label spaces covered by the training and the testing instances are disjoint. Thus, zero-shot

learning is a subeld of transfer learning [114, 115]. In transfer learning, knowledge contained in

the source domain and source task is transferred to the target domain for learning the model in

the target task [114, 115]. According to [23, 114], based on whether the feature spaces and label

spaces in the source and target domains/tasks are the same, transfer learning can be classied

into homogeneous transfer learning and heterogeneous transfer learning. In homogeneous transfer

learning, the feature spaces and the label spaces are the same, while in heterogeneous transfer

learning, the feature spaces and/or the label spaces are dierent. In zero-shot learning, the source

feature space is the feature space of training instances, and the target feature space is the fea-

ture space of testing instances. They are the same; both are X. However, the source label space

There are some works on zero-shot learning under the multilabel setting, which focuses on classifying instances that each

has more than one class label. We will separately discuss these works in Section 3.3.

ACM Transactions on Intelligent Systems and Technology, Vol. 10, No. 2, Article 13. Publication date: January 2019.

13:4 W. Wang et al.

is the seen class set S, while the target label space is the unseen class set U. They are dier-

ent. In view of this, zero-shot learning belongs to heterogeneous transfer learning. Specically,

it belongs to heterogeneous transfer learning with dierent label spaces (we briey refer to it as

HTL-DLS). Many existing methods for HTL-DLS are proposed for problems under the setting in

which there are some labeled instances for the target label space classes [23]. However, in zero-shot

learning, no labeled instances belonging to the target label space classes (the unseen classes) are

available. This makes the problems in zero-shot learning dierent from these problems studied in

HTL-DLS.

Auxiliary information. As no labeled instances belonging to the unseen classes are available,

to solve the zero-shot learning problem, some auxiliary information is necessary. Such auxiliary

information should contain information about all of the unseen classes. This is to guarantee that

each of the unseen classes is provided with corresponding auxiliary information. Meanwhile, the

auxiliary information should be related to the instances in the feature space. This is to guarantee

that the auxiliary information is usable.

In existing works, the approach to involve auxiliary information is inspired by the way human

beings recognize the world. Humans can perform zero-shot learning with the help of some seman-

tic background knowledge. For example, with the knowledge that “a zebra looks like a horse, and

with stripes,” we can recognize a zebra even without having seen one before, as long as we know

what a horse looks like and what the pattern “stripe” looks like [45]. In this way, the auxiliary

information involved by existing zero-shot learning methods is usually some semantic informa-

tion. It forms a space that contains both the seen and the unseen classes. As this space contains

semantic information, it is often referred to as the semantic space. Being similar to the feature

space, the semantic space is also usually a real number space. In the semantic space, each class has

a corresponding vector representation, which is referred to as the class prototype (or prototype for

short) of this class.

We denote T as the semantic space. Suppose T is M-dimensional; it is usually

.Denotet

∈T as the class prototype for seen class c

,andt

∈T as the class prototype for

unseen class c

.DenoteT

= {t

}

i=1

as the set of prototypes for seen classes, and T

= {t

}

i=1

the set of prototypes for unseen classes. Denote π (·) : S∪U →T as a class prototyping func-

tion that takes a class label as input and outputs the corresponding class prototype. In zero-shot

learning, along with the training instances D

, the class prototypes T

and T

are also involved

in obtaining the zero-shot classier f

(·). In Section 2, we will categorize and introduce dierent

kinds of semantic spaces.

We summarize the key notations used throughout this article in Table 1.

1.2 Learning Seings

In zero-shot learning, the goal is to learn the zero-shot classier f

(·). During model learning, if

information about the testing instances is involved, the learned model is transductive for these spe-

cic testing instances. In zero-shot learning, this transduction can be embodied in two progressive

degrees: transductive for specic unseen classes and transductive for specic testing instances.

This is dierent from the well-known transductive setting in semisupervised learning, which is

just for the testing instances. In the setting that is transductive for specic unseen classes, infor-

mation about the unseen classes is involved in model learning, and the model is optimized for

these specic unseen classes. In the setting that is transductive for specic testing instances, the

transductive degree goes further. The testing instances are also involved in model learning, and

In some works, for each class, there is more than one corresponding prototype in the semantic space. We will separately

discuss them in Section 3.3.

ACM Transactions on Intelligent Systems and Technology, Vol. 10, No. 2, Article 13. Publication date: January 2019.

A Survey of Zero-Shot Learning: Seings, Methods, and Applications 13:5

Table 1. Key Notations Used in This Article

Notation Description

X Feature space, which is D-dimensional

T Semantic space, which is M-dimensional

S, U Set of seen classes and set of unseen classes, respectively

, N

Number of training instances and number of testing instances, respectively

, N

Number of seen classes and number of unseen classes, respectively

The set of labeled training data from seen classes

The set of testing instances from unseen classes

Labels for testing instances

, y

) The ith labeled training instance: features x

∈Xand label y

∈S

The ith unlabeled testing instance: features x

∈X

The set of prototypes for seen classes and unseen classes, respectively

, t

) The ith seen class c

∈Sand its class prototype t

∈T

, t

) The ith unseen class c

∈Uand its class prototype t

∈T

π (·) A class prototyping function π (·) : S∪U →T

(·) A zero-shot classier f

(·) : X→U

Fig. 1. Dierent learning seings for zero-shot learning.

the model is optimized for these specic testing instances. Based on the degree of transduction,

we categorize zero-shot learning into three learning settings.

Denition 1.2 (Class-Inductive Instance-Inductive (CIII) Setting). Only labeled training instances

and seen class prototypes T

are used in model learning.

Denition 1.3 (Class-Transductive Instance-Inductive (CTII) Setting). Labeled training instances

, seen class prototypes T

, and unseen class prototypes T

are used in model learning.

Denition 1.4 (Class-Transductive Instance-Transductive (CTIT) Setting). Labeled training in-

stances D

, seen class prototypes T

, unlabeled testing instances X

, and unseen class prototypes

are used in model learning.

We summarize the above three learning settings in Figure 1. As we can see, from CIII to CTIT,

the classier f

(·) is learned with increasingly specic testing instances’ information. In machine-

learning methods, as the distributions of the training and the testing instances are dierent, the

performance of the model learned with the training instances will decrease when applied to the

testing instances. This phenomenon is more severe in zero-shot learning, as the classes covered by

the training and the testing instances are disjoint [6, 41]. In zero-shot learning, this phenomenon

is usually referred to as domain shift [41].

Under the CIII setting, as no information about the testing instances is involved in model learn-

ing, the problem of domain shift is severe in some methods under this setting. However, as the

models under this setting are not optimized for specic unseen classes and testing instances, when

ACM Transactions on Intelligent Systems and Technology, Vol. 10, No. 2, Article 13. Publication date: January 2019.

剩余36页未读，继续阅读

Brucechows

粉丝: 49
资源: 36

会员权益专享

A Survey of Zero-Shot Learning(零样本学习综述)

评论0

会员权益专享

最新资源

A Survey of Zero-Shot Learning(零样本学习综述)

评论0

a survey of deep meta-learning

深度元学习-A Survey of Deep Meta-Learning-翻译.docx

a survey of deep learning-based object detection

帮我检索20篇有关语义协同的外文文献

meta-learning in neural networks: a survey

a survey of modern deep learning based object detection models

帮我讲解总结一下Self-Supervised Learning for Recommender Systems: A Survey这篇论文

给我推荐20个比较流行的自动驾驶算法模型代码

deep learning for 3d point clouds: a survey

Object Tracking A Survey

a survey on transfer learning

元学习（meta learning）综述论文（2020年）

a comprehensive survey on transfer learning

有什么关系型数据库的论文综述推荐吗？

a survey on transfer learning下载

目标检测综述PPT-Object Detection in 20 Years: A Survey

a survey of transformers

a survey of visual transformers

a survey of compiler testing

基于人体骨架的异常行为动作识别国内外参考文献

会员权益专享

最新资源