《深度元学习》综述论文（2020年）

需积分: 10 92 浏览量更新于2023-03-16 评论收藏 2.85MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源评论

资源推荐

A Survey of Deep Meta-Learning

Mike Huisman m.huisman.8@umail.leidenuniv.nl

Jan N. van Rijn j.n.van.rijn@liacs.leidenuniv.nl

Aske Plaat a.plaat@liacs.leidenuniv.nl

Leiden Institute of Advanced Computer Science

Leiden University

Niels Bohrweg 1, 2333CA Leiden, The Netherlands

Abstract

Deep neural networks can achieve great successes when presented with large data sets and

suﬃcient computational resources. However, their ability to learn new concepts quickly is

quite limited. Meta-learning is one approach to address this issue, by enabling the network

to learn how to learn. The exciting ﬁeld of Deep Meta-Learning advances at great speed,

but lacks a uniﬁed, insightful overview of current techniques. This work presents just that.

After providing the reader with a theoretical foundation, we investigate and summarize key

methods, which are categorized into i) metric-, ii) model-, and iii) optimization-based tech-

niques. In addition, we identify the main open challenges, such as performance evaluations

on heterogeneous benchmarks, and reduction of the computational costs of meta-learning.

Keywords: Meta-learning, Learning to learn, Few-shot learning, Transfer learning, Deep

learning

1. Introduction

In recent years, deep learning techniques have achieved remarkable successes on various

tasks, including game-playing (Mnih et al., 2013; Silver et al., 2016), image recognition

(Krizhevsky et al., 2012; He et al., 2015), and machine translation (Wu et al., 2016). Despite

these advances, ample challenges remain to be solved, such as the large amounts of data and

training that are needed to achieve good performance. These requirements severely constrain

the ability of deep neural networks to learn new concepts quickly, one of the deﬁning aspects

of human intelligence (Jankowski et al., 2011; Lake et al., 2017).

Meta-learning has been suggested as one strategy to overcome this challenge (Naik and

Mammone, 1992; Schmidhuber, 1987; Thrun, 1998). The key idea is that meta-learning

agents improve their own learning ability over time, or equivalently, learn to learn. The

learning process is primarily concerned with tasks (set of observations) and takes place at

two diﬀerent levels: an inner- and an outer-level. At the inner-level, a new task is presented,

and the agent tries to quickly learn the associated concepts from the training observations.

This quick adaptation is facilitated by knowledge that it has accumulated across earlier

tasks at the outer-level. Thus, whereas the inner-level concerns a single task, the outer-level

concerns a multitude of tasks.

Historically, the term meta-learning has been used with various scopes. In its broadest

sense, it encapsulates all systems that leverage prior learning experience in order to learn new

arXiv:2010.03522v1 [cs.LG] 7 Oct 2020

Huisman, van Rijn, and Plaat

tasks more quickly (Vanschoren, 2018). This broad notion includes more traditional algo-

rithm selection and hyperparameter optimization techniques for Machine Learning (Brazdil

et al., 2008). In this work, however, we focus on a subset of the meta-learning ﬁeld which de-

velops meta-learning procedures to learn a good inductive bias for (deep) neural networks.

Henceforth, we use the term Deep Meta-Learning to refer to this subﬁeld of meta-learning.

The ﬁeld of Deep Meta-Learning is advancing at a quick pace, while it lacks a coherent,

unifying overview, providing detailed insights into the key techniques. Vanschoren (2018)

has surveyed meta-learning techniques, where meta-learning was used in the broad sense,

limiting its account of Deep Meta-Learning techniques. Also, many exciting developments

in deep meta-learning have happened after the survey was published. A more recent survey

by Hospedales et al. (2020) adopts the same notion of deep meta-learning as we do, but

aims for a broad overview, omitting technical details of the various techniques.

We attempt to ﬁll this gap by providing detailed explications of contemporary Deep

Meta-Learning techniques, using a uniﬁed notation. In addition, we identify current chal-

lenges and directions for future work. More speciﬁcally, we cover modern techniques in the

ﬁeld for supervised and reinforcement learning, that have achieved state-of-the-art perfor-

mance, obtained popularity in the ﬁeld, and presented novel ideas. Extra attention is paid

to MAML (Finn et al., 2017), and related techniques, because of their impact on the ﬁeld.

This work can serve as educational introduction to the ﬁeld of Deep Meta-Learning, and as

reference material for experienced researchers in the ﬁeld. Throughout, we will adopt the

taxonomy used by Vinyals (2017), which identiﬁes three categories of Deep Meta-Learning

approaches: i) metric-, ii) model-, and iii) optimization-based meta-learning techniques.

The remainder of this work is structured as follows. Section 2 builds a common founda-

tion on which we will base our overview of Deep Meta-Learning techniques. Section 3, 4, and

5 cover the main metric-, model-, and optimization-based meta-learning techniques, respec-

tively. Section 6 provides a helicopter view of the ﬁeld, and summarizes the key challenges

and open questions. Table 1 gives an overview of notation that we will use throughout this

paper.

2. Foundation

In this section, we build the necessary foundation for investigating Deep Meta-Learning

techniques in a consistent manner. To begin with, we contrast regular learning and meta-

learning. Afterwards, we brieﬂy discuss how Deep Meta-Learning relates to diﬀerent ﬁelds,

what the usual training and evaluation procedure looks like, and which benchmarks are often

used for this purpose. We ﬁnish this section by describing some applications and context of

the meta-learning ﬁeld.

2.1 The Meta Abstraction

In this subsection, we contrast base-level (regular) learning and meta-learning for two dif-

ferent paradigms, i.e., supervised and reinforcement learning.

1. Here, inductive bias refers to the assumptions of a model which guide predictions on unseen data

(Mitchell, 1980).

A Survey of Deep Meta-Learning

Expression Meaning

Meta-learning Learning meta-knowledge that can be used to learn new tasks more quckly

= (D

, D

test

) A task consisting of a labeled train and test set

Support set The train set D

associated with a task T

Query set The test set D

test

associated with a task T

Example input vector i in the support set

(One-hot encoded) label of example input x

from the support set

x Input in the query set

y A (one-hot encoded) label for input x

(f/g/h)

◦

Neural network function with parameters ◦

Inner-level At the level of a single task

Outer-level At meta-level: across tasks

Fast weights Parameters that were generated for a speciﬁc task/example

Base-learner Learner that works at the inner-level

Meta-learner Learner that operates at the outer-level

Input embedding Activation pattern in the ﬁnal layer of a neural network caused by the input

Task embedding An internal representation of a task in a network/system

SL Supervised Learning

RL Reinforcement Learning

Table 1: Some notation and meaning, which we use throughout this paper.

2.1.1 Regular Supervised Learning

In supervised learning, we wish to learn a function f

: X → Y that learns to map inputs

∈ X to their corresponding outputs y

∈ Y . Here, θ are model parameters (e.g. weights

in a neural network) that determine the function’s behavior. To learn these parameters, we

are given a data set of m observations: D = {(x

, y

)}

i=1

. Thus, given a data set D, learning

boils down to ﬁnding the correct setting for θ that minimizes an empirical loss function L

which must capture how the model is performing, such that appropriate adjustments to its

parameters can be made. In short, we wish to ﬁnd

:= arg min

(θ), (1)

where SL stands for “supervised learning". Note that this objective is speciﬁc to data set

D, meaning that our model f

may not generalize to examples outside of D. To measure

generalization, one could evaluate the performance on a separate test data set, which contains

unseen examples. A popular way to do this is through cross-validation, where one repeatedly

creates train and test splits D

, D

test

⊂ D and uses these to train and evaluate a model

respectively (Hastie et al., 2009).

Finding globally optimal parameters θ

is often computationally infeasible. We can,

however, approximate them, guided by pre-deﬁned meta-knowledge ω (Hospedales et al.,

2020), which includes, e.g., the initial model parameters θ, choice of optimizer, and learning

Huisman, van Rijn, and Plaat

rate schedule. As such, we approximate

≈ g

(D, L

), (2)

where g

is an optimization procedure that uses pre-deﬁned meta-knowledge ω, data set D,

and loss function L

, to produce updated weights g

(D, L

) that (presumably) perform

well on D.

2.1.2 Supervised Meta-Learning

In contrast, supervised meta-learning does not assume that any meta-knowledge ω is given,

or pre-deﬁned. Instead, the goal of meta-learning is to ﬁnd the best ω, such that our

(regular) base-learner can learn new tasks (data sets) as quickly as possible. Thus, whereas

supervised regular learning involves one data set, supervised meta-learning involves a group

of data sets. The goal is to learn meta-knowledge ω such that our model can learn many

diﬀerent tasks well. Thus, our model is learning to learn.

More formally, we have a probability distribution of tasks p(T ), and wish to ﬁnd optimal

meta-knowledge

∗

:= arg min

vp(T )

| {z }

Outer-level

, L

))

| {z }

Inner-level

]. (3)

Here, the inner-level concerns task-speciﬁc learning, while the outer-level concerns multiple

tasks. One can now easily see why this is meta-learning: we learn ω, which allows for quick

learning of tasks T

at the inner-level. Hence, we are learning to learn.

2.1.3 Regular Reinforcement Learning

In reinforcement learning, we have an agent that learns from experience. That is, it interacts

with an environment, modeled by a Markov Decision Process (MDP) M = (S, A, P, r, p

, γ, T ).

Here, S is the set of states, A the set of actions, P the transition probability distribution

deﬁning P (s

t+1

, a

), r : S × A → R the reward function, p

the probability distribution

over initial states, γ ∈ [0, 1] the discount factor, and T the time horizon (maximum number

of time steps) (Sutton and Barto, 2018; Duan et al., 2016).

At every time step t, the agent ﬁnds itself in state s

, in which the agent performs an

action a

, computed by a policy function π

(i.e., a

= π

)), which is parameterized by

weights θ. In turn, it receives a reward r

= r(s

, π

)) ∈ R and a new state s

t+1

. This

process of interactions continues until a termination criterion is met (e.g. ﬁxed time horizon

T reached). The goal of the agent is to learn how to act in order to maximize its expected

reward. The reinforcement learning (RL) goal is to ﬁnd

:= arg min

traj

t=0

r(s

, π

)), (4)

where we take the expectation over the possible trajectories traj = (s

, π

), ...s

, π

))

due to the random nature of MDPs (Duan et al., 2016). Note that γ is a hyperparameter

that can prioritize short- or long-term rewards by decreasing or increasing it, respectively.

A Survey of Deep Meta-Learning

Also in case of reinforcement learning it is often infeasible to ﬁnd the global optimum θ

and thus we settle for approximations. In short, given a learning method ω, we approximate

≈ g

, L

), (5)

where again T

is the given MDP, and g

is the optimization algorithm, guided by pre-deﬁned

meta-knowledge ω.

Note that in a Markov Decision Process (MDP), the agent knows the state at any given

time step t. When this is not the case, it becomes a Partially Observable Markov Decision

Process (POMDP), where the agent receives only observations O, and uses these to update

its belief with regard to the state it is in (Sutton and Barto, 2018).

2.1.4 Meta Reinforcement Learning

The meta abstraction has as its object a group of tasks, or Markov Decision Processes

(MDPs) in the case of reinforcement learning. Thus, instead of maximizing the expected

reward on a single MDP, the meta reinforcement learning objective is to maximize the

expected reward over various MDPs, by learning meta-knowledge ω. Here, the MDPs are

sampled from some distribution p(T ). So now, we wish to ﬁnd a set of parameters

∗

:= arg min

vp(T )

| {z }

Outer-level







traj

t=0

r(s

, π

)

))

| {z }

Inner-level







. (6)

2.1.5 Contrast with other Fields

Now that we have provided a formal basis for our discussion for both supervised and rein-

forcement meat-learning, it is time to contrast meta-learning brieﬂy with two related areas

of machine learning that also have the goal to improve the speed of learning. We will start

with transfer learning.

Transfer Learning In Transfer Learning, one tries to transfer knowledge of previous

tasks to new, unseen tasks (Pan and Yang, 2009; Taylor and Stone, 2009). As such, it

subsumes meta-learning, where we attempt to leverage meta-knowledge to learn new tasks

more quickly. A key property of meta-learning techniques is their meta-objective, which

explicitly aims to optimize performance across a distribution over tasks (as seen in previous

sections by taking the expected loss over a distribution of tasks). This objective need not

always be present in Transfer Learning techniques, e.g., when one pre-trains a model on a

large data set, and ﬁne-tunes the learned weights on a smaller data set.

Multi-task learning An other, closely related ﬁeld, is that of multi-task learning.

In multi-task learning a model is jointly trained to perform well on multiple ﬁxed tasks

(Hospedales et al., 2020). Meta-learning, in contrast, aims to ﬁnd a model that can learn

new (previously unseen) tasks quickly. This diﬀerence is illustrated in Figure 1.

剩余58页未读，继续阅读

syp_net

粉丝: 158
资源: 1196

会员权益专享

《深度元学习》综述论文（2020年）

评论0

会员权益专享

最新资源

《深度元学习》综述论文（2020年）

评论0

图表示深度学习综述（Deep Learning for Learning Graph Representations）【清华大学朱文武老师】.zip

元学习综述，meta learning，A Survey of Zero-Shot Learning + poster

2019-2020必看的十篇【深度学习领域综述论文】.zip

深度学习综述.pdf

写一篇基于深度学习文字识别的综述论文框架

深度学习文献综述笔记csdn

生成一篇深度学习配准综述

深度学习图像去噪综述

深度学习无监督图像分割综述

基于深度学习异常检测综述

2020年到2023年深度学习掌纹分类方法有什么，至少10个

模式识别与深度学习的综述

深度学习算法的研究综述

深度学习图像匹配综述

面向深度学习的多模态融合技术研究综述_何俊.pdf

写一篇模式识别与深度学习的综述

深度学习心脏超声图像分析综述

半监督目标检测深度学习方法综述

深度学习论文引言300字

深度学习方面的经典论文

会员权益专享

最新资源