![](https://csdnimg.cn/release/download_crawler_static/12889004/bg4.jpg)
SUBMITTED MANUSCRIPT TO IEEE TRANSACTIONS ON IMAGE PROCESSING 4
the Wasserstein-1 distance between the generated distribution
and the real distribution as
min
G
max
kDk
L
≤1
E
x∼p
data
[D(x)] − E
z∼p
z
[D(G(z))], (2)
where D is constrained to be the 1-Lipschitz function imple-
mented by weight clipping. Furthermore, WGAN-GP [25] im-
proves WGAN on the implementation of Lipschitz constraint
by imposing a gradient penalty on the discriminator instead
of weight clipping. In this work, we adopt WGAN-GP for the
adversarial learning.
Several works have been developed for the conditional
generation with given attributes or class labels [20], [26]–
[28]. Employing an auxiliary classifier or regressor, both AC-
GAN [27] and InfoGAN [28] learn the conditional generation
by mapping the generated images back to the conditional
signals. Inspired, in this work, we also map the edited face
images back to the given attributes forming the attribute
classification constraint. Different from AC-GAN [27], the
generated images do not participate in the training of the
auxiliary classifier.
III. ATTRIBUTE GAN (ATTGAN)
This section introduces the AttGAN approach for the editing
of binary facial attributes
3
. As shown in Fig. 2, our AttGAN
is comprised of two basic subnetworks, i.e., an encoder G
enc
and a decoder G
dec
, together with an attribute classifier C and
a discriminator D. In the following, we describe the design
principles of AttGAN and introduce the objectives for training
these components. Then we present an extension of AttGAN
for attribute style manipulation.
A. Testing Formulation
Given a face image x
a
with n binary attributes a =
[a
1
, ..., a
n
], the encoder G
enc
is used to encode x
a
into the
latent representation, denoted as
z = G
enc
(x
a
). (3)
Then the process of editing the attributes of x
a
to another at-
tributes b = [b
1
, ..., b
n
] is achieved by decoding z conditioned
on b, i.e.,
x
ˆ
b
= G
dec
(z, b), (4)
where x
ˆ
b
is the edited image expected to own the attribute b.
Thus the whole editing process is formulated as
x
ˆ
b
= G
dec
(G
enc
(x
a
), b). (5)
B. Training Formulation
It can be seen from Eq. (5) that the attribute editing problem
can be formally defined as the learning of the encoder G
enc
and decoder G
dec
. This learning problem is unsupervised,
because the ground truth of the editing, i.e. x
b
, is unavailable.
On one hand, the editing on the given face image x
a
is
expected to produce a realistic image with attributes b. For
3
each attribute is represented by 1/0 for with/without it and all attributes
are represented by a 1/0 sequence.
this purpose, an attribute classifier is used to constrain the
generated image x
ˆ
b
to correctly own the desired attributes,
i.e., the attribute prediction of x
ˆ
b
should be b. Meanwhile,
the adversarial learning is employed on x
ˆ
b
to ensure its
visual reality.
On the other hand, an eligible attribute editing should only
change those desired attributes, while keeping the other details
unchanged. To this end, the reconstruction learning is intro-
duced to 1) make the latent representation z conserve enough
information for the later recovery of the attribute-excluding
details, 2) enable the decoder G
dec
to restore the attribute-
excluding details from z. Specifically, for the given x
a
, the
generated image conditioned on its own attributes a, i.e.,
x
ˆa
= G
dec
(z, a) (6)
should approximate x
a
itself, i.e., x
ˆa
→ x
a
.
In summary, the relation between the attributes a/b and the
latent representation z is implicitly modeled in two aspects:
1) the interaction between z and b in the decoder should
produce an realistic image x
ˆ
b
with correct attributes, and 2)
the interaction between z and a in the decoder should produce
an image x
ˆa
approximating the input x
a
itself.
Attribute Classification Constraint. As mentioned above,
it is required that the generated image x
ˆ
b
should correctly
own the new attributes b. Therefore, we employ an attribute
classifier C to constrain the generated image x
ˆ
b
to own the
desired attributes, i.e., C(x
ˆ
b
) → b, formulated as follows,
min
G
enc
,G
dec
L
cls
g
= E
x
a
∼p
data
,b∼p
attr
[`
g
(x
a
, b)], (7)
`
g
(x
a
, b) =
n
P
i=1
−b
i
log C
i
(x
ˆ
b
) − (1 − b
i
) log(1 − C
i
(x
ˆ
b
)), (8)
where p
data
and p
attr
indicate the distribution of real images
and the distribution of attributes, C
i
(x
ˆ
b
) indicates the predic-
tion of the i
th
attribute, and `
g
(x
a
, b) is the summation of
binary cross entropy losses of all attributes.
The attribute classifier C is trained on the input images with
their original attributes, by the following objective,
min
C
L
cls
c
= E
x
a
∼p
data
[`
r
(x
a
, a)], (9)
`
r
(x
a
, a) =
n
P
i=1
−a
i
log C
i
(x
a
) − (1 − a
i
) log(1 − C
i
(x
a
)). (10)
Reconstruction Loss. Furthermore, the reconstruction
learning aims for satisfactory preservation of attribute-
excluding details. To this end, the decoder should learn to
reconstruct the input image x
a
by decoding the latent represen-
tation z conditioned on the original attributes a. The learning
objective is formulated as
min
G
enc
,G
dec
L
rec
= E
x
a
∼p
data
[kx
a
− x
ˆa
k
1
], (11)
where we use the `
1
loss rather than `
2
loss to suppress
the blurriness.
Adversarial Loss. The adversarial learning between the
generator (including the encoder and decoder) and discrim-
inator is introduced to make the generated image x
ˆ
b
visually
realistic. Following WGAN [24], the adversarial losses for the
the discriminator and generator are formulated as below,