Multi-Dimensional Classification via Sparse Label Encoding
Bin-Bin Jia
1 2
Min-Ling Zhang
1 3
Abstract
In multi-dimensional classification (MDC), there
are multiple class variables in the output space
with each of them corresponding to one hetero-
geneous class space. Due to the heterogeneity
of class spaces, it is quite challenging to consid-
er the dependencies among class variables when
learning from MDC examples. In this paper, we
propose a novel MDC approach named SLEM
which learns the predictive model in an encoded
label space instead of the original heterogeneous
one. Specifically, SLEM works in an encoding-
training-decoding framework. In the encoding
phase, each class vector is mapped into a real-
valued one via three cascaded operations includ-
ing pairwise grouping, one-hot conversion and
sparse linear encoding. In the training phase, a
multi-output regression model is learned within
the encoded label space. In the decoding phase,
the predicted class vector is obtained by adapting
orthogonal matching pursuit over outputs of the
learned multi-output regression model. Experi-
mental results clearly validate the superiority of
SLEM against state-of-the-art MDC approaches.
1. Introduction
In traditional supervised learning, the semantics of objects
are usually characterized by only one output variable, e.g.,
multi-class classification. However, in some real-world ap-
plications, the semantics of objects need to be characterized
along different dimensions. For example, the e-commerce
websites should categorize laptops from different dimen-
sions (e.g., brand, operating system, CPU, GPU, etc.) to
make it more convenient for consumers to choose the right
laptop for themselves. In fact, similar requirements widely
1
School of Computer Science and Engineering, Southeast Uni-
versity, Nanjing 210096, China
2
College of Electrical and Infor-
mation Engineering, Lanzhou University of Technology, Lanzhou
730050, China
3
Key Lab. of Computer Network and Information
Integration (Southeast University), Ministry of Education, China.
Correspondence to: Min-Ling Zhang <zhangml@seu.edu.cn>.
Proceedings of the
38
th
International Conference on Machine
Learning, PMLR 139, 2021. Copyright 2021 by the author(s).
exist in various fields, e.g., text classification (Shatkay et al.,
2008), bioinformatics (Rodr
´
ıguez et al., 2012), resource al-
location (Al Muktadir et al., 2019), ecology (Verma et al.,
2021), etc. These special applications can be naturally for-
malized under the multi-dimensional classification (MDC)
learning framework (Read et al., 2014a; Ma & Chen, 2018;
Jia & Zhang, 2020a; Wang et al., 2020). In MDC, each
example is represented by a single instance while associat-
ed with multiple class variables. Here, each class variable
corresponds to one specific class space which characterizes
the semantics of objects from one dimension.
Formally speaking, let
X = R
d
be the input (feature) s-
pace, and
Y = C
1
× C
2
× ··· × C
q
be the output space
which corresponds to the Cartesian product of
q
class s-
paces. Here, each class space
C
j
(1 ≤ j ≤ q)
consists
of
K
j
possible class labels, i.e.,
C
j
= {c
j
1
, c
j
2
, . . . , c
j
K
j
}
.
Given the MDC training set
D = {(x
i
, y
i
) | 1 ≤ i ≤ m}
with
m
training examples, for each example
(x
i
, y
i
) ∈ D
,
x
i
= [x
i1
, x
i2
, . . . , x
id
]
>
∈ X
is a
d
-dimensional feature
vector and y
i
= [y
i1
, y
i2
, . . . , y
iq
]
>
∈ Y is the class vector
associated with
x
i
, where each component
y
ij
is one possi-
ble item in
C
j
, i.e.,
y
ij
∈ C
j
. The MDC task aims to learn
a predictive model
f : X 7→ Y
from
D
which can return a
proper class vector f(x) ∈ Y for unseen instance x.
To solve the MDC problem, we can independently deal with
each dimension which is actually a multi-class classifica-
tion problem. Nonetheless, this strategy does not consider
potential dependencies among class spaces which would de-
generate its generalization ability. Therefore, most existing
MDC studies focus on how to model class dependencies
more appropriately, e.g., specifying a chaining structure over
class variables (Zaragoza et al., 2011; Read et al., 2014b),
partitioning class spaces into several groups (Read et al.,
2014a), learning a direct acyclic graph structure over class
variables (Bielza et al., 2011; Gil-Begue et al., 2021), etc.
Due to the heterogeneity of class spaces, it is quite chal-
lenging to directly consider the dependencies among class
variables in the original output space as most existing MDC
approaches do. In this paper, we attempt to learn the pre-
dictive model which solves the MDC problem in its trans-
formed label space. Accordingly, we propose a novel ap-
proach named SLEM, i.e., Sparse Label Encoding for Multi-
dimensional classification, which works in an encoding-