H. He et al. Engineering Applications of Artificial Intelligence 70 (2018) 109–122
Fig. 1. General framework of activity recognition using wavelet tensor fuzzy clustering scheme.
firstly divided into shorter time segments with the same time length
𝐿 for the useful information retrieval. Thus, there are totally 𝑉 = 𝑃 × 𝑄
sensor signals for one action sample. To facilitate distinction between
vectors, matrices and tensors, in this paper, vectors are denoted by
lowercase boldface letters (𝐚, 𝐛, …), matrices by uppercase boldface
(𝐀, 𝐁, …) and tensors by calligraphic letters (, , …). Their elements
are denoted with indices in brackets. Therefore, the dataset of hu-
man activity recognition can be expressed as 𝐗 = 𝐗
𝑚
, where 𝐗
𝑚
=
[𝐱
11
, 𝐱
12
, … , 𝐱
1𝑄
, … , 𝐱
𝑝𝑞
, … , 𝐱
𝑃 1
, 𝐱
𝑃 2
, … , 𝐱
𝑃 𝑄
]
𝑇
and 𝐗
𝑚
∈ 𝑅
𝑉 ×𝐿
, 𝑚 =
1, 2, … , 𝑀, 𝑝 = 1, 2, … , 𝑃 , 𝑞 = 1, 2, … , 𝑄. Each sensor signal is denoted
as 𝐱
𝑝𝑞
= [𝑥
1
, 𝑥
2
, … , 𝑥
𝐿
]. Table 1 lists mathematical symbols in this paper.
With wavelet packet decomposition up to 𝐽 level, every sensor
signal is decomposed into scaling coefficients in the low frequency
bands and wavelet coefficients in high frequency bands. As a result,
each signal 𝐱
𝑝𝑞
of 𝐗
𝑚
is converted into a wavelet coefficient vector
𝐰
𝑝𝑞
. A wavelet packet coefficient matrix 𝐖
𝑚
∈ 𝑅
𝑉 ×𝐿
can be obtained
for 𝐗
𝑚
, where 𝐖
𝑚
= [𝐰
11
, 𝐰
12
, … , 𝐰
1𝑄
, … , 𝐰
𝑝𝑞
, … , 𝐰
𝑃 1
, 𝐰
𝑃 2
, … , 𝐰
𝑃 𝑄
]
𝑇
.
Each row vector 𝐰
𝑝𝑞
of 𝐖
𝑚
consists of 𝐻 sub-bands of one signal,
where 𝐰
𝑝𝑞
= [𝑤
1
𝑝𝑞
, 𝑤
2
𝑝𝑞
, … , 𝑤
𝐿
𝑝𝑞
], 𝐻 = 2
𝐽
. Each sub-band length is 𝑙 and
𝑙 = 𝐿∕2
𝐽
.
It is known that the main frequency of human action signals is low.
The high-frequency band of the signal usually dominated by various
noise (Wang et al., 2007). Moreover, due to the diversity of human
activities, there are some variation in the high frequency band for the
same activity signal of different subjects or even of the same subject.
In order to undermine the adverse effects of intraclass variability of
sensor signals on the activity recognition, WP coefficients in the high
frequency domain are deleted and only 𝐻
′
(𝐻
′
< 𝐻) frequency sub-
bands are remained for pattern recognition. This truncation process
also further removes the signal noise and reduces the dimensionality
of wavelet packet coefficient vectors of sensor signals. Thus, as shown
in Fig. 2, an activity matrix sample 𝐗
𝑚
is transformed into a wavelet
packet coefficient matrix 𝐖
′
𝑚
∈ 𝑅
𝑉 ×𝐿
′
after the DWPT, where 𝐿
′
< 𝐿.
2.2. Wavelet features
Generally, the energy of a signal is regarded as one important feature
that indicates the intensity of action. But due to the interclass similarity
of activities, different activities may reveal the same amount of energy
(Bulling et al., 2014). This problem can be solved by the analysis
of WP coefficients in sub-bands. Various activities with the similar
signal energy often have different energy in different frequency sub-
bands. Therefore, computing the wavelet energy in every sub-band can
discriminate the activities with similar energy but different frequencies.
Furthermore, wavelet packet coefficients in every sub-band indicate
how closely the base wavelet and the signal correlate on the interval
of time determined by the wavelet support. Sensor signals of different
activities with same frequency are often decomposed into different
coefficients. As a result, the mean value of wavelet packet coefficients
in each frequency sub-band, i.e. 𝑤
ℎ
and the difference between 𝑤
ℎ
and
𝑤
ℎ−1
, i.e. 𝑤
𝑑
ℎ
, are also chosen as the features of one activity sample.
Suppose 𝐰
ℎ
is the wavelet packet coefficient vector of one sensor signal
𝐱
𝑝𝑞
in ℎth frequency sub-band at the 𝐽 th level. The wavelet energy of 𝐰
ℎ
can be expressed as 𝐸
ℎ
=
𝑙
𝑗=1
𝑤
𝑗
ℎ
2
, the mean value of wavelet packet
coefficients 𝑤
ℎ
=
1
𝑙
𝑙
𝑗=1
𝑤
𝑗
ℎ
and difference of wavelet packet coefficients
𝑤
𝑑
ℎ−1
= 𝑤
ℎ
− 𝑤
ℎ−1
. An activity sample 𝐗
𝑖
is converted into wavelet
feature matrix 𝐅
𝑚
= [𝐟
11
, 𝐟
12
, … , 𝐟
1𝑄
, … , 𝐟
𝑝𝑞
, … , 𝐟
𝑃 1
, 𝐟
𝑃 2
, … , 𝐟
𝑃 𝑄
]
𝑇
, where
𝐟
𝑝𝑞
= [𝐸
1
, 𝐸
2
, … , 𝐸
𝐻
′
, 𝑤
1
, 𝑤
2
, … , 𝑤
𝐻
′
, 𝑤
𝑑
1
, 𝑤
𝑑
2
, … , 𝑤
𝑑
𝐻
′
−1
].
2.3. Tensorization of wavelet feature matrix
It should be noted that 𝑄 signals obtained from one inertial sensor
unit (such as accelerometers, gyroscopes or magnetometers) record the
actions with multiple parameters (such as speed, angle and direction) in
three dimension space. Wavelet features of one sensor unit on one part of
the body can be independently used for describe the activity variation.
All signals of all body sensor units comprehensively depict an activity
from all measuring parts of the body. Therefore, if regarding signals of
111