3814 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 62, NO. 6, JUNE 2015
HMM-Driven Robust Probabilistic Principal
Component Analyzer for Dynamic
Process Fault Classification
Jinlin Zhu, Zhiqiang Ge, Member, IEEE, and Zhihuan Song
Abstract—In this paper, a novel hidden Markov model
(HMM)-driven robust latent variable model (LVM) is pro-
posed for fault classification in dynamic industrial pro-
cesses. A robust probabilistic model with Student’s t
mixture output is designed for tolerating outliers. Based
on the robust LVM, the probabilistic structure is further
developed into a classifier form so as to incorporate vari-
ous types of process information during model acquisition.
After that, the robust probabilistic classifier is extended
within the HMM framework so as to characterize the time-
domain stochastic uncertainties. The model parameters are
derived through the expectation–maximization algorithm.
For performance validation, the developed model is tested
on the Tennessee Eastman benchmark process.
Index Terms—Expectation–maximization (EM), hidden
Markov model (HMM), mixture model, outliers, robust prob-
abilistic principal component analyzers, robust sequential
data modeling.
I. INTRODUCTION
P
ROCESS monitoring is of great importance for industrial
plants due to the complex manufacturing process and
costly equipment [1]. The early detection of process faults
can not only prevent potential serious damage to instruments
or the environment but also keep operators safe and help to
reduce economic loss. Traditional process monitoring methods,
represented by the model-based approach, rely on in-depth
analytical expressions of the monitored process. In most cases,
however, the detailed kinetic properties can hardly be obtained
completely [2]. As an alternative, databased process monitor-
ing methods demand little requirements for rigorous system
models, and the underlying key information can be effectively
extracted from the readily available historical data [3]. For
this reason, databased monitoring techniques have been widely
researched over the past few decades [4], [5].
Principal component analysis (PCA) and partial least squares
are probably the most popular databased monitoring methods
Manuscript received April 14, 2014; revised June 21, 2014, August
2, 2014, September 10, 2014, October 19, 2014, and November 25,
2014; accepted December 14, 2014. Date of publication January 30,
2015; date of current version May 8, 2015. This work was supported
in part by the National Natural Science Foundation of China under
Grant 61273167 and in part by the National Project 973 under Grant
2012CB720500.
The authors are with the State Key Laboratory of Industrial Control
Technology, Department of Control Science and Engineering, Zhejiang
University, Hangzhou 310027, China (e-mail: gezhiqiang@zju.edu.cn).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIE.2015.2396877
[6]. Both methods are built upon information of the steady-state
operating scenarios and characterize those internal variances
which are assumed to be different from other working condi-
tions. However, a common issue for these methods is that the
uncertainties or noise has been neglected during the modeling
phase. Recently, the probabilistic modification of PCA called
probabilistic PCA (PPCA) has been presented and applied suc-
cessfully along with the mixture formulation [mixture PPCA
(MPPCA)] in nonlinear/non-Gaussian monitoring areas [7].
Compared with the original PCA, the probabilistic based latent
variable models (LVMs) show more desirable performances,
and more importantly, the Bayesian inference method pro-
vides a unified framework for comprehensive modeling and
monitoring.
In essence, LVMs, s uch as PCA and PPCA, are all con-
structed in a static manner on the basis of normal operating
conditions. Thus, information from other operating conditions
is totally ignored. Meanwhile, one can resort to elaborately
designed indices such as T
2
and Q statistics for further fault
detection and contribution plots for diagnosis. To improve the
monitoring efficiency, some supervised modeling techniques
have been reported, such as neural networks (NNs), sup-
port vector machines (SVMs), and Gaussian mixture models
(GMMs) [8], [9]. The supervised monitoring methods can be
constructed from the whole sample space, and the mechanism
is to view the detection and diagnosis as a single classification
task. For example, the NNs and SVMs are black box models
that map the original data space into some sophisticated but
well-organized high-dimensional spaces to make the discrim-
inant analysis. Despite the feasibility, the data explanatory
abilities are beyond discussion. In contrast, GMMs conduct
the modeling directly on the sampled data set, and the global
distribution in the range can be elegantly approximated by a
finite set of local distributions. Compared with NNs and SVMs,
GMMs can be more flexible and intuitive.
Despite the appealing benefits and successful applications,
a main drawback for all the aforementioned methods is the
lack of a proper mechanism to deal with outliers, which can
be frequently encountered in practical industrial process [10].
Generally speaking, outliers can be regarded as irregular data.
Irregular data come from several aspects such as sensor failure,
network transmission errors, computer malfunction, errors in
database software, and data recording errors [11]. In many
irregular data-related studies, outliers are usually detected and
smoothed by averaging [12]. A previous study showed that the
0278-0046 © 2015
IEEE. Personal use is permitted, but republication/redistribution requires IEEE per mission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.