Contents lists available at ScienceDirect
Neurocomputing
journa l homepa ge: www.elsevier.com/locate/neucom
A parameter selection method of the deterministic anti-annealing algorithm
for network exploring
Bianfang Chai
a,c,
⁎
, Jinghong Wang
b
, Jian Yu
c
a
Department of Information Engineering, Hebei GEO University, Hebei 050031, China
b
College of Information Technology, Hebei Normal University, Shijiazhuang 050024, China
c
Beijing Key Lab of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing 100044, China
ARTICLE INFO
Communicated by Zidong Wang
Keywords:
Mixture model
Community detection
Deterministic anti-annealing EM algorithm
Convergence rate
Jacobian matrix
ABSTRACT
The traditional expectation maximization (EM) algorithm for the mixture model can explore the structural
regularities of a network efficiently. But it always traps into local maxima. A deterministic annealing EM
(DAEM) algorithm is put forward to solve this problem. However, it brings about the problem of convergence
speed. A deterministic anti-annealing expectation maximization (DAAEM) algorithm not only prevents poor
local optima, but also improves the convergence speed. Thus, the DAAEM algorithm is used to estimate
parameters of the mixture model. This algorithm always sets its initial parameter β
0
by experience, which maybe
get trapped into meaningless results due to too small β
0
, or converge to local maxima more frequently due to too
large β
0
. A parameter selection method for β
0
is designed. In our method, the convergence rate of the DAAEM
algorithm for mixture model is first derived from Jacobian matrix of the posterior probabilities. Then the
theoretical lower bound of β
0
is computed based on the convergence rate at meaningless points. In our
experiments we select β
0
by rounding up the lower bound to the nearest tenth. Experiments on real and
synthetic networks demonstrate that the parameter selection method is valid, and the performance of the
DAAEM algorithm beginning from the selected parameter is better than the EM and DAEM algorithms for
mixture model. In addition, we find that the convergence rate of the DAAEM algorithm is affected by assortative
mixing by degree of a network.
1. Introduction
Networks have gained significant attention for representing com-
plex systems, and analyzing them helps us understand the systems.
When we have no prior on networks, it is necessary to analyze them by
automatics. Many analysis techniques for these networks have emerged
in the past few years, and community detection [1] is a popular one. It
has become useful for many reasons, such as suppressing the complex-
ity of the whole network and identifying the key nodes in networks, etc.
Up to now, a huge amount of methods for the task of community
detection have been developed, including hierarchical clustering,
divisive clustering, modularity-based methods, etc. They just focus on
detecting tightly connected subgraphs. But complex networks may have
many other types of structures, including core-periphery, hierarchical,
multipartite structures, or the mixture of them, etc. Recently, some
models have been provided to detect a more wide variety of structures
besides tightly connected subgraphs. These models are mainly classi-
fied by two categories. One category is ones based on the stochastic
block model (SBM) [2], whose algorithms estimate parameters by the
Gibbs sampling method [3], the variational EM algorithm [4–6], the
variational Bayes methods [7], the belief propagation method [8], etc.
The time complexities of these algorithms are approximately
mc(
2
,
where m and c respectively denote the number of edges and clusters.
The other category is mixture model [9] for network exploring, whose
time complexity is
mc(
. By contrast, the EM algorithm for mixture
model (EMMM) [9] is more efficient than the algorithms for models
based on the SBM.
However, it is well known that the traditional EM algorithm always
converges to poor local maxima. The DAEM algorithm [10] has been
provided to overcome local maximum problem. It starts with
β=≃0
0
and slowly increases β to 1. At each β, the DAEM algorithm
executes the EM algorithm. This increases the convergence time,
especially on data with skewed mixing coefficients and large overlap
among clusters. The anti-annealing EM algorithm [11] starts from
>
and slowly decreases it down to 1. It improves the speed by
restricting the amount of overlaps, but trends to converge to poor local
optima more frequently. The DAAEM algorithm [11] not only prevents
the EM algorithm from getting trapped into local optima, but also
http://dx.doi.org/10.1016/j.neucom.2016.11.050
Received 14 September 2015; Received in revised form 16 August 2016; Accepted 27 November 2016
⁎
Corresponding author at: Department of Information Engineering, Hebei GEO University, Hebei 050031, China.
E-mail address: chaibianfang@163.com (B. Chai).
Neurocomputing 226 (2017) 192–199
Available online 30 November 2016
0925-2312/ © 2016 Elsevier B.V. All rights reserved.
MARK