2 Method
In this section, the framework and method proposed for cell
tracking and mitosis detection are introduced.
Framework Our deep learning-based framework is shown
in Fig. 1. It consists of two main parts: a sampling pro-
cess (gre en) and the TDNNs. The TDNNs are com posed of
a tracking part (blue), detection part (red), and convolu-
tional layers. The target stem cell is only marked out using
a bounding box at the start frame, and then the system
tracks the cell and detects its mitosis automatically. This
bounding box covering the target cell is denoted as
z ¼fx; y; h; lg. Where fx; yg denotes the center coordinate
of the box, and fh; lg denotes the height and width of the
box. Subsequently, each incoming frame of the cell image
sequence is processed as follows:
1. First, z is obtained from the previous frame (contains
image and position information of the target cell) , and
a particle filter-based random sampling algorithm is
applied to the current frame to generate n candidate
position set Z ¼ z
1
; z
2
; ...; z
n
;
2. The cell tracking part of the network confirms the
target cell region z
0
from candidate positions set Z in
the current frame. Set z ¼ z
0
;
3. The mitosis detection part of the networks detects
whether the target cell is splitting; and
4. Process the next frame and return to step 1.
After step 2, the target cell is chosen from candidat es in
the current frame, while step 3 establishes whether the
target cell is undergoing mitosis. The framework repeats
this process frame by frame to track the cell and detect its
mitosis simultaneously. Additional details are as follows:
2.1 Initialization
In the initialization phase, the original position of the target
cell z
init
, which is denoted with a bounding box, is initial-
ized manually at the beginning frame (shown in Fig. 1).
2.2 Particle filter-based method
The particle filter approach [25], used during the tracking
process for sampling as shown in green in Fig. 1,isa
sequential Bayesian estimation approach that recursively
infers the hidden state of the target. It is the dominant
approach in visual tracking. Mathematically, tracking aims
at solving the problem of verifying the most probable state
of the target at each time step t based on the observation
results of the previous time step t 1[26]:
s
t
¼ arg max ps
t
jy
1:t1
¼ arg max
Z
ps
t
js
t1
ps
t1
jy
1:t1
ds
t1
:
ð1Þ
where s
t
and y
t
denote latent state and observation vari-
ables at time t, respectively. Bayes rule is used to update
the posterior distribution of the state variable here when a
new observation y
t
arrives:
ps
t
jy
1:t
¼
py
t
js
t
ðÞps
t
jy
1:t1
ðÞ
py
t
jy
1:t1
ðÞ
:
ð2Þ
Particle filter approximates the true poster state distribu-
tion ps
t
jy
1:t1
ðÞby a set of n sample s, called part icles ,
fs
t
i
g
n
i¼1
with corresponding importance weights fw
t
i
g
n
i¼1
which sum to 1. Particles are drawn from an im portance
distribution qs
t
js
1:t1
; y
1:t
ðÞ. For cell tracking, the state
variable s represents the position of the cell, which is
denoted as z ¼fx; y; h; lg. Here, a normal distribution is
proposed to model each dimension of qs
t
js
t1
ðÞindepen-
dently. The particle filter is used as a moti on model for
trackers to generate the probable states of the target at
each time step. The tracking result is the particle wi th the
largest probability output determined by different obser-
vation models, which the deep neural network was used
for this work.
For the cell tracking task, as shown in Fig. 2, the particle
filter algorithm is used for sampling the candidate positions
Candicated
posions
Top K
Paral filter
based
sampling
CNNs
Tracking
part
Detecon
part
Top K
TDNNs
Paral filter
based
sampling
Fig. 1 Overview of the proposed framework. Blue: tracking part;
green: particle filter-based sampling process; and red: mitosis
detection part (color figure online)
TDNNs
Top K
Partial filter
…
…
Partial filter
observational model
Fig. 2 Particle filter method in tracking phase
Neural Computing and Applications
123