DISCRIMINATIVE SEMI-MARKOV MODELS FOR AUTOMATED MITOTIC PHASE
LABELLING
A. El-Labban, A. Zisserman
∗
Department of Engineering Science,
University of Oxford,
United Kingdom
Y. Toyoda, A.W. Bird, A. Hyman
Max Planck Institute of Molecular
Cell Biology and Genetics,
Dresden, Germany
ABSTRACT
With the widespread use of time-lapse data to understand cel-
lular function, there is a need for tools which facilitate high-
throughput analysis of data. We present a system for auto-
mated segmentation and mitotic phase labelling based on a
wide margin discriminative Semi-Markov Model. This work
takes the novel approach of using temporal features evaluated
over the whole of the mitotic phases rather than over single
frames, thereby capturing the distinctive behaviour over the
phases. This approach extends and substantially improves on
our previous approach of using dynamic time warping to align
temporal feature signals to a reference.
1. INTRODUCTION
In the field of cell biology, there is an increasing use of time-
lapse data to understand cellular function. Using automated
microscopes, large numbers of images can be acquired, deliv-
ering 3D videos of cell samples over time. Analysing the im-
ages manually is extremely time consuming as there are typ-
ically thousands of individual images in any given sequence.
Additionally, decisions made by those analysing the images,
e.g. labelling a cell cycle stage, can be subjective, especially
around transition boundaries between stages, leading to in-
consistencies in the annotation. There is therefore a need for
tools which facilitate automated high-throughput analysis.
The objective of this paper is to automatically identify and
track the individual cells throughout a time-lapse sequence,
and to label the mitotic phase for each cell at every time point.
Over large volumes of data, such outputs can be used to derive
statistics on cellular function.
Existing approaches to mitotic phase labelling follow a
number of common stages [1, 2, 3]: frames are first seg-
mented into individual cells using image processing opera-
tions such as adaptive thresholding and watershed algorithms,
and the cells are then tracked throughout the sequence. To
classify the cell phase, each segmented cell is described by
a feature vector which can include the shape, size, intensity
∗
This work was supported by EU ERC grant VisRec no. 228180.
(max, min, mean, variance) and texture. A prediction of the
cell phase for each frame is then obtained by classifying the
feature vector, e.g. by using a support vector machine (SVM).
A final labelling can then be obtained using a temporal Hid-
den Markov Model (HMM) [4] across the track to correct the
individual frame predictions. We follow the standard seg-
mentation and tracking stages of this framework here (Sec-
tion 2). The novelty in this work is that, instead of attempt-
ing to label the cell phase for each frame independently, the
cell cycle stages are labelled using a discriminative Semi-
Markov Model (SMM) using features evaluated over tempo-
ral segments of cell tracks. By evaluating features over seg-
ments rather than on a per frame basis, the SMM model is
able to capture the distinctive behaviour during mitoic phases,
as illustrated in Figure 1. This immediately produces seg-
ments with contiguous labellings without the need for the
two-stage approach of individual frame labelling smoothed
with an HMM. It also avoids the dependence on the unreal-
istic exponentially decaying duration model of HMMs. This
is particularly useful as under many experimental conditions
the duration of mitotic stages is altered, but the appearance
of the chromatin is not. Detecting these changes can be key
to elucidating insights into where a particular defect is occur-
ring. This approach significantly improves on our previous
work [5], which used dynamic time warping (DTW) to map
temporal signals to labelled references to obtain labelled seg-
ments. The learnt SMM provides a richer model which out-
performs the less general approach of DTW.
2. DATA & PREPROCESSING
Data. For the purposes of this paper, we use the MitoPhase
dataset of [5]. This dataset consists of 54 3D time-lapse image
sequences of HeLa (human epithelial adenocarcinoma) cells
containing a fluorescently tagged chromatin marker, histone
H2B-mCherry, acquired with a 60× microscope objective. Of
these sequences, 23 were obtained under conditions in which
one of three proteins required for timely progression though
mitosis (TACC3, CLTC, or GTSE1) were depleted from cells
by RNAi. The videos have a temporal spacing of either 1