A Convolutional Neural Network based Single-frame
Super-Resolution for Lensless Blood Cell Counting
Xiwei Huang
1
, Yu Jiang
2
, Hang Xu
2
, Xu Liu
2
, Han Wei Hou
3
, Mei Yan
4
, and Hao Yu*
2
1
Ministry of Education Key Lab of RF Circuits and Systems, Hangzhou Dianzi University, China
2
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore
3
Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
4
Illumina Inc., San Francisco, CA, USA
*Email: haoyu@ntu.edu.sg
Abstract—This paper presents one Convolutional Neural
N
etwork based single-frame Super-Resolution processing
(CNNSR) for lensless blood cell counting, which takes one single
low-resolution cell shadow image as the input and outputs an
improved high-resolution one for better cell detection. Due to the
advantage of lightweight and fully feed-forward structure,
CNNSR is highly efficient and requires minimum resource for
hardware implementation. One lensless imaging prototype
integrating a 1.1-µm pixel-pitch back-side illuminated (BSI)
CMOS image sensor and a microfluidic channel is further
demonstrated, which shows clear detection of <2-µm platelet cells
in the blood sample solution for point-of-care diagnostics.
Keywords—convolutional neural network; super-resolution;
microfluidic cytometer; CMOS image sensor
I.
I
NTRODUCTION
Full blood count is one most commonly performed test to
provide an overview of a patient's general health status based on
cell count of each type, i.e., white blood cell (WBC), red blood
cells (RBC), and platelets [1]. The test is usually done through
an automated flow cytometer, which is however bulky and
expensive that is not suitable for point-of-care personized
diagnostics. With the recent development of microfluidics-based
lab-on-a-chip technology, miniaturized lensless microfluidic
imaging system is maturing as a competitive modality [2-3]. In
a lensless imaging system, CMOS image sensor (CIS) is
integrated beneath a microfluidic channel to record the
diffracted shadow images of cells flowing through the channel
without any magnification, as shown in Fig. 1(a). Typically, the
captured shadow images suffer from low resolution, limiting the
detection accuracy.
To overcome the resolution limitation, the image sensor
pixel pitch can be diminished to reduce undersampling. And the
distance between cells and sensor array can be decreased to
minimize the diffraction effect. For example, one can use
backside illuminated (BSI) CIS with photodiodes on top, as
illustrated in Fig. 1(b). Moreover, system-level computational
method such as super-resolution (SR) image processing has been
proposed to reconstruct high-resolution (HR) images from
captured low-resolution (LR) images. The existing SR
processing applied in lensless system is mostly multi-frame
based, in which multiple captured sub-pixel-shifted LR images
of the same object are synthesized into a single HR image [4].
The sub-pixel shifts can be generated by either flowing the cell
Metal 1
Metal 2
Top Metal
BSI
Pixel
P+
N+
P-Si Substrate
TX
N+
Pinned
Photodiode
Floating
Diffusion
Passivation
Metal 3
Metal 4
Metal 5
Cell
Lensless
Imaging
Poly
BSI CMOS Image Sensor
(b)
White Light Source Illumination
PDMS
Microfluidic
Channel
Blood
Cell
Sample
(a)
Inlet Outlet
(c)
LR Cell
Image Input
HR Cell
Image Output
Feature
Extraction
Non-linear
Mapping
Reconstruction
f
1
xf
1
f
2
xf
2
f
3
xf
3
N1 feature maps
of LR image
N2 feature maps
of HR image
Fig.1. (a) Lensless microfluidic imaging system setup, (b) BSI CMOS image
sensor pixel, (c) CNNSR processing flow.
samples through the microfluidic channel, or shifting the light
source, or sequentially activating multiple light sources at
different positions [5]. However, the main problem is that the
system needs to capture, store, and process multiple LR images
in sequence, therefore limiting the practical throughput as it
requires large storage, and is not amenable for on-chip hardware
implementation.
The development of single image SR processing algorithms
is therefore imperative. Previous work [6] introduces an on-chip
implemented single-frame SR approach that has high processing
speed by simply interpolating LR image. However, it cannot
truly improve the resolution as no fine features (high frequency
components) are recovered. Recently, learning-based
approaches have attracted more attention in which the mapping
function between LR and HR exemplar images from the training
set can be learned [2,7-8]. In [2], an extreme learning machine
(ELM) based single-frame SR is proposed to improve the
resolution but has limited accuracy for large training sets. This
work proposes a Convolutional Neural Network (CNN0 based
single-frame SR processing (CNNSR), which is lightweight,
fully feed-forward, and possible for hardware implementation,
as shown in Fig. 1(c). Furthermore, a lensless microfluidic
imaging system prototype with CNNSR is demonstrated for
blood cell detection and counting.
II. CNN-
BASED
S
INGLE
-
FRAME
S
UPER
-R
ESOLUTION
Convolutional neural network (CNN) has been widely
adopted in deep learning recently when dealing with large data
This work was supported by the NTU-ifood Grant from Singapore and
the NSFC (No. 61501156) from China.
978-1-5090-2959-4/16/$31.00 ©2016 IEEE