
Performance Evaluation of Deep Learning Tools in
Docker Containers
Pengfei Xu
Department of Computer Science
Hong Kong Baptist University
Email: pengfeixu@comp.hkbu.edu.hk
Shaohuai Shi
Department of Computer Science
Hong Kong Baptist University
Email: csshshi@comp.hkbu.edu.hk
Xiaowen Chu
Department of Computer Science
Hong Kong Baptist University
Email: chxw@comp.hkbu.edu.hk
Abstract—With the success of deep learning techniques in
a broad range of application domains, many deep learning
software frameworks have been developed and are being updated
frequently to adapt to new hardware features and software
libraries, which bring a big challenge for end users and system
administrators. To address this problem, container techniques are
widely used to simplify the deployment and management of deep
learning software. However, it remains unknown whether con-
tainer techniques bring any performance penalty to deep learning
applications. The purpose of this work is to systematically
evaluate the impact of docker container on the performance of
deep learning applications. We first benchmark the performance
of system components (IO, CPU and GPU) in a docker container
and the host system and compare the results to see if there’s any
difference. According to our results, we find that computational
intensive jobs, either running on CPU or GPU, have small
overhead indicating docker containers can be applied to deep
learning programs. Then we evaluate the performance of some
popular deep learning tools deployed in a docker container and
the host system. It turns out that the docker container will not
cause noticeable drawbacks while running those deep learning
tools. So encapsulating deep learning tool in a container is a
feasible solution.
I. INTRODUCTION
Ever since the great success of deep learning techniques
in many application domains, more and more deep learning
software tools have been developed by different research
institutions and companies for both academic research and
commercial use [1]. Popular tools like Caffe [2], CNTK [3],
MXNet [4], TensorFlow [5], Torch [6], etc. are still being
actively developed and their new versions are being released
frequently, which brings significant software management
challenge to system administrators. It is even worse when
different tools or different versions of the same tool need to
be installed in a system that is shared by multiple users. A
practical solution to simplify the management of deep learning
tools is to make use of docker containers so that environmental
setting conflicts can be easily resolved by packaging a software
and its all required libraries into a single image [7]. Despite its
popularity in practical usage, there lacks a systematic analysis
on the performance overhead brought by docker containers for
deep learning tools. This paper aims to investigate the impact
of docker containers on the performance of deep learning tools.
A typical deep learning training workflow involves data
access from/to disk drives and intensive data processing on
CPUs and/or accelerators such as GPUs [8]. Therefore we
evaluate the performance of CPU, GPU, disk I/O, and the
overall deep learning training with and without dock container,
respectively. For CPU performance, we make use of two
classical and representative benchmarks, HPL and HPCG. For
GPU performance, a set of GPU programs are selected to test
different types of GPU operations. Disk I/O performance can
be another important factor when huge amount of data are
fed to neural networks during training process [9]. We test
I/O performance from several aspects, including I/O access
latency, random access throughput, and sequential access
throughput. At last, we evaluate the training performance of
five popular deep learning software tools with different neural
network models and datasets. Based on our experimental re-
sults, we find that docker containers have negligible overhead
in computing-intensive tasks on both CPU and GPU. The I/O
performance of sequential access under the docker container
is found to be at the same level as the host system. When
it comes to random access, we observe even shorter response
time on docker container than on the host system using one
of the tested disk drives. This is because docker containers
can make better use of the NAND cache on the hard disk to
gain faster random data access. Since each factor mentioned
above has satisfactory results on docker, it is not surprising
to find that running deep learning tools in docker containers
has negligible overhead compared to running on host systems
directly.
This paper is organized as follows. Background of deep
learning and docker containers and related work are introduced
in Section II. The design of our experiments is presented in
Section III. We show our experimental results and analysis in
Section IV. We conclude our work in Section V.
II. BACKGROUND
A. Deep Learning
Deep learning is a class of machine learning techniques
which powers great number of facets in our everyday life.
Deep neural networks are built of many processing layers
and are able to learn features from a mass of data with
various stages of abstraction [9]. This technology has many
applications like speech recognition [10] [11] [12] [13], image
recognition [14] [15] [16] [17], natural language processing
[18] [19] [20], and the list is getting longer and longer.
Comparing with conventional machine leaning techniques,
arXiv:1711.03386v1 [cs.DC] 9 Nov 2017