Comparison and Combination of State-of-the-art
Techniques for Handwritten Character Recognition:
Topping the MNIST Benchmark
Daniel Keysers
keysers@iupr.net
IUPR Research Group
German Research Center for Artificial Intelligence
and Technical University Kaiserslautern
May 2006
Abstract
Although the recognition of isolated handwritten digits has been a re-
search topic for many years, it continues to be of interest for the research
community and for commercial applications. We show that despite the
maturity of the field, different approaches still deliver results that vary
enough to allow improvements by using their combination. We do so
by choosing four well-motivated state-of-the-art recognition systems for
which results on the standard MNIST benchmark are available. When
comparing the errors made, we observe that the errors made differ be-
tween all four systems, suggesting the use of classifier combinaiton. We
then determine the error rate of a hypothetical system that combines the
output of the four systems. The result obtained in this manner is an error
rate of 0.35% on the MNIST data, the best result published so far. We
furthermore discuss the statistical significance of the combined result and
of the results of the individual classifiers.
1 Introduction
The recognition of handwritten digits is a topic of practical importance because
of applications like automated form reading and handwritten zip-code process-
ing. It is also a subject that has continued to produce much research effort over
the last decades for several reasons:
• The problem is prototypical for image processing and pattern recognition,
with a small number of classes.
• Standard benchmark data sets exist that make it easy to obtain valid
results quickly.
1