Efficacy of a Deep Learning System for
Detecting Glaucomatous Optic Neuropathy
Based on Color Fundus Photographs
Zhixi Li, MD,
1,
* Yifan He, BS,
2,
* Stuart Keel, PhD,
3,
* Wei Meng, BS,
2
Robert T. Chang, PhD,
4
Mingguang He, MD, PhD
1,3
Purpose: To assess the performance of a deep learning algorithm for detecting referable glaucomatous optic
neuropathy (GON) based on color fundus photographs.
Design: A deep learning system for the classification of GON was developed for automated class ification of
GON on color fundus photographs.
Participants: We retrospectively included 48 116 fundus photographs for the development and validation of
a deep learning algorithm.
Methods: This study recruited 21 traine d ophthalmologists to classify the photographs. Referable GON was
defined as vertical cup-to-disc ratio of 0.7 or more and other typical changes of GON. The reference standard was
made until 3 graders achieved agreement. A separate validation dataset of 8000 fully gradable fundus photo-
graphs was used to assess the performance of this algorithm.
Main Outcome Measures: The area under receiver operator characteristic curve (AUC) with sensitivity and
specificity was applied to evaluate the efficacy of the deep learning algor ithm detecting referable GON.
Results: In the validation dataset, this deep learning system achieved an AUC of 0.986 with sensitivity of
95.6% and specificity of 92.0%. The most common reasons for false-negative grading (n ¼ 87) were GON with
coexisting eye conditions (n ¼ 44 [50.6%]), including pathologic or high myopia (n ¼ 37 [42.6%]), diabetic reti-
nopathy (n ¼ 4 [4.6%]), and age-related macular degeneration (n ¼ 3 [3.4%]). The leading reason for false-p ositive
results (n ¼ 480) was having other eye conditions (n ¼ 458 [95.4%]), mainly including physiologic cupping
(n ¼ 267 [55.6%]). Misclassification as false-positive results amidst a normal-appearing fundus occurred in only
22 eyes (4.6%).
Conclusions: A deep learning system can detect referable GON with high sensitivity and specificity. Coexis-
tence of high or pathologic myopia is the most common cause resulting in false-negative results. Physiologic cupping
and pathologic myopia were the most common reasons for false-positive results. Ophthalmology 2018;125:1199-
1206 ª 2018 by the American Academy of Ophthalmology
Glaucoma is a leading cause of irreversible blindness
worldwide.
1e3
A recent global meta-analysis of 50
population-based studies reported the pooled glaucoma
prevalence (age range, 40e80 years) to be 3.5%,
3
corresponding to an estimated 64.3 million individuals
worldwide. As a result of population growth and ageing,
this figure is expected to increase to 112 million by 2040.
3
Most vision loss resulting from glaucoma is avoidable
through early detection and treatment strategies.
4e6
Despite
this, approximately 85% of cases among the Singapore
Chinese, the same rate for African American population of
United States, and even an overall rate of 50% among the
cases in the United States are undiagnosed.
7e12
High rates of
undiagnosed disease can be attributed to chronic glaucoma
often being asymptomatic until central visual acuity is
affected in the advanced stages of disease. As glaucoma
advances from the early to late stage, care costs increase by
4-fold, posing a significant financial burden wordwide.
13
The assessment of the optic disc and retinal nerve fiber
layer (RNFL) are the foundation of glauco ma diagnosis,
although a dilated clinical fundus examination after mydri-
asis has been recommended for its advantage of offering a
stereoscopic view of the optic disc.
14
However, monoscopic
optic disc photographs offer some key advantages, including
convenience and affordability. Furthermore, the
Glaucomatous Optic Neuropathy Evaluation project
demonstrated that subjective assessments of monoscopic
optic disc photographs provide an equal diagnostic
accuracy for glaucoma when c ompared with stereoscopic
photographs.
15
Nevertheless, manual assessment of the
optic disc is labor intensive and highly dependent on
image interpretation by trained specialists. This
significantly impacts the cost effectiveness of glaucoma
screening,
16e18
and as a result, glaucoma screening strate-
gies are not widely implemented in the general
population.
18e20
1199ª 2018 by the American Academy of Ophthalmology
Published by Elsevier Inc.
https://doi.org/10.1016/j.ophtha.2018.01.023
ISSN 0161-6420/18