
Articles
https://doi.org/10.1038/s41551-018-0195-0
© 2018 Macmillan Publishers Limited, part of Springer Nature. All rights reserved. © 2018 Macmillan Publishers Limited, part of Springer Nature. All rights reserved.
1
Google Research, Google, Mountain View, CA, USA.
2
Verily Life Sciences, South San Francisco, CA, USA.
3
Division of Cardiovascular Medicine,
Stanford School of Medicine, Stanford, CA, USA.
4
These authors contributed equally: Ryan Poplin, Avinash V. Varadarajan, Lily Peng and Dale R. Webster.
*e-mail: lhpeng@google.com
R
isk stratification is central to identifying and managing groups
at risk for cardiovascular disease, which remains the leading
cause of death globally
1
. Although the availability of cardio-
vascular disease risk calculators, such as the Pooled Cohort equa-
tions
2
, Framingham
3–5
and Systematic COronary Risk Evaluation
(SCORE)
6,7
, is widespread, there are many efforts to improve
risk predictions. Phenotypic information, particularly of vascu
-
lar health, may further refine or reclassify risk prediction on an
individual basis. Coronary artery calcium is one such example,
for which it has been shown that additional signals from imaging
improve risk stratification
8
. The current standard-of-care for the
screening of cardiovascular disease risk requires a variety of vari
-
ables derived from the patient’s history and blood samples, such
as age, gender, smoking status, blood pressure, body mass index
(BMI), glucose and cholesterol levels
9
. Most cardiovascular risk
calculators use some combination of these parameters to identify
patients at risk of experiencing either a major cardiovascular event
or cardiac-related mortality within a pre-specified time period, such
as ten years. However, some of these parameters may be unavail
-
able. For example, in a study from the Practice INNovation And
CLinical Excellence (PINNACLE) electronic-health-record-based
cardiovascular registry, the data required to calculate the 10-year
risk scores were available for less than 30% of the patients
10
. This
was largely due to missing cholesterol values
10
, which is not surpris-
ing given that a fasting blood draw is required to obtain these data.
In this situation, BMI can be used in the place of lipids for a prelimi
-
nary assessment of cardiovascular health
11–13
. We therefore explored
whether additional signals for cardiovascular risk can be extracted
from retinal images, which can be obtained quickly, cheaply and
non-invasively in an outpatient setting.
Markers of cardiovascular disease, such as hypertensive reti
-
nopathy and cholesterol emboli, can often manifest in the eye.
Furthermore, because blood vessels can be non-invasively visual
-
ized from retinal fundus images, various features in the retina,
such as vessel calibre
14–20
, bifurcation or tortuosity
21
, microvascular
changes
22,23
and vascular fractal dimensions
24–26
, may reflect the sys-
temic health of the cardiovascular system as well as future risk. The
clinical utility of such features still requires further study. In this
work, we demonstrate the extraction and quantification of multiple
cardiovascular risk factors from retinal images using deep learning.
Machine learning has been leveraged for many years for a vari
-
ety of classification tasks, including the automated classification
of eye disease. However, much of the work has focused on ‘feature
engineering’, which involves computing explicit features specified
by experts
27,28
. Deep learning is a family of machine-learning tech-
niques characterized by multiple computation layers that allow an
algorithm to learn the appropriate predictive features on the basis
of examples rather than requiring features to be hand-engineered
29
.
Recently, deep convolutional neural networks—a special type of
deep-learning technique that has been optimized for images—have
been applied to produce highly accurate algorithms that diagnose
diseases, such as melanoma
30
and diabetic retinopathy
31,32
, from
medical images, with comparable accuracy to that of human experts.
Results
We developed deep-learning models using retinal fundus images
from 48,101 patients from the UK Biobank (http://www.ukbio
-
bank.ac.uk/about-biobank-uk) and 236,234 patients from EyePACS
(http://www.eyepacs.org) and validated these models using images
from 12,026 patients from the UK Biobank and 999 patients from
EyePACS (Table 1). The mean age was 56.9 ± 8.2 years on the UK
Biobank clinical validation dataset and 54.9 ± 10.9 years in the
EyePACS-2K clinical validation dataset. The UK Biobank popula
-
tion was predominantly Caucasian, while the EyePACS patients
were predominantly Hispanic. Haemoglobin A1c (HbA1c) mea
-
surements were available only in 60% of the EyePACS population.
Because this population consisted of mostly diabetic patients pre
-
senting for diabetic retinopathy screening, the mean HbA1c level of
this population was 8.2 ± 2.1%—well above the normal range. UK
Biobank participants were recruited from a UK general population,
Prediction of cardiovascular risk factors from
retinal fundus photographs via deep learning
Ryan Poplin
1,4
, Avinash V. Varadarajan
1,4
, Katy Blumer
1
, Yun Liu
1
, Michael V. McConnell
2,3
,
Greg S. Corrado
1
, Lily Peng
1,4
* and Dale R. Webster
1,4
Traditionally, medical discoveries are made by observing associations, making hypotheses from them and then designing and
running experiments to test the hypotheses. However, with medical images, observing and quantifying associations can often
be difficult because of the wide variety of features, patterns, colours, values and shapes that are present in real data. Here, we
show that deep learning can extract new knowledge from retinal fundus images. Using deep-learning models trained on data
from 284,335 patients and validated on two independent datasets of 12,026 and 999 patients, we predicted cardiovascular
risk factors not previously thought to be present or quantifiable in retinal images, such as age (mean absolute error within 3.26
years), gender (area under the receiver operating characteristic curve (AUC) = 0.97), smoking status (AUC = 0.71), systolic
blood pressure (mean absolute error within 11.23 mmHg) and major adverse cardiac events (AUC = 0.70). We also show that
the trained deep-learning models used anatomical features, such as the optic disc or blood vessels, to generate each prediction.
NATURE BIOMEDICAL ENGINEERING | VOL 2 | MARCH 2018 | 158–164 | www.nature.com/natbiomedeng
158