Modern Code Review: A Case Study at Google
Caitlin Sadowski, Emma Söderberg,
Luke Church, Michal Sipko
Google, Inc.
{supertri,emso,lukechurch,sipkom}@google.com
Alberto Bacchelli
University of Zurich
bacchelli@ifi.uzh.ch
ABSTRACT
Employing lightweight, tool-based code review of code changes
(aka modern code review) has become the norm for a wide
variety of open-source and industrial systems. In this pa-
p er, we make an exploratory investigation of modern code
review at Google. Google introduced code review early on
and evolved it over the years; our study sheds light on why
Go ogl e introduced this practice and analyzes its current
status, after the process has been refined through decades of
co de changes and millions of code reviews. By means of 12
interviews, a survey with 44 respondents, and the analysis
of review logs for 9 million reviewed changes, we investigate
motivations behind code review at Google, current pract ices,
and developers’ satisfaction and challenges.
CCS CONCEPTS
• Software and its engineering → Software maintenance tools
;
ACM Reference format:
Caitlin Sadowski, Emma Söderberg,
Luke Church, Michal Sipko and Alberto Bacchelli. 2018. Modern
Code Review: A Case Study at Google. In Proceedings of 40th
International Conference on Software Engineering: Software En-
gineering in Practice Track, Gothenburg, Sweden, May 27-June 3,
2018 (ICSE-SEIP ’18), 10 pages.
DOI: 10.1145/3183519.3183525
1 INTRODUCTION
Peer code review, a manual inspection of source code by
developers other than the author, is recognized as a valuable
to ol for improving the quality of software projects [
2
,
3
]. In
1976, Fagan formalized a highly structured process for code
reviewing—code inspections [
16
]. Over the years, researchers
provided evidence on the benefits of code inspection, espe-
cially for defect finding, but the cumbersome, time-consuming,
and synchronous nature of this appro ach hindered its uni-
versal adoption in practice [
37
]. Nowadays, most organiza-
tions adopt more lightweight code review practices to limit
the inefficiencies of inspections [
33
]. Modern code review is
(1) informal (in contrast to Fagan-style), (2) tool-based [
32
],
(3) asynchronous, and (4) focused on reviewing code changes.
Permission to make digital or hard copies of part or all of this work
for personal or classroom use is granted without fee provided that
copies are not made or distributed for profit or commercial advantage
and that copies bear this notice and the full citation on the first page.
Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
ICSE-SEIP ’18, Gothenburg, Sweden
©
2018 Copyright held by the owner/author(s). 978-1-4503-5659-
6/18/05. . . $15.00
DOI: 10.1145/3183519.3183525
An open research challenge is understanding which prac-
tices represent valuable and effective methods of review in this
novel context. Rigby and Bird quantitativel y analyzed code
review data from software projects spanning varying domains
as well as organizations and found five strongly convergent
aspect s [
33
], which they conjectured can be prescriptive to
other projects. The analysis of Rigby and Bird is based on the
value of a broad perspective (that analyzes multiple projects
from different contexts). For the development of an empirical
b ody of knowledge, championed by Basili [
7
], it is essential
to also consider a focused and longitudinal perspective that
analyzes a single case. This paper expands on work by Rigby
and Bird to focus on the review practices and characteristics
established at Google, i.e., a company with a multi-decade
history of code review and a high-volume of daily reviews to
learn from. This paper can be (1) prescriptive to practitioners
p erforming code review and (2) compelling for researchers
who want to understand and support this novel process.
Co de review has been a required part of software develop-
ment at Google since very early on in the company’s history;
b ecause it was introduced so early on, it has become a core
part of Google culture. The process a nd tooli ng for code
review at Google have been iteratively refined for more than
a decade and is applied by more than 25,000 developers
making more than 20,000 source code changes each workday,
in dozens of offices around the world [30].
We conduct our analysis in the form of an exploratory
investigation focusing on three aspects of code review, in line
with and expanding on the work by Rigby and Bi rd [
33
]:
(1) The motivations driving code review, (2) the current
practices, and (3) the perception of developers on code review,
fo cusing on challenges encountered with a specific review
(breakdowns in the review process) and satisfacti on. Our
research method combines input from multiple data sources:
12 semi-structured interviews with Google developers, an
internal survey sent to engineers who recently sent changes
to review with 44 responses, and log data from Google’s code
review tool pertaining to 9 million reviews over two years.
We find that the process at Google is markedly lighter
weight than in other contexts, based on a single reviewer,
quick i terations, small changes, and a tight integration with
the code review tool. Breakdowns still exist, however, due to
the complexity of the interactions that occur around code re-
view. Nevertheless, developers consider this process valuable,
confirm that it works well at scale, and conduct it for several
reasons that also depend on the relationship between author
and reviewers. Finally, we find evidence on the use of the code
review tool beyond collaborative review and corroboration
for the importance of code review as an educational tool.