
Automated Cross-Browser Compatibility Testing
Ali Mesbah
∗
Electrical and Computer Engineering
University of British Columbia
Vancouver, BC, Canada
amesbah@ece.ubc.ca
Mukul R. Prasad
Trusted Systems Innovation Group
Fujitsu Laboratories of America
Sunnyvale, CA, USA
mukul.prasad@us.fujitsu.com
ABSTRACT
With the advent of Web 2.0 applications and new browsers,
the cross-browser compatibility issue is becoming increas-
ingly important. Although the problem is widely recognized
among web developers, no systematic approach to tackle
it exists today. None of the current tools, which provide
screenshots or emulation environments, specifies any notion
of cross-browser compatibility, much less check it automat-
ically. In this paper, we pose the problem of cross-browser
compatibility testing of modern web applications as a ‘func-
tional consistency’ check of web application behavior across
different web browsers and present an automated solution
for it. Our approach consists of (1) automatically analyzing
the given web application under different browser environ-
ments and capturing the behavior as a finite-state machine;
(2) formally comparing the generated models for equivalence
on a pairwise-basis and exposing any observed discrepan-
cies. We validate our approach on several open-source and
industrial case studies to demonstrate its effectiveness and
real-world relevance.
Categories and Subject Descriptors
D.2.5 [Software Engineering]: Testing and Debugging
General Terms
Reliability, Verification
Keywords
Dynamic analysis, web testing, cross-browser compatibility
1. INTRODUCTION
Web applications pervade all aspects of human activity to-
day. The web browser continues to be the primary gateway
channeling the end-user’s interaction with the web applica-
tion. It has been well known for some time that different
∗
This author was a visiting researcher at Fujitsu Laborato-
ries of America, when this work was done.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
ICSE ’11, May 21–28, 2011, Waikiki, Honolulu, HI, USA
Copyright 2011 ACM 978-1-4503-0445-0/11/05 ...$10.00.
web browsers render web content somewhat differently [18,
24, 25, 26]. However, the scope and impact of this problem
has been rapidly growing due to two, fairly recent trends.
First, modern, rich-content web applications have a heavy
client-side behavioral footprint, i.e., they are designed to ex-
ecute significant elements of their behavior exclusively on the
client-side, typically within the web browser. Further, tech-
nologies such as Ajax [12], Flash, and event-handling for
dynamic HTML, which support this thick-client behavior,
are the very aspects in which web browsers differ.
Second, recent years have seen an explosion in the num-
ber of available web browsers. There are nearly 100 different
web browsers available today [31]. Coupled with the differ-
ent kinds and versions of operating systems in which these
operate, this yields several hundred different client-side en-
vironments in which a web application can be used, each
with a slightly different client-side rendering of the applica-
tion [7, 9]. In the context of the above problem, the key
contributions of this paper are:
• We define the cross-browser compatibility problem and
present an in-depth discussion of the issues surround-
ing possible solutions to it.
• We present a systematic, fully-automated solution for
cross-browser compatibility testing that can expose a
substantial fraction of the cross-browser issues in mod-
ern dynamic web applications.
• We validate our approach on several open-source as
well as industrial case studies to demonstrate its effi-
cacy and real-world relevance.
The rest of the paper is organized as follows. In the next
section we discuss the genesis of the cross-browser compati-
bility problem and how it plays out in modern web applica-
tions. We present a formal characterization of the problem
and put the solutions offered by this paper in that context.
Section 3 reviews related work on cross-browser testing. In
Section 4 we present our proposed approach for performing
cross-browser compatibility testing, followed by a descrip-
tion of the salient implementation aspects of that approach,
in Section 5. Section 6 presents an empirical evaluation of
our approach on some open-source as well as industrial case
studies. In Section 7 we discuss the strengths and weak-
nesses of our solution and lessons learnt from our experience.
We conclude the paper in Section 8.
2. CROSS-BROWSER COMPATIBILITY
Problem Genesis. The cross-browser compatibility
(CBC) problem is almost as old as the web browser it-
self. There are several reasons for its genesis and growth
in recent years. The earliest instances of this problem had