Refactoring the FreeBSD Kernel with Checked C
Junhan Duan,
∗
Yudi Yang,
∗
Jie Zhou, and John Criswell
Department of Computer Science
University of Rochester
Abstract—Most modern operating system kernels are written
in C, making them vulnerable to buffer overflow and buffer
over-read attacks. Microsoft has developed an extension to the
C language named Checked C which provides new source
language constructs that allow the compiler to prevent NULL
pointer dereferences and spatial memory safety errors through
static analysis and run-time check insertion. We evaluate the use
of Checked C on operating system kernel code by refactoring
parts of the FreeBSD kernel to use Checked C extensions. We
describe our experience refactoring the code that implements
system calls and UDP and IP networking. We then evaluate the
refactoring effort and the performance of the refactored kernel.
It took two undergraduate students approximately three months
to refactor the system calls, the network packet (mbuf) utility
routines, and parts of the IP and UDP processing code. Our
experiments show that using Checked C incurred no performance
or code size overheads.
Index Terms—memory safety, safe C, FreeBSD
I. INTRODUCTION
Most modern operating system (OS) kernels, such as
Linux [14] and FreeBSD [39], are implemented in C. However,
due to C’s lack of memory safety and type safety, OS kernels
written in C can have exploitable memory safety errors. These
memory safety errors may lead to the kernel having denial of
service (DoS) [3], [4], [6], [8], privilege escalation [4], [8], and
arbitrary code execution [6], [7] vulnerabilities. Memory safety
errors in OS kernels are extremely dangerous as kernels form
the foundation of the entire software stack; successful exploita-
tion of kernel memory safety errors can lead to memory cor-
ruption and information disclosure [16]. Such attacks can even
bypass defenses such as Data Execution Prevention (DEP) [2]
and Address Space Layout Randomization (ASLR) [19], [41].
To mitigate these vulnerabilities, one could rewrite the
OS kernel in a safe language; examples include Singular-
ity [30] (Sing# [26]), Mirage [38] (OCaml), Redox [24] and
Tock [36] (Rust), and Biscuit [23] (Go). However, rewriting
the OS kernel requires significant programmer effort. Another
approach is to use automatic compiler transformations to
enforce memory safety [21], [22] or Control Flow Integrity
(CFI) [9]. Memory safety transformations on kernel code
have high overhead (some kernel latency increase by as much
as 35× [21]) while CFI approaches [20], [27], [29] fail to
mitigate non-control data attacks [17].
Checked C [25], [44] is a new safe C dialect from Microsoft
that extends the C language with new pointer types for which
the compiler automatically performs NULL pointer checks and
array bounds checks; the compiler either proves that use of the
∗
Junhan Duan and Yudi Yang are co-primary authors.
pointer is safe at compile time or inserts run-time checks to
ensure safety at run-time. Checked C has low performance
overhead (only 8.6% on selected benchmarks [25]), and its
spatial safety checks mitigate both control-data [43], [48] and
non-control data [17] attacks.
In this paper, we investigate the use of Checked C in
refactoring parts of an OS kernel to be spatially memory
safe. Specifically, we quantify the performance and code size
overheads of using Checked C on OS kernel code and identify
challenges and solutions to using Checked C on OS kernel
code. We chose FreeBSD 12 stable to harden with Checked C
as FreeBSD uses Clang and LLVM [35] as its default compiler;
Checked C is based on Clang and LLVM [25]. Since there are
millions of lines of code inside the FreeBSD kernel, we refac-
tored kernel components that we think are part of the kernel’s
attack surface. Specifically, we refactored code that copies
data between application memory and kernel memory i.e.,
code using the copyin() and copyout() functions [5],
as such code, if incorrect, would allow applications to read
or corrupt the OS kernel’s memory. We also refactored code
in the TCP/IP stack as the TCP/IP stack processes data from
untrusted networks. Specifically, we modified the mbuf utility
routines [46] which manipulate packet headers and parts of the
UDP and IP processing code.
The rest of this paper is organized as follows: Section II
provides background on Checked C. Section III describes the
threat model. Section IV discusses how we refactored the
FreeBSD kernel to use Checked C features. Section V dis-
cusses our refactoring efforts. Section VI presents the results of
our evaluation on the refactored FreeBSD kernel. Section VII
discusses related work, and Section VIII concludes.
II. BACKGROUND ON CHECKED C
Checked C [25], [44] is a new safe C dialect. Checked C’s
design distinguishes itself from previous safe-C works [18],
[32], [34] in that it enables programmers to easily convert
existing C code into safe Checked C code incrementally, and it
maintains high backward compatibility. Currently, Checked C
provides no temporal memory safety protections, so our work
does not discuss temporal memory safety. In this section,
we briefly introduce Checked C’s key features: new types of
pointers (Section II-A), checked regions (Section II-B), and
bounds-safe interfaces (Section II-C).
A. New Pointer Types
Checked C extends C with three new types of pointers
for spatial memory safety: ptr<T>, array_ptr<T>, and