![](https://csdnimg.cn/release/download_crawler_static/15119405/bg6.jpg)
1 Introduction
As performance gains from transistor shrinks have slowed, microprocessor designers have
increasingly turned to adding special-purpose accelerators to continue improving the perfor-
mance of their chips. These accelerators generally focus on computation-heavy tasks, such
as graphics audio/video decoding, cryptography, and, more recently, machine learning. In
this work, we focus on accelerating a much more mundane task: data movement.
Data movement within memory makes up a significant proportion of many workloads.
An investigation of a Google datacenter by Kanev, et al. [4] found that approximately 5%
of CPU cycles were spent inside the memcpy() and memmove() library routines. Memory-
memory copies are used especially heavily in kernel IO drivers (such as disk and network
device drivers), which spend most of their time copying data between device, kernel, and
userspace buffers.
There has been quite a bit of work done to reduce the overhead of copying data between
IO devices and main memory. Traditional DMA engines can lower the CPU overhead of
copying from device buffers to main memory by offloading this copying to the device itself.
However, the DMA engine generally does not have access to the page table, so it cannot
use virtual addresses. Since pages in userspace are usually not guaranteed to be resident in
memory, the operating system device drivers must allocate kernel-space buffers for the DMA
engine to write into and then copy the data again into the userspace buffer.
Some messaging technologies, such as Infiniband, have sought to completely obviate
additional copies by having the device write directly into the userspace buffer. However, these
so-called "zero-copy" DMA protocols shift a lot of the burden of buffer management from the
operating system developer to the application programmer. The application must explicitly
allocate memory for DMA and pin the virtual addresses in memory. This is potentially quite
expensive, since the operating system must translate all of the virtual addresses and page
them into physical memory up front. Also, while the pages are pinned, the physical frames
cannot be reused by the operating system for other tasks. The application must also manage
any concurrency between the device’s use of the memory region and its own. Application
frameworks can hide the complexity of low-level API calls, but this usually comes at the cost
of a significant performance overhead.
To solve these issues, we propose a coprocessor that can offload memory-memory copies.
The coprocessor has access to the CPU’s page table walker and thus can perform its own
virtual memory translations. It only requires intervention from the processor if the page
it is translating is not resident in memory and needs to be paged in. Once the page has
been moved into memory, the accelerator can resume the copy operation at the point it left
off. This removes the need to allocate a separate kernel buffer or pin userspace buffers in
memory.
2 Related Work
The most common form of memory-memory copy acceleration is to use vectorized or SIMD
load-store operations. This is the tactic taken by the glibc memcpy() function. On archi-
tectures which have these sorts of instructions (such as modern x86 machines), the memcpy
3