Nightcore：微服务的低延迟、可扩展无服务器计算
需积分: 0 170 浏览量
更新于2024-06-30
收藏 1.23MB PDF 举报
"45-段佳昂-（2021 ASPLOS） Nightcore - 高效且可扩展的无服务器计算"
在2021年ASPLOS会议上，Zhipeng Jia和Emmett Witchel提出了Nightcore，这是一种针对低延迟、交互式微服务的高效且可扩展的无服务器计算框架。Nightcore旨在解决当前无服务器平台存在的问题，即运行时存在毫秒级开销，无法满足某些需要亚毫秒级延迟的交互式微服务需求。
无服务器架构，或称函数即服务（FaaS），已经成为构建灵活且大规模在线服务的主流软件工程方法。无服务器函数特别适合实现微服务中的无状态RPC处理器，作为容器化RPC服务器的替代方案。然而，现有的无服务器平台在执行过程中存在显著的毫秒级延迟，这对于需要严格亚毫秒级响应时间的交互应用来说是不理想的。
Nightcore无服务器函数运行时通过引入微秒级开销的隔离机制，解决了这个问题。设计者对影响微秒级性能的各种因素进行了深入考虑，包括调度、内存管理和通信效率等。它提供基于容器的隔离，确保不同函数之间的安全性与独立性，同时保持极低的延迟性能。
为了实现这一目标，Nightcore可能采用了以下关键技术：
1. **优化调度策略**：针对微服务的特性，可能采用了更智能的调度算法，以减少函数启动和上下文切换的时间。
2. **内存管理优化**：可能使用了定制的内存分配器，减少内存分配和释放操作的开销，提升内存访问速度。
3. **轻量级隔离技术**：采用轻量级容器或者unikernel技术，以减小资源隔离带来的性能损失。
4. **高效通信机制**：通过减少函数间通信的延迟，例如使用共享内存或高效的缓存一致性协议，提高整体系统性能。
5. **预热和缓存策略**：可能包含函数预热机制，提前加载和准备常用函数，减少首次调用时的启动延迟。同时，利用缓存技术提高数据访问速度。
6. **硬件优化**：可能充分利用现代硬件特性，如CPU的多核心、高速缓存和加速器，来提高并行处理能力和计算效率。
Nightcore的出现，对于那些需要快速响应和高并发的实时应用，如金融交易、游戏服务、物联网(IoT)应用和实时数据分析等，提供了更具吸引力的解决方案。通过降低运行时开销，Nightcore使得无服务器架构可以更好地适应那些对延迟敏感的场景，进一步推动了微服务架构在云环境中的广泛应用。
Nightcore: Eicient and Scalable Serverless Computing for Latency-Sensitive, Interactive Microservices ASPLOS ’21, April 19ś23, 2021, Virtual, USA
66
,
106
]. Once built with monolithic architectures, interactive on-
line services are undergoing a shift to microservice architectures [
1
,
4
,
5
,
42
,
47
], where a large application is built by connecting loosely
coupled, single-purpose microservices. On the one hand, microser-
vice architectures provide software engineering benets such as
modularity and agility as the scale and complexity of the application
grows [
36
,
49
]. On the other hand, staged designs for online services
inherently provide better scalability and reliability, as shown in
pioneering works like SEDA [
105
]. However, while the interactive
nature of online services implies an end-to-end service-level objec-
tives (SLO) of a few tens of milliseconds, individual microservices
face more strict latency SLOs ś at the sub-millisecond-scale for leaf
microservices [100, 110].
Microservice architectures are more complex to operate com-
pared to monolithic architectures [
22
,
35
,
36
], and the complexity
grows with the number of microservices. Although microservices
are designed to be loosely coupled, their failures are usually very de-
pendent. For example, one overloaded service in the system can eas-
ily trigger failures of other services, eventually causing cascading
failures [
3
]. Overload control for microservices is dicult because
microservices call each other on data-dependent execution paths,
creating dynamics that cannot be predicted or controlled from the
runtime [
38
,
48
,
88
,
111
]. Microservices are often comprised of ser-
vices written in dierent programming languages and frameworks,
further complicating their operational problems. By leveraging fully
managed cloud services (e.g., Amazon’s DynamoDB [
6
], Elastic-
Cache [
7
], S3 [
19
], Fargate [
12
], and Lambda [
15
]), responsibilities
for scalability and availability (as well as operational complexity)
are mostly shifted to cloud providers, motivating serverless microser-
vices [20, 33, 41, 43ś45, 52, 53].
Serverless Microservices. Simplifying the development and man-
agement of online services is the largest benet of building microser-
vices on serverless infrastructure. For example, scaling the service
is automatically handled by the serverless runtime, deploying a new
version of code is a push-button operation, and monitoring is inte-
grated with the platform (e.g., CloudWatch [
2
] on AWS). Amazon
promotes serverless microservices with the slogan łno server is eas-
ier to manage than no serverž [
44
]. However, current FaaS systems
have high runtime overheads (Table 1) that cannot always meet
the strict latency requirement imposed by interactive microservices.
Nightcore lls this performance gap.
Nightcore focuses on mid-tier services implementing stateless
business logic in microservice-based online applications. These
mid-tier microservices bridge the user-facing frontend and the data
storage, and t naturally in the programming model of serverless
functions. Online data intensive (OLDI) microservices [
100
] repre-
sent another category of microser vices, where the mid-tier service
fans out requests to leaf microservices for parallel data processing.
Microservices in OLDI applications are mostly stateful and memory
intensive, and therefore are not a good t for serverless functions.
We leave serverless support of OLDI microservices as future work.
The programming model of serverless functions expects func-
tion invocations to be short-lived, which seems to contradict the
assumption of service-oriented architectures which expect services
to be long-running. However, FaaS systems like AWS Lambda al-
lows clients to maintain long-lived connections to their API gate-
ways [
8
], making a serverless function łservice-likež. Moreover,
because AWS Lambda re-uses execution contexts for multiple func-
tion invocations [
13
], users’ code in serverless functions can also
cache reusable resources (e.g., database connections) between invo-
cations for better performance [17].
Optimizing FaaS Runtime Overheads. Reducing start-up laten-
cies, especially cold-start latencies, is a major research focus for FaaS
runtime overheads [
57
,
64
,
67
,
89
,
90
,
98
]. Nightcore assumes su-
cient resources have been provisioned and relevant function con-
tainers are in warm states which can be achieved on AWS Lambda
by using provisioned concurrency (AWS Lambda strongly recom-
mends provisioned concurrency for latency-critical functions [
40
]).
As techniques for optimizing cold-start latencies [
89
,
90
] become
mainstream, they can be applied to Nightcore.
Invocation latency overheads of FaaS systems are largely over-
looked, as recent studies on serverless computing focus on data
intensive workloads such as big data analysis [
75
,
95
], video analyt-
ics [
59
,
69
], code compilation [
68
], and machine learning [
65
,
98
],
where function execution times range from hundreds of millisec-
onds to a few seconds. However, a few studies [
62
,
84
] point out that
the millisecond-scale invocation overheads of current FaaS systems
make them a poor substrate for microservices with microsecond-
scale latency targets. For serverless computing to be successful in
new problem domains [
71
,
76
,
84
], it must address microsecond-
scale overheads.
3 DESIGN
Nightcore is designed to run serverless functions with sub
-
millisec-
ond
-
scale execution times, and to eciently process internal func-
tion calls, which are generated during the execution of a serverless
function (not by an external client). Nightcore exposes a serverless
function interface that is similar to AWS Lambda: users provide
stateless function handlers written in supported programming lan-
guages. The only addition to this simple interface is that Nightcore’s
runtime library provides APIs for fast internal function invocations.
3.1 System Architecture
Figure 2 depicts Nightcore’s design which mirrors the design of
other FaaS systems starting with the separation of frontend and
backend. Nightcore’s frontend is an API gateway for serving ex-
ternal function requests and other management requests (e.g., to
register new functions), while the backend consists of a number
of independent worker servers. This separation eases availability
and scalability of Nightcore, by making the frontend API gateway
fault tolerant and horizontally scaling backend worker servers. Each
worker server runs a Nightcore engine process and function con-
tainers, where each function container has one registered serverless
function, and each function has only one container on each worker
server. Nightcore’s engine directly manages function containers
and communicates with worker threads within containers.
Internal Function Calls. Nightcore optimizes internal function
calls locally on the same worker server, without going through the
API gateway. Figure 2 depicts this fast path in Nightcore’s runtime
154
剩余14页未读，继续阅读
145 浏览量
点击了解资源详情
135 浏览量
2021-04-22 上传
2021-04-22 上传
228 浏览量
2021-04-22 上传
2015-07-27 上传
218 浏览量
白小俗
- 粉丝: 37
- 资源: 302
我的内容管理
展开
最新资源
- C++指针详解,经典介绍,比较全面
- A*B 大数相乘 算法 很具有研究性。无错误！
- 动态规划经典题目及解答
- MyEclipse 6 Java 开发中文教程.
- C语言-编程修养（推荐）
- 飞思卡尔中文资料(Freescale)-MC9S08AC16数据手册
- 0V7620中文资料
- ucos exercise
- freescale codewarrir中文资料
- STL_Alexander_Lee_Meng
- STL_tutorial_reference
- 5种JSP页面显示为乱码的解决方法
- I2C 协议标准中文版
- Cisco IOS Programing Guide.pdf
- 人脸识别技术综述所采用的基本方法
- UML+for+Java+Programmers中文版.pdf
