Thrift：跨语言服务实现的高效工具

需积分: 10 129 浏览量更新于2024-09-16 收藏 84KB PDF 举报

"Thrift是一个由Facebook开发的软件库和代码生成工具，用于加速高效、可扩展后端服务的开发和实施。它旨在通过将每个语言中通常需要最多定制的部分抽象成一个在各语言中实现的通用库，实现跨编程语言的高效、可靠通信。Thrift允许开发者在一个语言中定义数据类型和服务接口，并自动生成构建RPC（远程过程调用）客户端和服务器所需的所有代码。本文档详细介绍了Thrift的设计动机、设计选择以及一些有趣的实现细节。" Thrift 是一个跨语言的服务实现框架，最初由Facebook创建，目的是解决在快速增长的流量和复杂网络结构下，不同编程语言之间的高效通信问题。其主要特点包括： 1. **数据类型和接口定义**：Thrift 提供了一种语言中立的方式来定义数据类型和服务接口，这些定义存储在接口定义语言（IDL）文件中。这使得开发者可以在一个地方定义服务，然后在不同的编程语言中使用。 2. **代码生成**：根据这些IDF文件，Thrift 工具可以生成特定语言的客户端和服务器端代码，减少了跨语言通信时的手动编码工作。 3. **RPC机制**：Thrift 使用RPC来实现服务间的通信。它处理了序列化、网络传输和反序列化等底层细节，使得开发者可以专注于业务逻辑，而无需关心通信细节。 4. **多语言支持**：Thrift 支持多种编程语言，如C++, Java, Python, PHP, Ruby等，这意味着服务可以使用任何支持的语言编写，并与使用其他语言的客户端进行通信。 5. **高性能和可扩展性**：Thrift 设计的目标是高效和可扩展，它可以处理大量并发请求，适合构建大规模分布式系统。 6. **灵活的架构**：Thrift 可以适应不同的传输协议，如HTTP, TCP, UDP等，同时支持同步和异步通信模式，提供了一定程度的灵活性。 7. **社区和开源**：Thrift 是一个开源项目，拥有活跃的社区，持续更新和改进，提供了丰富的示例和库，方便开发者使用和贡献。在实际应用中，Thrift 通常用于构建微服务架构，其中不同服务可以使用不同的语言实现，通过Thrift 实现高效的通信。此外，由于其跨语言特性，Thrift 还常用于构建跨平台的应用，如服务器和移动设备之间的通信。 Thrift 的设计选择和实现细节包括优化的序列化算法、错误处理机制、以及对不同平台和环境的适配。在实现上，Thrift 尽可能减少网络通信中的开销，如通过压缩数据和高效的缓冲管理来提高性能。总结起来，Thrift 是一种强大的工具，它简化了跨语言服务开发，提高了系统的可扩展性和效率，是构建分布式系统和实现多语言通信的理想选择。

Thrift: Scalable Cross-Language Services Implementation

Mark Slee, Aditya Agarwal and Marc Kwiatkowski

Facebook, 156 University Ave, Palo Alto, CA

{mcslee,aditya,marc}@facebook.com

Abstract

Thrift is a software library and set of code-generation tools devel-

oped at Facebook to expedite development and implementation of

efﬁcient and scalable backend services. Its primary goal is to en-

able efﬁcient and reliable communication across programming lan-

guages by abstracting the portions of each language that tend to

require the most customization into a common library that is imple-

mented in each language. Speciﬁcally, Thrift allows developers to

deﬁne datatypes and service interfaces in a single language-neutral

ﬁle and generate all the necessary code to build RPC clients and

servers.

This paper details the motivations and design choices we made

in Thrift, as well as some of the more interesting implementation

details. It is not intended to be taken as research, but rather it is an

exposition on what we did and why.

1. Introduction

As Facebook’s trafﬁc and network structure have scaled, the re-

source demands of many operations on the site (i.e. search, ad se-

lection and delivery, event logging) have presented technical re-

quirements drastically outside the scope of the LAMP framework.

In our implementation of these services, various programming lan-

guages have been selected to optimize for the right combination of

performance, ease and speed of development, availability of exist-

ing libraries, etc. By and large, Facebook’s engineering culture has

tended towards choosing the best tools and implementations avail-

able over standardizing on any one programming language and be-

grudgingly accepting its inherent limitations.

Given this design choice, we were presented with the challenge

of building a transparent, high-performance bridge across many

programming languages. We found that most available solutions

were either too limited, did not offer sufﬁcient datatype freedom,

or suffered from subpar performance.

The solution that we have implemented combines a language-

neutral software stack implemented across numerous programming

languages and an associated code generation engine that trans-

forms a simple interface and data deﬁnition language into client

and server remote procedure call libraries. Choosing static code

generation over a dynamic system allows us to create validated

code that can be run without the need for any advanced introspec-

tive run-time type checking. It is also designed to be as simple as

possible for the developer, who can typically deﬁne all the neces-

sary data structures and interfaces for a complex service in a single

short ﬁle.

Surprised that a robust open solution to these relatively common

problems did not yet exist, we committed early on to making the

Thrift implementation open source.

See Appendix A for a discussion of alternative systems.

In evaluating the challenges of cross-language interaction in a net-

worked environment, some key components were identiﬁed:

Types. A common type system must exist across programming lan-

guages without requiring that the application developer use custom

Thrift datatypes or write their own serialization code. That is, a C++

programmer should be able to transparently exchange a strongly

typed STL map for a dynamic Python dictionary. Neither program-

mer should be forced to write any code below the application layer

to achieve this. Section 2 details the Thrift type system.

Transport. Each language must have a common interface to bidirec-

tional raw data transport. The speciﬁcs of how a given transport is

implemented should not matter to the service developer. The same

application code should be able to run against TCP stream sockets,

raw data in memory, or ﬁles on disk. Section 3 details the Thrift

Transport layer.

Protocol. Datatypes must have some way of using the Transport

layer to encode and decode themselves. Again, the application

developer need not be concerned by this layer. Whether the service

uses an XML or binary protocol is immaterial to the application

code. All that matters is that the data can be read and written

in a consistent, deterministic matter. Section 4 details the Thrift

Protocol layer.

Versioning. For robust services, the involved datatypes must pro-

vide a mechanism for versioning themselves. Speciﬁcally, it should

be possible to add or remove ﬁelds in an object or alter the argu-

ment list of a function without any interruption in service (or, worse

yet, nasty segmentation faults). Section 5 details Thrift’s versioning

system.

Processors. Finally, we generate code capable of processing data

streams to accomplish remote procedure calls. Section 6 details the

generated code and TProcessor paradigm.

Section 7 discusses implementation details, and Section 8 describes

our conclusions.

2. Types

The goal of the Thrift type system is to enable programmers to

develop using completely natively deﬁned types, no matter what

programming language they use. By design, the Thrift type system

does not introduce any special dynamic types or wrapper objects.

It also does not require that the developer write any code for object

serialization or transport. The Thrift IDL (Interface Deﬁnition Lan-

guage) ﬁle is logically a way for developers to annotate their data

structures with the minimal amount of extra information necessary

to tell a code generator how to safely transport the objects across

languages.

2.1 Base Types

The type system rests upon a few base types. In considering which

types to support, we aimed for clarity and simplicity over abun-

下载后可阅读完整内容，剩余7页未读，立即下载

sssjszz

粉丝: 0
资源: 1

Thrift：跨语言服务实现的高效工具

Apache Thrift 使用说明

CSS_namespace_thrift接口文档

thrift实现http协议案例

thrift基础文档

Apache Thrift 0.9.2 API文档汉化版发布

探索EzBake Provenance: Thrift服务的文档出处管理

thrift 0.9.3 文档

Apache Thrift说明

thrift

thrift 0.9.2

最新资源