没有合适的资源?快使用搜索试试~ 我知道了~
首页理解分布式系统:关键概念与实践概述
理解分布式系统:关键概念与实践概述
需积分: 10 11 下载量 40 浏览量
更新于2024-07-18
1
收藏 806KB PDF 举报
"《分布式系统入门(英文版)》是一本深入浅出的教程,旨在为读者提供一个理解和掌握现代分布式系统核心理念的基础平台。作者的目标是为那些希望进一步探索诸如亚马逊的Dynamo、谷歌的BigTable、Apache的Hadoop等复杂分布式系统背后的理论和实践的读者,提供一个易于理解的引导。 本书的核心关注点在于介绍分布式系统的基本概念,包括但不限于数据一致性、分布式存储、网络通信、负载均衡和容错机制。信息传输速度接近光速和独立组件的故障独立性是分布式编程必须面对的两个关键挑战,它们反映了分布式环境中的基本特性:距离和多样性。这些约束决定了系统的可能设计范围。 作者强调,通过阅读这本书,读者不仅能建立起处理分布式问题的基本框架,还能获得对分布式系统工作原理的直观理解,而不被技术细节淹没。随着互联网的普及,读者可以根据自己的兴趣选择深入学习特定主题,例如分布式计算模型、分布式数据库或分布式文件系统。 书中会详细介绍如何应对由于信息传递延迟和独立组件的故障带来的挑战,如何设计能够处理大规模数据和高并发请求的系统,以及如何在分布式环境中实现数据一致性策略。此外,还将探讨分布式系统中的常见设计模式,如复制、分片、一致性哈希等,以及它们在实际项目中的应用。 《分布式系统入门(英文版)》是一本旨在帮助读者跨越分布式系统门槛的实用指南,无论是对初学者还是有经验的开发人员,都能从中受益匪浅,了解到如何在这个充满挑战但也充满无限可能的领域中创新和成长。"
资源详情
资源推荐
![](https://csdnimg.cn/release/download_crawler_static/10383718/bgc.jpg)
A system model
A key property of distributed systems is distribution. More specifically, programs in a
distributed system:
run concurrently on independent nodes ...
are connected by a network that may introduce nondeterminism and message loss ...
and have no shared memory or shared clock.
There are many implications:
each node executes a program concurrently
knowledge is local: nodes have fast access only to their local state, and any
information about global state is potentially out of date
nodes can fail and recover from failure independently
messages can be delayed or lost (independent of node failure; it is not easy to
distinguish network failure and node failure)
and clocks are not synchronized across nodes (local timestamps do not correspond
to the global real time order, which cannot be easily observed)
A system model enumerates the many assumptions associated with a particular system
design.
System model
a set of assumptions about the environment and facilities on which a distributed system is
implemented
System models vary in their assumptions about the environment and facilities. These
assumptions include:
what capabilities the nodes have and how they may fail
how communication links operate and how they may fail and
properties of the overall system, such as assumptions about time and order
A robust system model is one that makes the weakest assumptions: any algorithm written
for such a system is very tolerant of different environments, since it makes very few and
very weak assumptions.
On the other hand, we can create a system model that is easy to reason about by making
strong assumptions. For example, assuming that nodes do not fail means that our
algorithm does not need to handle node failures. However, such a system model is
unrealistic and hence hard to apply into practice.
Let's look at the properties of nodes, links and time and order in more detail.
Nodes in our system model
Nodes serve as hosts for computation and storage. They have:
the ability to execute a program
the ability to store data into volatile memory (which can be lost upon failure) and into
12/68
![](https://csdnimg.cn/release/download_crawler_static/10383718/bgd.jpg)
stable state (which can be read after a failure)
a clock (which may or may not be assumed to be accurate)
Nodes execute deterministic algorithms: the local computation, the local state after the
computation, and the messages sent are determined uniquely by the message received
and local state when the message was received.
There are many possible failure models which describe the ways in which nodes can fail. In
practice, most systems assume a crash-recovery failure model: that is, nodes can only fail
by crashing, and can (possibly) recover after crashing at some later point.
Another alternative is to assume that nodes can fail by misbehaving in any arbitrary way.
This is known as Byzantine fault tolerance. Byzantine faults are rarely handled in real world
commercial systems, because algorithms resilient to arbitrary faults are more expensive to
run and more complex to implement. I will not discuss them here.
Communication links in our system model
Communication links connect individual nodes to each other, and allow messages to be
sent in either direction. Many books that discuss distributed algorithms assume that there
are individual links between each pair of nodes, that the links provide FIFO (first in, first out)
order for messages, that they can only deliver messages that were sent, and that sent
messages can be lost.
Some algorithms assume that the network is reliable: that messages are never lost and
never delayed indefinitely. This may be a reasonable assumption for some real-world
settings, but in general it is preferable to consider the network to be unreliable and subject
to message loss and delays.
A network partition occurs when the network fails while the nodes themselves remain
operational. When this occurs, messages may be lost or delayed until the network partition
is repaired. Partitioned nodes may be accessible by some clients, and so must be treated
differently from crashed nodes. The diagram below illustrates a node failure vs. a network
partition:
It is rare to make further assumptions about communication links. We could assume that
links only work in one direction, or we could introduce different communication costs (e.g.
latency due to physical distance) for different links. However, these are rarely concerns in
13/68
![](https://csdnimg.cn/release/download_crawler_static/10383718/bge.jpg)
commercial environments except for long-distance links (WAN latency) and so I will not
discuss them here; a more detailed model of costs and topology allows for better
optimization at the cost of complexity.
Timing / ordering assumptions
One of the consequences of physical distribution is that each node experiences the world
in a unique manner. This is inescapable, because information can only travel at the speed
of light. If nodes are at different distances from each other, then any messages sent from
one node to the others will arrive at a different time and potentially in a different order at the
other nodes.
Timing assumptions are a convenient shorthand for capturing assumptions about the
extent to which we take this reality into account. The two main alternatives are:
Synchronous system model
Processes execute in lock-step; there is a known upper bound on message transmission
delay; each process has an accurate clock
Asynchronous system model
No timing assumptions - e.g. processes execute at independent rates; there is no bound on
message transmission delay; useful clocks do not exist
The synchronous system model imposes many constraints on time and order. It essentially
assumes that the nodes have the same experience: that messages that are sent are
always received within a particular maximum transmission delay, and that processes
execute in lock-step. This is convenient, because it allows you as the system designer to
make assumptions about time and order, while the asynchronous system model doesn't.
Asynchronicity is a non-assumption: it just assumes that you can't rely on timing (or a "time
sensor").
It is easier to solve problems in the synchronous system model, because assumptions
about execution speeds, maximum message transmission delays and clock accuracy all
help in solving problems since you can make inferences based on those assumptions and
rule out inconvenient failure scenarios by assuming they never occur.
Of course, assuming the synchronous system model is not particularly realistic. Real-world
networks are subject to failures and there are no hard bounds on message delay. Real
world systems are at best partially synchronous: they may occasionally work correctly and
provide some upper bounds, but there will be times where messages are delayed
indefinitely and clocks are out of sync. I won't really discuss algorithms for synchronous
systems here, but you will probably run into them in many other introductory books because
they are analytically easier (but unrealistic).
The consensus problem
During the rest of this text, we'll vary the parameters of the system model. Next, we'll look
at how varying two system properties:
whether or not network partitions are included in the failure model, and
14/68
剩余67页未读,继续阅读
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://profile-avatar.csdnimg.cn/default.jpg!1)
gongwayne
- 粉丝: 1
- 资源: 19
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![](https://csdnimg.cn/release/wenkucmsfe/public/img/voice.245cc511.png)
最新资源
- 新型矿用本安直流稳压电源设计:双重保护电路
- 煤矿掘进工作面安全因素研究:结构方程模型
- 利用同位素位移探测原子内部新型力
- 钻锚机钻臂动力学仿真分析与优化
- 钻孔成像技术在巷道松动圈检测与支护设计中的应用
- 极化与非极化ep碰撞中J/ψ的Sivers与cos2φ效应:理论分析与COMPASS验证
- 新疆矿区1200m深孔钻探关键技术与实践
- 建筑行业事故预防:综合动态事故致因理论的应用
- 北斗卫星监测系统在电网塔形实时监控中的应用
- 煤层气羽状水平井数值模拟:交替隐式算法的应用
- 开放字符串T对偶与双空间坐标变换
- 煤矿瓦斯抽采半径测定新方法——瓦斯储量法
- 大倾角大采高工作面设备稳定与安全控制关键技术
- 超标违规背景下的热波动影响分析
- 中国煤矿选煤设计进展与挑战:历史、现状与未来发展
- 反演技术与RBF神经网络在移动机器人控制中的应用
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035111.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![](https://csdnimg.cn/release/wenkucmsfe/public/img/green-success.6a4acb44.png)