Cloud Computing：理论与实践

需积分: 10 11 浏览量更新于2024-07-26 收藏 6.06MB PDF 举报

"这篇文档是Dan C. Marinescu教授撰写的一份关于云计算的教程，涵盖了云计算的基本概念、高性能计算、并行与分布式系统等多个方面。文档深入探讨了云计算的起源、服务模型、伦理问题以及安全挑战，并提供了进一步阅读的资料和练习题目。" 云计算是一种通过互联网提供计算资源和服务的模式，它允许用户按需访问硬件、软件和存储资源，而无需直接拥有或管理这些基础设施。这种模式基于网络中心化计算和内容分发，强调资源共享和大规模计算能力。高性能计算（High Performance Computing, HPC）是云计算的一个关键组成部分，它涉及在多台计算机上并行处理大量数据，以实现更快的计算速度和更高的计算能力。HPC通常应用于科学模拟、天气预报、生物信息学等领域，解决复杂的问题。 MapReduce是一种编程模型，由Google开发，用于大规模数据集的并行处理。它将大数据处理任务分解为两个主要步骤：Map（映射）和Reduce（规约）。Map阶段将输入数据分成键值对，然后在分布式环境中并行处理；Reduce阶段聚合Map阶段的结果，处理和汇总数据。 Hadoop是开源的分布式计算框架，设计用来处理和存储大量数据。它基于Java，支持MapReduce，能够高效地处理PB级别的数据。Hadoop的核心组件包括HDFS（Hadoop Distributed File System）和YARN（Yet Another Resource Negotiator），分别负责数据存储和资源管理。在文档中，作者还讨论了云计算的交付模型，包括基础设施即服务（IaaS）、平台即服务（PaaS）和软件即服务（SaaS），以及云服务提供商如何为不同类型的用户提供服务。此外，伦理问题如数据隐私、数据主权和责任划分也是云计算领域的重要议题。并行和分布式系统是云计算实现高效运行的基础。并行计算涉及多处理器系统，其中的任务被分割成多个部分，同时在多个处理器上执行，提高整体计算效率。分布式系统则涉及跨越多个网络节点的计算，它们共同处理任务，提供高可用性和容错性。文档中还提到了通信协议、逻辑时钟和消息传递规则等概念，这些都是保证分布式系统正确运行的关键技术。这份云计算教程深入浅出地介绍了云计算的基础理论、实践应用及其与并行和分布式计算的关联，对于理解云计算的原理和应用具有很高的价值。

• Computing and communication resources (CPU cycles, storage, network bandwidth)

are shared and resources can be aggregated to support data-intensive applications.

Multiplexing leads to a higher resource utilization; indeed, when multiple applications

share a system their peak demands for resources are not synchronized and the average

system utilization increases. On the other hand, the management of large pools of

resources poses new challenges as complex systems are subject to phase transitions.

New resource management strategies, such as self-organization, and decisions based

on approximate knowledge of the state of the system must be considered. Ensuring

Quality of Service (QoS) guarantees is extremely challenging in such environments, as

total performance isolation is elusive.

• Data sharing facilitates collaborative activities. Indeed, many applications in science,

engineering, as well as, industrial, ﬁnancial, governmental applications require multiple

types of analysis of shared data sets and multiple decisions carried out by groups

scattered around the globe. Open software development sites are another example

of such collaborative activities. Data sharing poses not only security and privacy

challenges but also requires mechanisms for access control for authorized users and for

detailed logs of the history of data changes.

• Cost reduction. Concentration of resources creates the opportunity to pay-as-you-go

for computing and thus, eliminates the initial investment and reduces signiﬁcantly the

maintenance and operation costs of the local computing infrastructure.

• User convenience and elasticity, the ability to accommodate workloads with very large

peak-to-average ratios.

It is very hard to point out a single technological or architectural development that trig-

gered the movement towards network-centric computing and network-centric content. This

movement is the result of a cumulative eﬀect of developments in microprocessor, storage,

and networking technologies coupled with architectural advancements in all these areas and

last, but not least, with advances in software systems, tools, programming languages and

algorithms to support distributed and parallel computing.

Along the years we have witnessed the breathtaking evolution of solid state technologies

which led to the development of multi-core and many-core processors. Quad-core processors

such as the AMD Phenom II X4, the Intel i3, i5, and i7, and hexa-core processors such as

AMD Phenom II X6 and Intel Core i7 Extreme Edition 980X are now used in the servers

populating computer clouds. The proximity of multiple cores on the same die allows the

cache coherency circuitry to operate at a much higher clock-rate than it would be possible

if the signals were to travel oﬀ-chip.

Storage technology has also evolved dramatically. For example, solid state disks such as

RamSan-440 allow systems to manage very high transaction volumes and larger numbers

of concurrent users. RamSan-440 uses DDR2 (double-data-rate) RAM to deliver 600 000

sustained random IOPS and over 4GB/second of sustained random read or write band-

width, with latency of less than 15 microseconds and it is available in 256 GB and 512 GB

conﬁgurations. The price of memory has dropped signiﬁcantly; at the time of this writing

the price of 1 GB module for a PC is approaching $10. Optical storage technologies and

ﬂash memories are widely used nowadays.

The thinking in software engineering has also evolved and new models have emerged. The

three-tier model is a software architecture and a software design pattern. The presentation

tier is the topmost level of the application; typically, it runs on a desktop PC or workstation

and uses a standard graphical user interface and displays information related to services

such as browsing merchandize, purchasing, and shopping cart contents. The presentation

tier communicates with other tiers by sending the results to the browser/client tier and

all other tiers in the network. The application/logic tier controls the functionality of an

application and may consist of one or more separate modules running on a workstation or

application server; it may be multi-tiered itself and then the architecture is called an n-tier

architecture. The data tier controls the servers where the information is stored; it runs a

relational database management system (RDBMS) on a database server or a mainframe

and contains the computer data storage logic. The data tier keeps data independent from

application servers or processing logic and improves scalability and performance. Any of

the tiers can be replaced independently; for example, a change of operating system in the

presentation tier would only aﬀect the user interface code.

1.2 Peer-to-peer systems

The distributed systems discussed in Section 2 allow access to resources in a tightly con-

trolled environment. System administrators enforce security rules and control the allocation

of physical, rather than virtual resources. In all models of network centric computing prior

to utility computing a user maintained direct control of the software and the data residing

on remote systems.

This user-centric model, in place since the early 1960s, was challenged in the 1990s

by the peer-to-peer (P2P) model. Peer-to-peer (P2P) systems can be regarded as one of

the precursors of today’s clouds. This new model for distributed computing promoted the

idea of low-cost access to storage and CPU cycles provided by participant systems; in this

case, the resources are located in diﬀerent administrative domains. The P2P systems are

self-organizing and decentralized, while the servers in a cloud are in a single administrative

domain and have a central management.

P2P systems exploit the network infrastructure to provide access to distributed comput-

ing resources. Decentralized applications developed in the 1980s such as SMTP (Simple Mail

Transfer Protocol), a protocol for Email distribution, and NNTP (Network News Transfer

Protocol), an application protocol for dissemination of news articles, are early examples

of P2P systems. Systems developed in the late 1990s such as the music-sharing system

Napster gave participants access to storage distributed over the network, while the ﬁrst

volunteer-based scientiﬁc computing, SETI@home used free cycles of participating systems

to carry out compute-intensive tasks.

The P2P model represents a signiﬁcant departure from the client-server model, the cor-

nerstone of distributed applications for several decades. P2P systems have several desirable

properties [306]:

• require a minimally dedicated infrastructure, as resources are contributed by the par-

ticipating systems;

• are highly decentralized;

• are scalable, the individual nodes are not required to be aware of the global state;

• are resilient to faults and attacks, as few of their elements are critical for the delivery

of service and the abundance of resources can support a high degree of replication;

• individual nodes do not require excessive network bandwidth as servers used in case

of the client-server model do; and last but not least,

• the systems are shielded from censorship due to the dynamic and often unstructured

system architecture.

The undesirable properties of peer-to-peer systems are also notable: decentralization

raises the question if P2P systems can be managed eﬀectively and provide the security

required by various applications; the fact that they are shielded from censorship makes

them a fertile ground for illegal activities including distribution of copyrighted content.

In spite of its problems, the new paradigm was embraced by applications other than ﬁle

sharing. Since 1999 new P2P applications such as the ubiquitous Skype, a voice over IP

telephony service

, data streaming applications such as Cool Streaming [386] and BBC’s

online video service, content distribution networks such as CoDeeN [368], and volunteer

computing applications based on the BOINC (Berkeley Open Infrastructure for Networking

Computing) platform [21], have proved their appeal to users. For example, Skyp e reported

in 2008 that 276 million registered users have used more than 100 billion minutes for voice

and video calls. The site www.boinc.berkeley.edu reports that at the end of June 2012

volunteer computing involved more than 275 000 individuals and more than 430 000 com-

puters providing a monthly average of almost 6.3 PetaFLOPS. It is also reported that the

peer-to-peer traﬃc accounts for a very large fraction of the Internet traﬃc, with estimates

ranging from 40% to more than 70%.

Many groups from industry and academia rushed to develop and test new ideas tak-

ing advantage of the fact that P2P applications do not require a dedicated infrastructure.

Applications such as Chord [334] and Credence [366] address issues critical for the eﬀective

operation of decentralized systems. Chord is a distributed lookup protocol to identify the

node where a particular data item is stored. The routing tables are distributed and, while

other algorithms for locating an object require the nodes to be aware of most of the nodes of

the network, Chord maps a key related to an object to a node of the network using routing

information about a few nodes only.

Credence is an object reputation and ranking scheme for large-scale P2P ﬁle sharing

systems. Reputation is of paramount importance for systems which often include many

unreliable and malicious nodes. In the decentralized algorithm used by Credence each

client uses local information to evaluate the reputation of other nodes and shares its own

assessment with its neighbors. The credibility of a node depends only on the votes it casts;

each node computes the reputation of another node based solely on the degree of matching

with its own votes and relies on like-minded peers. Overcite [337] is a P2P application to

aggregate documents based on a three-tier design. The web front-ends accept queries and

display the results while servers crawl through the web to generate indexes and to perform

Skype allows close to 700 million registered users from many countries around the globe to communicate

using a proprietary voice-over-IP protocol. The system developed in 2003 by Niklas Zennstr¨om and Julius

Friis was acquired by Microsoft in 2011 and nowadays is a hybrid P2P and client-server system.

keyword searches; the web back-ends store documents, meta-data, and coordination state

on the participating systems.

The rapid acceptance of the new paradigm triggered the development of a new commu-

nication protocol allowing hosts at the network periphery to cope with the limited network

bandwidth available to them. BitTorrent is a peer-to-peer ﬁle sharing protocol enabling a

node to download/upload large ﬁles from/to several hosts simultaneously.

The P2P systems diﬀer in their architecture. Some do not have any centralized infras-

tructure, while others have a dedicated controller, but this controller is not involved in

resource-intensive operations. For example, Skype has a central site to maintain the user

accounts; the users sign in and pay for speciﬁc activities at this site. The controller for

a BOINC platform maintains membership and is involved in task distribution to partic-

ipating systems. The nodes with abundant resources in systems without any centralized

infrastructure often act as supernodes and maintain information useful to increasing the

system eﬃciency e.g., indexes of the available content.

Regardless of the architecture, P2P systems are built around an overlay network, a

virtual network superimposed over the real network. Methods to construct such an overlay,

discussed in Section 7.10, consider a graph G = (V, E) where V is the set of N vertices and

E is the set of links between them.

Each node maintains a table of overlay links connecting it with other nodes of this virtual

network, each node being identiﬁed by its IP addresses. Two types of overlay networks,

unstructured and structured, are used by P2P systems. Random walks starting from a

few bootstrap nodes are usually used by systems desiring to join an unstructured overlay.

Each node of a structured overlay has a unique key which determines its position in the

structure; the keys are selected to guarantee a uniform distribution in a very large name

space. Structured overlay networks use key-based routing (KBR); given a starting node v

and a key k the function KBR(v

, k) returns the path in the graph from v

to the vertex

with key k. Epidemic algorithms discussed in Section 7.12 are often used by unstructured

overlays to disseminate network topology.

1.3 Cloud computing - an old idea whose time has come

Once the technological elements were in place it was only a matter of time until the eco-

nomical advantages of cloud computing became apparent. Due to the economy of scale

large data centers, centers with more than 50 000 systems, are more economical to operate

than medium size centers which have around 1 000 systems. Large data centers equipped

with commodity computers experience a 5 to 7 times decrease of resource consumption, in-

cluding energy, compared to medium size centers [25]. The networking costs, in dollars per

Mbit/sec/month, are 95/13 = 7.1 larger and the storage costs, in dollars per Gbyte/month,

are 2.2/0.4 = 5.7 larger for medium size centers. Medium size centers have a larger admin-

istrative overhead, one system administrator for 140 systems versus one for 1 000 systems

for large centers.

Data centers are very large consumers of electric energy to keep the servers and the

networking infrastructure running and for cooling. For example, there are 6 000 data centers

in the U.S and in 2006 they reportedly consumed 61 billion KWh, 1.5% of all electric energy

in the U.S., at a cost of $4.5 billion. The power demanded by data centers was predicted

to double from 2006 to 2011. Peak instantaneous demand was predicted to increase from 7

GW in 2006 to 12 GW in 2011, requiring the construction of 10 new power plants. In the

US the energy costs diﬀer from state to state, e.g., 1 KWh costs 3.6 cents in Idaho, 10 cents

in California, and 18 cents in Hawaii thus, data centers should be placed at sites with low

energy cost.

The term “computer cloud” is overloaded as it covers infrastructures of diﬀerent sizes,

with diﬀerent management, and a diﬀerent user population. Several types of clouds are

envisioned:

• Private Cloud - the infrastructure is operated solely for an organization, It may be

managed by the organization or a third party and may exist on or oﬀ the premises of

the organization.

• Community Cloud - the infrastructure is shared by several organizations and supports

a speciﬁc community that has shared concerns (e.g., mission, security requirements,

policy, and compliance considerations). It may be managed by the organizations or a

third party and may exist on premises or oﬀ premises.

• Public Cloud - the infrastructure is made available to the general public or a large

industry group and is owned by an organization selling cloud services.

• Hybrid Cloud - the infrastructure is a composition of two or more clouds (private,

community, or public) that remain unique entities but are bound together by stan-

dardized or proprietary technology that enables data and application portability (e.g.,

cloud bursting for load-balancing between clouds).

A private cloud could provide the computing resources needed for a large organization,

e.g., a research institution, a university, or a corporation. The argument that a private

cloud does not support utility computing is based on the observation that an organization

has to invest in the infrastructure and the a user of a private cloud does pays as it consumes

resources [25]. Nevertheless, a private cloud could use the same hardware infrastructure as

a public one; its security requirements will be diﬀerent from those for a public cloud and

the software running on the cloud is likely to be restricted to a speciﬁc domain.

A natural question to ask is: Why cloud computing could be successful when other

paradigms have failed? The reasons why cloud computing could be successful can be grouped

in several general categories: technological advances, a realistic system model, user conve-

nience, and ﬁnancial advantages. A non-exhaustive list of reasons for the success of cloud

computing includes:

• Cloud computing is in a better position to exploit recent advances in software, net-

working, storage, and processor technologies. Cloud computing is promoted by large

IT companies where these new technological developments take place and these com-

panies have a vested interest to promote the new technologies.

• A cloud consists of a homogeneous set of hardware and software resources in a single

administrative domain. In this setup security, resource management, fault-tolerance,

and quality of service are less challenging than in a heterogeneous environment with

resources in multiple administrative domains.

剩余403页未读，继续阅读

plmoknhlg

粉丝: 5
资源: 33

Cloud Computing：理论与实践

最新云计算教程.pdf

谷歌云计算教程PART【2】

google云计算教程

Google云计算教程

linux云计算教程之openstack

云计算教程考试用111

英文原版谷歌云计算 教程

HCNANPIE云计算教程.txt

Linux云计算教程之openstack

云计算，云计算相关论文，网络存储，谷歌云计算教程PPT

最新资源

英文原版谷歌云计算教程