2015年Mahmoud Parsian的《Data Algorithms》英文电子版

4星 · 超过85%的资源需积分: 10 119 浏览量更新于2024-07-21 5 收藏 12.13MB PDF 举报

《数据算法》(DataAlgorithms)是Mahmoud Parsian所著的一本专业书籍，于2015年首次出版。该书版权属于Mahmoud Parsian，并在美国印制，由O'Reilly Media公司发行，地址位于加利福尼亚州塞巴斯托波尔。O'Reilly出版社不仅提供纸质版本，也支持在线购买，可通过safaribooksonline.com获取更多资源。对于教育、商业或销售推广用途，读者可以联系出版社的机构销售部门获取更多信息。这本书的编辑团队包括Ann Spencer和Marie Beaugureau担任编辑，Matthew Hacker负责生产编辑，Rachel Monaghan负责校对，Rachel Head担任校对员，Judith McConville负责索引制作，David Futato负责内部设计，Ellie Volckhausen负责封面设计，而Rebecca Demarest则担任插图师。首次发布日期为2015年7月10日，同时提供了一个在线修订历史链接，以便读者查询已发布的具体细节。《数据算法》的主题覆盖了广泛的计算机科学领域，聚焦于数据处理和算法设计。书中深入探讨了各种数据结构和算法的核心概念，包括但不限于排序、搜索、图论、动态规划等，这些都是现代信息技术和软件开发中的基础组成部分。它旨在帮助读者理解并掌握这些算法的工作原理，以及如何在实际问题中高效地应用它们。书中强调了理论与实践相结合的原则，通过实例分析和代码示例来讲解复杂的概念，使读者能够更好地理解和掌握算法的实现技巧。同时，作者还提供了详尽的解释和注意事项，确保信息的准确性和实用性。尽管出版方和作者已经尽力确保内容的准确性，但读者在使用时仍需关注在线修订历史，以便获取最新的更正和更新信息。《Data Algorithms》是一本适合数据科学、计算机科学专业人员和开发者提升算法技能，以及对数据处理有深入学习需求者的必备参考书。通过阅读和实践书中的内容，读者将能提升编程能力，解决实际问题，并跟上快速发展的IT技术步伐。

Why Use MapReduce?

As we’ve discussed, MapReduce works on the premise of “scaling out” by adding more

commodity servers. This is in contrast to “scaling up,” by adding more resources, such

as memory and CPUs, to a single node in a system); this can be very costly, and at some

point you won’t be able to add more resources due to cost and software or hardware

limits. Many times, there are promising main memory–based algorithms available for

solving data problems, but they lack scalability because the main memory is a

bottleneck. For example, in DNA sequencing analysis, you might need over 512 GB of

RAM, which is very costly and not scalable.

If you need to increase your computational power, you’ll need to distribute it across

more than one machine. For example, to do DNA sequencing of 500 GB of sample data,

it would take one server over four days to complete just the alignment phase; using 60

servers with MapReduce can cut this time to less than two hours. To process large

volumes of data, you must be able to split up the data into chunks for processing, which

are then recombined later. MapReduce/Hadoop and Spark/Hadoop enable you to

increase your computational power by writing just two functions: map() and reduce().

So it’s clear that data analytics has a powerful new tool with the MapReduce paradigm,

which has recently surged in popularity thanks to open source solutions such as Hadoop.

In a nutshell, MapReduce provides the following benefits:

Programming model + infrastructure

The ability to write programs that run on hundreds/thousands of machines

Automatic parallelization and distribution

Fault tolerance (if a server dies, the job will be completed by other servers)

Program/job scheduling, status checking, and monitoring

Hadoop and Spark

Hadoop is the de facto standard for implementation of MapReduce applications. It is

composed of one or more master nodes and any number of slave nodes. Hadoop

simplifies distributed applications by saying that “the data center is the computer,” and

by providing map() and reduce() functions (defined by the programmer) that allow

application developers or programmers to utilize those data centers. Hadoop

implements the MapReduce paradigm efficiently and is quite simple to learn; it is a

powerful tool for processing large amounts of data in the range of terabytes and

petabytes.

In this book, most of the MapReduce algorithms are presented in a cookbook format

(compiled, complete, and working solutions) and implemented in

Java/MapReduce/Hadoop and/or Java/Spark/Hadoop. Both the Hadoop and Spark

frameworks are open source and enable us to perform a huge volume of computations

and data processing in distributed environments.

These frameworks enable scaling by providing “scale-out” methodology. They can be

set up to run intensive computations in the MapReduce paradigm on thousands of

servers. Spark’s API has a higher-level abstraction than Hadoop’s API; for this reason,

we are able to express Spark solutions in a single Java driver class.

Hadoop and Spark are two different distributed software frameworks. Hadoop is a

MapReduce framework on which you may run jobs supporting the map(), combine(),

and reduce() functions. The MapReduce paradigm works well at one-pass

computation (first map(), then reduce()), but is inefficient for multipass algorithms.

Spark is not a MapReduce framework, but can be easily used to support a MapReduce

framework’s functionality; it has the proper API to handle map() and reduce()

functionality. Spark is not tied to a map phase and then a reduce phase. A Spark job can

be an arbitrary DAG (directed acyclic graph) of map and/or reduce/shuffle phases.

Spark programs may run with or without Hadoop, and Spark may use HDFS (Hadoop

Distributed File System) or other persistent storage for input/output. In a nutshell, for a

given Spark program or job, the Spark engine creates a DAG of task stages to be

performed on the cluster, while Hadoop/MapReduce, on the other hand, creates a DAG

with two predefined stages, map and reduce. Note that DAGs created by Spark can

contain any number of stages. This allows most Spark jobs to complete faster than they

would in Hadoop/MapReduce, with simple jobs completing after just one stage and

more complex tasks completing in a single run of many stages, rather than having to be

split into multiple jobs. As mentioned, Spark’s API is a higher-level abstraction than

剩余1056页未读，继续阅读

xinconan2

粉丝: 269
资源: 399

2015年Mahmoud Parsian的《Data Algorithms》英文电子版

Data Algorithms(O'Reilly,2016)

Data.Structures.and.Algorithms.in.Swift

nx.algorithms.centrality是什么

ModuleNotFoundError: No module named 'torch.distributed.algorithms.join

ModuleNotFoundError: No module named 'torch.distributed.algorithms.join'

module 'ray.rllib.algorithms.ppo' has no attribute 'DEFAULT_CONFIG'

AttributeError: module 'networkx.algorithms' has no attribute 'min_weight_matching'

报错：ModuleNotFoundError: No module named 'pymoo.algorithms.moo.sms'

module 'cryptography.hazmat.primitives.ciphers.algorithms' has no attribute 'DES'

AttributeError: module 'networkx.algorithms.community' has no attribute 'infomap'

最新资源