Spark入门指南：实战Apache大数据分析框架

需积分: 3 108 浏览量更新于2024-07-21 收藏 10.72MB PDF 举报

"《Learning Spark》是由Apache Spark的创始人Holden Karau、Andy Konwinski、Patrick Wendell和Matei Zaharia共同编著的一本权威入门书籍。这本专著的ISBN为978-1-449-35862-4，定价分别为美国版US$39.99和加拿大版CAN$45.99，对于那些寻求高效处理大数据应用的人来说，这本书是不可或缺的参考。书中重点介绍了Spark，一个开源的集群计算系统，它显著提升了数据分析的速度，支持使用Python、Java和Scala等编程语言编写简洁高效的代码。Spark特别强调并行处理，只需寥寥几行代码，就能表达复杂的分布式工作负载，无论是简单的批量作业，还是实时流处理和机器学习任务，都能轻松应对。作者们深入浅出地讲解了Spark的关键特性，包括分布式数据集、内存缓存以及交互式Shell（如Spark Shell），帮助数据科学家和工程师快速上手，充分利用Spark的强大功能。此外，书中还着重探讨了如何利用Spark在数据量日益增长的各个领域中实现高效工作，满足大数据时代的需求。知名数据科学家Ben Lorica高度评价该书，认为它是初学者理解和掌握Spark的最佳指南，对想要在大数据处理领域取得突破的专业人士具有极高的价值。如果你正在寻找一本能够让你快速洞悉Spark底层原理，并学会如何运用其进行实际项目开发的书籍，《Learning Spark》无疑是一个理想的选择。通过阅读本书，你将不仅学会如何编写高效代码，还能深入了解如何在Spark的世界里最大化数据处理能力。"

Our  Java  examples  are  written  to  work with Java  version  6 and
higher. Java 8 introduces a new syntax called lambdas that makes
writing  inline  functions  much  easier,  which  can  simplify  Spark
code. We have chosen not to take advantage of this syntax in most
of our examples, as most organizations are not yet using Java 8. If
you would like to try Java 8 syntax, you can see the Databricks blog
post on this topic. Some of the examples will also be ported to Java
8 and posted to the book’s GitHub site.
This book is here to help you get your job done. In general, if example code is offered
with this book, you may use it in your programs and documentation. You do not
need to contact us for permission unless you’re reproducing a significant portion of
the code. For example, writing a program that uses several chunks of code from this
book does not require permission. Selling or  distributing a CD-ROM of examples
from O’Reilly books  does require permission.  Answering a question  by citing this
book and quoting example code does not require permission. Incorporating a signifi‐
cant  amount  of  example  code  from  this book into your  product’s  documentation
does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the
title, author, publisher, and ISBN. For example: “Learning Spark by Holden Karau,
Andy  Konwinski,  Patrick  Wendell,  and  Matei  Zaharia  (O’Reilly).  Copyright  2015
Databricks, 978-1-449-35862-4.”
If you feel your use of code examples falls outside fair use or the permission given
above, feel free to contact us at permissions@oreilly.com.
Safari® Books Online
Safari Books Online is an on-demand digital library that deliv‐
ers  expert  content  in  both  book  and  video  form  from  the
world’s leading authors in technology and business.
Technology professionals, software developers, web designers, and business and crea‐
tive  professionals  use Safari  Books  Online  as  their  primary resource  for  research,
problem solving, learning, and certification training.
Safari Books Online offers a range of plans and pricing for enterprise, government,
education, and individuals.
Members  have  access  to  thousands  of  books,  training  videos,  and  prepublication
manuscripts  in  one  fully  searchable  database  from  publishers  like  O’Reilly  Media,
Prentice  Hall  Professional,  Addison-Wesley  Professional,  Microsoft  Press,  Sams,
Que, Peachpit Press, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan
xiv  |  Preface

A Unified Stack

The Spark project contains multiple closely integrated components. At its core, Spark

is a “computational engine” that is responsible for scheduling, distributing, and mon‐

itoring applications consisting of many computational tasks across many worker

machines, or a computing cluster. Because the core engine of Spark is both fast and

general-purpose, it powers multiple higher-level components specialized for various

workloads, such as SQL or machine learning. These components are designed to

interoperate closely, letting you combine them like libraries in a software project.

A philosophy of tight integration has several benefits. First, all libraries and higher-

level components in the stack benefit from improvements at the lower layers. For

example, when Spark’s core engine adds an optimization, SQL and machine learning

libraries automatically speed up as well. Second, the costs associated with running the

stack are minimized, because instead of running 5–10 independent software systems,

an organization needs to run only one. These costs include deployment, mainte‐

nance, testing, support, and others. This also means that each time a new component

is added to the Spark stack, every organization that uses Spark will immediately be

able to try this new component. This changes the cost of trying out a new type of data

analysis from downloading, deploying, and learning a new software project to

upgrading Spark.

Finally, one of the largest advantages of tight integration is the ability to build appli‐

cations that seamlessly combine different processing models. For example, in Spark

you can write one application that uses machine learning to classify data in real time

as it is ingested from streaming sources. Simultaneously, analysts can query the

resulting data, also in real time, via SQL (e.g., to join the data with unstructured log‐

files). In addition, more sophisticated data engineers and data scientists can access

the same data via the Python shell for ad hoc analysis. Others might access the data in

standalone batch applications. All the while, the IT team has to maintain only one

system.

Here we will briefly introduce each of Spark’s components, shown in Figure 1-1.

2 | Chapter 1: Introduction to Data Analysis with Spark

剩余273页未读，继续阅读

ok_我的心

粉丝: 11
资源: 1

Spark入门指南：实战Apache大数据分析框架

Learning Spark .pdf 2015出版 高清

Learning.Spark.pdf(英文版)+图解Spark核心技术与案例实战.pdf

Learning Spark pdf

LearningSpark

Learning Spark

Learning Spark SQL

Learning Spark.pdf

Learning Spark SQL epub

LearningSpark：学习使用Spark的Scala示例

learning spark笔记17-spark sql

最新资源

Learning Spark .pdf 2015出版高清