深度解析Apache Spark 2.x：实战提升大数据处理速度与机器学习

需积分: 9 196 浏览量更新于2024-07-18 收藏 16.1MB PDF 举报

"《精通Apache Spark 2.x》是一本全面介绍Apache Spark 2.x技术的第二版教程。本书首先对Spark生态系统进行全面概述，让你对项目Catalyst和Tungsten有所了解。这两个组件是Spark的核心优化技术，Catalyst负责编译和优化Spark的查询计划，而Tungsten则通过内存共享和低延迟通信提高性能。书中深入探讨了内存管理与二进制处理的重要性，如何利用它们来提升数据处理速度。Cache-aware Computation部分教你如何智能利用缓存，减少磁盘IO，提高计算效率。Code Generation章节则展示了如何通过代码生成技术，进一步加速执行过程。这些技术的运用使得Spark能够在大规模数据分析中展现出超乎寻常的高效。接着，作者带你探索Spark在机器学习领域的应用，介绍了如何整合H2O和Deeplearning4j，让Spark支持更强大的AI能力。同时，书中还涵盖了Jupyter Notebook和Zeppelin这两种流行的交互式数据科学环境，以及如何在云平台上部署Spark，如Docker和Kubernetes的集成。本书特别关注Apache Spark 2.2的新特性，如DataFrame和Dataset API的全面使用，这提供了更简洁、一致的编程接口。SparkML的详细介绍展示了如何构建高级的自动化机器学习管道，使得数据科学家能够更快地开发和部署模型。此外，新引入的GraphFrames API使得图分析成为可能，这对于社交网络分析、推荐系统等领域具有重要意义。版权信息表明，本书由Romeo Kienzler撰写，由Packt Publishing出版，并强调所有内容未经许可不得复制或传播。尽管作者和出版社已尽最大努力确保信息的准确性，但读者需理解本书提供的内容并非无条件担保，不承担因本书导致的直接或间接损失的责任。关于商标的使用，Packt Publishing力求规范，但并不能保证所有提及的公司和产品商标的完整性和准确性。《精通Apache Spark 2.x》是一本深度且实用的指南，适合希望在大数据处理和Spark技术上提升的专业人士阅读和实践。"

Preface

[ 2 ]

$IBQUFS, Structured Streaming, talks about Structured Streaming – a new way of defining

continuous applications using the DataFrame and Dataset APIs.

$IBQUFS, Classical MLlib, introduces you to MLlib, the de facto standard for machine

learning when using Apache Spark.

$IBQUFS, Apache SparkML, introduces you to the DataFrame-based machine learning

library of Apache Spark: the new first-class citizen when it comes to high performance and

massively parallel machine learning.

$IBQUFS, Apache SystemML, introduces you to Apache SystemML, another machine

learning library capable of running on top of Apache Spark and incorporating advanced

features such as a cost-based optimizer, hybrid execution plans, and low-level operator re-

writes.

$IBQUFS, Deep Learning on Apache Spark using H20 and DeepLearning4j, explains that deep

learning is currently outperforming one traditional machine learning discipline after the

other. We have three open source first-class deep learning libraries running on top of

Apache Spark, which are H2O, DeepLearning4j, and Apache SystemML. Let's understand

what Deep Learning is and how to use it on top of Apache Spark using these libraries.

$IBQUFS, Apache Spark GraphX, talks about Graph processing with Scala using GraphX.

You will learn some basic and also advanced graph algorithms and how to use GraphX to

execute them.

$IBQUFS, Apache Spark GraphFrames, discusses graph processing with Scala using

GraphFrames. You will learn some basic and also advanced graph algorithms and also how

GraphFrames differ from GraphX in execution.

$IBQUFS, Apache Spark with Jupyter Notebooks on IBM DataScience Experience, introduces a

Platform as a Service offering from IBM, which is completely based on an Open Source

stack and on open standards. The main advantage is that you have no vendor lock-in.

Everything you learn here can be installed and used in other clouds, in a local datacenter, or

on your local laptop or PC.

$IBQUFS, Apache Spark on Kubernetes, explains that Platform as a Service cloud providers

completely manage the operations part of an Apache Spark cluster for you. This is an

advantage but sometimes you have to access individual cluster nodes for debugging and

tweaking and you don't want to deal with the complexity that maintaining a real cluster on

bare-metal or virtual systems entails. Here, Kubernetes might be the best solution.

Therefore, in this chapter, we explain what Kubernetes is and how it can be used to set up

an Apache Spark cluster.

Preface

[ 5 ]

Click on Code Downloads & Errata.3.

Enter the name of the book in the Search box.4.

Select the book for which you're looking to download the code files.5.

Choose from the drop-down menu where you purchased this book from.6.

Click on Code Download.7.

Once the file is downloaded, please make sure that you unzip or extract the folder using the

latest version of:

WinRAR / 7-Zip for Windows

Zipeg / iZip / UnRarX for Mac

7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at IUUQTHJUIVCDPN1BDLU1VCM

TIJOH.BTUFSJOH"QBDIF4QBSLY. We also have other code bundles from our rich

catalog of books and videos available at

IUUQTHJUIVCDPN1BDLU1VCMJTIJOH. Check

them out!

Downloading the color images of this book

We also provide you with a PDF file that has color images of the screenshots/diagrams used

in this book. The color images will help you better understand the changes in the output.

You can download this file from IUUQTXXXQBDLUQVCDPNTJUFTEFGBVMUGJMFTEPXO

PBET.BTUFSJOH"QBDIF4QBSLY@$PMPS*NBHFTQEG.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do

happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-

we would be grateful if you could report this to us. By doing so, you can save other readers

from frustration and help us improve subsequent versions of this book. If you find any

errata, please report them by visiting IUUQXXXQBDLUQVCDPNTVCNJUFSSBUB, selecting

your book, clicking on the Errata Submission Form link, and entering the details of your

errata. Once your errata are verified, your submission will be accepted and the errata will

be uploaded to our website or added to any list of existing errata under the Errata section of

that title.

剩余344页未读，继续阅读

tinywinzip

粉丝: 1

深度解析Apache Spark 2.x：实战提升大数据处理速度与机器学习

Mastering Apache Spark(掌握Apache Spark)英文版.pdf

Mastering Apache Spark 2.X(2nd) epub

Mastering Apache Spark 无水印pdf 0分

Packt.Mastering.Apache.Spark

Mastering.Apache.Spark.178397146

Packt.Mastering.Apache.Cassandra.3rd.Edition

mastering apache spark 2.x second edition

Mastering Apache Spark 2.x - Second Edition

Mastering Machine with Spark 2.x（精通Spark 2.x）

mastering-apache-spark2.4.2.pdf

最新资源