Spark 2.0：数据科学与机器学习实战指南

1星需积分: 10 59 浏览量更新于2024-07-19 收藏 9.43MB PDF 举报

"《Spark for Data Science》是一本深入讲解大数据处理和机器学习技术的教程，专为数据科学专业人士设计，基于最新的Spark版本2.0。作者Srinivas Duvvuri和Bikramaditya Singhal共同撰写了这本书，主要针对那些希望在数据分析领域施展才华的读者。该书不仅提供了丰富的理论知识，还包含了大量的实战案例，帮助读者从基础到高级全面掌握Apache Spark这一强大的数据处理工具。书中内容涵盖了Spark的核心概念，如分布式计算、内存计算的优势、DataFrame和Dataset API的高效操作，以及如何利用Spark进行实时流处理和批处理。此外，重点放在了Spark在机器学习场景中的应用，包括但不限于分类、聚类、回归、深度学习等，展示了如何通过Spark进行模型训练、特征工程和模型评估。值得注意的是，版权方面，所有版权信息受到严格保护，未经出版商Packt Publishing事先书面许可，不得以任何形式复制、存储或传输书中的内容。尽管作者和出版商已尽力确保信息的准确性，但本书提供的内容不带任何保证，无论是明示还是默示，均不对由此产生的任何损失或声称的损失负责。 Packt Publishing在书中对提及的所有公司和产品名称进行了商标标识，但并不保证这些信息的准确性。首次出版日期为2016年9月，这表明它反映了当时的技术发展水平，对于当今的数据科学家来说，它仍然是一个宝贵的资源，可以帮助他们跟上技术的最新进展并提升自己的专业技能。《Spark for Data Science》是一本实用且具有前瞻性的指南，适合想要深入理解和运用Spark进行大数据分析和机器学习的读者，无论是初学者还是经验丰富的专业人士，都能从中获益匪浅。"

[ viii ]

Scope of development

303

Expectations

303

Presentation options

304

Interactive notebooks 304

References 304

Web API 305

References 305

PMML and PFA 305

References 306

Development and testing

306

References 307

Data quality management

308

The Scala advantage

308

Spark development status

310

Spark 2.0's features and enhancements

310

Unifying Datasets and DataFrames 310

Structured Streaming 311

Project Tungsten phase 2 311

What's in store?

312

The big data trends

312

Summary

314

References

315

Index

316

Preface

In this smart age, data analytics is the key to sustaining and promoting business growth.

Every business is trying to leverage their data as much possible with all sorts of data science

tools and techniques to progress along the analytics maturity curve. This sudden rise in

data science requirements is the obvious reason for scarcity of data scientists. It is very

difficult to meet the market demand with unicorn data scientists who are experts in

statistics, machine learning, mathematical modelling as well as programming.

The availability of unicorn data scientists is only going to decrease with the increase in

market demand, and it will continue to be so. So, a solution was needed which not only

empowers the unicorn data scientists to do more, but also creates what Gartner calls as

“Citizen Data Scientists”. Citizen data scientists are none other than the developers,

analysts, BI professionals or other technologists whose primary job function is outside of

statistics or analytics but are passionate enough to learn data science. They are becoming

the key enabler in democratizing data analytics across organizations and industries as a

whole.

There is an ever going plethora of tools and techniques designed to facilitate big data

analytics at scale. This book is an attempt to create citizen data scientists who can leverage

Apache Spark’s distributed computing platform for data analytics.

This book is a practical guide to learn statistical analysis and machine learning to build

scalable data products. It helps to master the core concepts of data science and also Apache

Spark to help you jump start on any real life data analytics project. Throughout the book, all

the chapters are supported by sufficient examples, which can be executed on a home

computer, so that readers can easily follow and absorb the concepts. Every chapter attempts

to be self-contained so that the reader can start from any chapter with pointers to relevant

chapters for details. While the chapters start from basics for a beginner to learn and

comprehend, it is comprehensive enough for a senior architects at the same time.

What this book covers

Chapter 1, Big Data and Data Science – An Introduction, this chapter discusses briefly about

the various challenges in big data analytics and how Apache Spark solves those problems

on a single platform. This chapter also explains how data analytics has evolved to what it is

now and also gives a basic idea on the Spark stack.

Preface

[ 2 ]

Chapter 2, The Spark Programming Model, this chapter talks about the design considerations

of Apache Spark and the supported programming languages. It also explains the Spark core

components and covers the RDD API in details, which is the basic building block of Spark.

Chapter 3, Introduction to DataFrames, this chapter explains about the DataFrames, which

are the most handy and useful component for the data scientists to work at ease. It explains

about Spark SQL and the Catalyst optimizer that empowers DataFrames. Also, various

DataFrames operations are demonstrated with code examples.

Chapter 4, Unified Data Access, this chapter talks about the various ways we source data

from different sources, consolidate and work in a unified way. It covers the streaming

aspect of real time data collection and operating on them. It also talks about the under-the-

hood fundamentals of these APIs.

Chapter 5, Data Analysis on Spark, this chapter discuss about the complete data analytics

lifecycle. With ample code examples, it explains how to source data from different sources,

prepare the data using data cleaning and transformation techniques, and perform

descriptive and inferential statistics to generate hidden insights from data.

Chapter 6, Machine Learning, this chapter explains various machine learning algorithms,

how they are implemented in the MLlib library and how they can be used with the pipeline

API for a streamlined execution. This chapter covers the fundamentals of all the algorithms

covered so it could serve as a one stop reference.

Chapter 7, Extending Spark with SparkR, this chapter is primarily intended for the R

programmers who want to leverage Spark for Data Analytics. It explains how to program

with SparkR and how to use the machine learning algorithms of R libraries.

Chapter 8, Analyzing Unstructured Data, this chapter discusses only about unstructured

data analysis. It explains how to source unstructured data, process it and perform machine

learning on it. It also covers some of the dimension reduction techniques which were not

covered in the “Machine Learning” chapter.

Chapter 9, Visualizing Big Data, in this chapter, readers learn various visualization

techniques that are supported on Spark. It explains the different kinds of visualization

requirements of data engineers, data scientists and business users; and also suggests right

kinds of tools and techniques. It also talks about leveraging IPython/Jupyter notebook and

Zeppelin, an Apache project for data visualization.

剩余337页未读，继续阅读

juniper123

粉丝: 0
资源: 5

Spark 2.0：数据科学与机器学习实战指南

Spark for Data Science epub

Spark for Data Science Cookbook 无水印pdf

data science with spark

Spark for Data Science spark2.0

Spark for Data Science mobi

Mastering Spark for Data Science

Spark for Data Science Cookbook mobi

Spark for Data Science Cookbook azw3

Spark for Data Science Cookbook [2017]

Spark for Data Science pdf 完整 积分最少

最新资源

Spark for Data Science pdf 完整积分最少