Hadoop入门指南：大数据智能应用详解

需积分: 13 179 浏览量更新于2024-07-23 收藏 5.02MB PDF 举报

《Hadoop入门教程》是由Tom White撰写的一本经典指南，专注于大数据智能推荐类应用的实战入门。该书在2009年首次出版，版权归属于Tom White，享有所有权利。本书是Hadoop技术领域的权威之作，适合那些希望深入了解和掌握这个分布式计算框架的读者。 Hadoop，作为一个开源的大数据处理框架，由Doug Cutting等人开发，其核心目标是解决海量数据存储和处理的问题。它主要包括Hadoop Distributed File System (HDFS) 和MapReduce两个关键组件。HDFS是一个分布式文件系统，能够高效地存储大量数据，而MapReduce则提供了一种简化的方式来并行处理大量数据，通过将复杂任务分解为一系列可管理的小任务来提高效率。《Hadoop：The Definitive Guide》详细介绍了Hadoop的体系结构、安装配置、数据模型、编程模型以及如何编写MapReduce任务。书中涵盖了从基础概念到高级用法的全方位内容，包括如何设计和优化分布式应用程序，以及如何处理数据清洗、数据挖掘和机器学习等应用场景。此外，本书还介绍了Hadoop生态系统中的其他重要组件，如Hive（用于SQL查询的大数据仓库）、Pig（基于Pig Latin的数据流语言）、HBase（NoSQL数据库）和YARN（Yet Another Resource Negotiator，负责资源调度）。读者可以借此了解如何利用这些工具扩展Hadoop的功能，并构建更全面的大数据解决方案。对于初学者来说，书中提供了大量的示例代码和实战项目，帮助读者通过实践学习。同时，书中还包括了对Hadoop未来发展趋势的讨论，让读者能紧跟技术前沿。《Hadoop入门教程》是一本全面而实用的指南，不仅适合希望入门Hadoop的工程师，也是数据科学家、架构师和技术管理人员提升大数据处理能力的宝贵参考书籍。无论你是想构建分布式系统，还是寻求处理海量数据的新方法，这本书都将为你提供扎实的基础和深入的理解。

Given this, I was very pleased when I learned that Tom intended to write a book about

Hadoop. Who could be better qualified? Now you have the opportunity to learn about

Hadoop from a master—not only of the technology, but also of common sense and

plain talk.

—Doug Cutting

Shed in the Yard, California

xiv | Foreword

Preface

Martin Gardner, the mathematics and science writer, once said in an interview:

Beyond calculus, I am lost. That was the secret of my column’s success. It took me so

long to understand what I was writing about that I knew how to write in a way most

readers would understand.

In many ways, this is how I feel about Hadoop. Its inner workings are complex, resting

as they do on a mixture of distributed systems theory, practical engineering, and com-

mon sense. And to the uninitiated, Hadoop can appear alien.

But it doesn’t need to be like this. Stripped to its core, the tools that Hadoop provides

for building distributed systems—for data storage, data analysis, and coordination—

are simple. If there’s a common theme, it is about raising the level of abstraction—to

create building blocks for programmers who just happen to have lots of data to store,

or lots of data to analyze, or lots of machines to coordinate, and who don’t have the

time, the skill, or the inclination to become distributed systems experts to build the

infrastructure to handle it.

With such a simple and generally applicable feature set, it seemed obvious to me when

I started using it that Hadoop deserved to be widely used. However, at the time (in

early 2006), setting up, configuring, and writing programs to use Hadoop was an art.

Things have certainly improved since then: there is more documentation, there are

more examples, and there are thriving mailing lists to go to when you have questions.

And yet the biggest hurdle for newcomers is understanding what this technology is

capable of, where it excels, and how to use it. That is why I wrote this book.

The Apache Hadoop community has come a long way. Over the course of three years,

the Hadoop project has blossomed and spun off half a dozen subprojects. In this time,

the software has made great leaps in performance, reliability, scalability, and manage-

ability. To gain even wider adoption, however, I believe we need to make Hadoop even

easier to use. This will involve writing more tools; integrating with more systems; and

“The science of fun,” Alex Bellos, The Guardian, May 31, 2008, http://www.guardian.co.uk/science/

2008/may/31/maths.science.

writing new, improved APIs. I’m looking forward to being a part of this, and I hope

this book will encourage and enable others to do so, too.

Administrative Notes

During discussion of a particular Java class in the text, I often omit its package name,

to reduce clutter. If you need to know which package a class is in, you can easily look

it up in Hadoop’s Java API documentation for the relevant subproject, linked to from

the Apache Hadoop home page at http://hadoop.apache.org/. Or if you’re using an IDE,

it can help using its auto-complete mechanism.

Similarly, although it deviates from usual style guidelines, program listings that import

multiple classes from the same package may use the asterisk wildcard character to save

space (for example: import org.apache.hadoop.io.*).

The sample programs in this book are available for download from the website that

accompanies this book: http://www.hadoopbook.com/. You will also find instructions

there for obtaining the datasets that are used in examples throughout the book, as well

as further notes for running the programs in the book, and links to updates, additional

resources, and my blog.

What’s in This Book?

The rest of this book is organized as follows. Chapter 2 provides an introduction to

MapReduce. Chapter 3 looks at Hadoop filesystems, and in particular HDFS, in depth.

Chapter 4 covers the fundamentals of I/O in Hadoop: data integrity, compression,

serialization, and file-based data structures.

The next four chapters cover MapReduce in depth. Chapter 5 goes through the practical

steps needed to develop a MapReduce application. Chapter 6 looks at how MapReduce

is implemented in Hadoop, from the point of view of a user. Chapter 7 is about the

MapReduce programming model, and the various data formats that MapReduce can

work with. Chapter 8 is on advanced MapReduce topics, including sorting and joining

data.

Chapters 9 and 10 are for Hadoop administrators, and describe how to set up and

maintain a Hadoop cluster running HDFS and MapReduce.

Chapters 11, 12, and 13 present Pig, HBase, and ZooKeeper, respectively.

Finally, Chapter 14 is a collection of case studies contributed by members of the Apache

Hadoop community.

xvi | Preface

剩余525页未读，继续阅读

vera_J

粉丝: 0
资源: 5

Hadoop入门指南：大数据智能应用详解

hadoop入门教程

hadoop从入门到精通课件pdf

Hadoop教程.pdf

Hadoop入门简介.pdf

hadoop入门指南.pdf

hadoop入门01.pdf

hadoop入门介绍.pdf

分布式计算开源框架Hadoop入门实践.pdf

Hadoop开发者入门专刊.pdf

Hadoop学习文档.pdf

最新资源