《Hadoop启蒙指南》：轻松探索大数据世界

hadoop

bigdata

需积分: 10 8 浏览量更新于2024-07-21 1 收藏 7.54MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源推荐

About Authors

Mark contributes to a number of Hadoop-based projects, and his open source projects can be found

on GitHub [http://github.com/markkerzner]. He writes about Hadoop and other technologies in his blog

[http://shmsoft.blogspot.com/].

Mark does Hadoop training for individuals and corporations; his classes are hands-on and draw heavily

on his industry experience.

Links:

LinkedIn [https://www.linkedin.com/in/markkerzner] || GitHub [https://github.com/markkerzner] || Per-

sonal blog [http://shmsoft.blogspot.com/] || Twitter [https://twitter.com/markkerzner]

Sujee Maniyam (Author)

Sujee Maniyam is an experienced/hands-on Big Data architect. He has been developing software for the

past 12 years in a variety of technologies (enterprise, web and mobile). He currently focuses on Hadoop,

Big Data and NoSQL, and Amazon Cloud Services. His clients include early stage startups and enterprise

companies.

Sujee stays active in the Hadoop / Open Source community. He runs a developer focused Meetup called

'Big Data Gurus' [http://www.meetup.com/BigDataGurus/]. He has also presented at a variety of Meetups

Sujee contributes to Hadoop projects and his open source projects can be found on GitHub. He writes

about Hadoop and other technologies on his website.

Sujee does Hadoop training for individuals and corporations; his classes are hands-on and draw heavily

on his industry experience

Links:

LinkedIn [http://www.linkedin.com/in/sujeemaniyam] || GitHub [https://github.com/sujee] || Tech writ-

ings [http://sujee.net/tech/articles/] || Tech talks [http://sujee.net/tech/talks/] || BigDataGuru meetup [http://

www.meetup.com/BigDataGurus/]

Rebecca Kerzner (Illustrator)

The delightful artwork appearing in this book was created by Rebecca Kerzner. She is seventeen and stud-

ies at Beren Academy in Houston, TX. She has attended the Glassell school of art for many summers.

Her collection of artwork can be seen here [https://picasaweb.google.com/110574221375288281850/Re-

beccaSArt].

Rebecca started working on Hadoop illustrations two years ago, when she was fifteen.

It is interesting to follow her artistic progress. For example, the first version of the

"Hadoop Zoo" can be found here [https://plus.google.com/photos/110574221375288281850/al-

bums/5850230050814801649/5852442815905345938?banner=pwa]. Note the programmer's kids who

"came to see the Hadoop Zoo" are all pretty young. But as the artist grew, so also did her heroes, and

in the current version of the same sketch [https://plus.google.com/photos/110574221375288281850/al-

bums/5850230050814801649/5852442453552955282?banner=pwa] you can see the same characters, but

older, perhaps around seventeen years of age. Even the Hadoop elephant appears more mature and serious.

All of Rebecca's Hadoop artwork can be viewed here [https://plus.google.com/pho-

tos/110574221375288281850/albums/5850230050814801649?banner=pwa].

Why do I Need Hadoop ?

3.2. Hadoop allows to capture new or more da-

Some times organizations don't capture a type of data, because it was too cost prohibitive to store it. Since

Hadoop provides storage at reasonable cost, this type of data can be captured and stored.

One example would be web site click logs. Because the volume of these logs can be very high, not many

organizations captured these. Now with Hadoop it is possible to capture and store the logs

3.3. With Hadoop, you can store data longer

To manage the volume of data stored, companies periodically purge older data. For example only logs

for the last 3 months could be stored and older logs were deleted. With Hadoop it is possible to store the

historical data longer. This allows new analytics to be done on older historical data.

For example, take click logs from a web site. Few years ago, these logs were stored for a brief period of

time to calculate statics like popular pages ..etc. Now with Hadoop it is viable to store these click logs

for longer period of time.

3.4. Hadoop provides scalable analytics

There is no point in storing all the data, if we can't analyze them. Hadoop not only provides distributed

storage, but also distributed processing as well. Meaning we can crunch a large volume of data in parallel.

The compute framework of Hadoop is called Map Reduce. Map Reduce has been proven to the scale of

peta bytes.

Chapter 9, Introduction To MapReduce [31]

3.5. Hadoop provides rich analytics

Native Map Reduce supports Java as primary programming language. Other languages like Ruby, Python

and R can be used as well.

Of course writing custom Map Reduce code is not the only way to analyze data in Hadoop. Higher level

Map Reduce is available. For example a tool named Pig takes english like data flow language and translates

them into Map Reduce. Another tool Hive, takes SQL queries and runs them using Map Reduce.

Business Intelligence (BI) tools can provide even higher level of analysis. Quite a few BI tools can work

with Hadoop and analyze data stored in Hadoop. For a list of BI tools that support Hadoop please see this

chapter : Chapter 13, Business Intelligence Tools For Hadoop and Big Data [52]

剩余71页未读，继续阅读

robertyuzj

粉丝: 23
资源: 75

《Hadoop启蒙指南》：轻松探索大数据世界

图解hadoop：Hadoop Illuminated（hadoop 入门资料）

Hadoop照亮Hadoop illuminated

hadoop命令hadoop classpath

hadoop3.1.4 hadoop-core

hadoop1.0和hadoop3.0

for host in hadoop101 hadoop102 hadoop103

hadoop2.0和hadoop3.0

hadoop 中hadoop-env.sh配置

如何新创建Hadoop集群，三台服务器分别命名hadoop105、hadoop106、hadoop107。Namenode和JobHistory在hadoop105启动，yarn在hadoop106启动，2NN在hadoop107启动。

hadoop1.0与hadoop2.0

原生Hadoop和采购hadoop

[ys@hadoop102 conf]$ hive Cannot find hadoop installation: $HADOOP_HOME or $HADOOP_PREFIX must be set or hadoop must be in the path

hive Cannot find hadoop installation: $HADOOP_HOME or $HADOOP_PREFIX must be set or hadoop must be in the path

Hello Hadoop Goodbye Hadoop

写出如下程序，使用hadoop用户登录linux操作系统，启动hadoop (hadoop的安装目录为“/usr/local/hadoop”),为hadoop用户在HDFS中创建用户目录“/user/hadoop”。

hadoop怎么打开切换到：[hadoop]/etc/hadoop目录

Cannot find hadoop installation: $HADOOP_HOME or $HADOOP_PREFIX must be set or hadoop must be in the path

Hadoop安装目录/app/hadoop

hadoop101:50010,hadoop102:50010,hadoop103:50010

Hadoop Common、Hadoop distributed file system ( HDFS) 、Hadoop MapReduce 以 及 Hadoop Yarn 四大模块详细介绍

最新资源

Hadoop Common、Hadoop distributed file system ( HDFS) 、Hadoop MapReduce 以及 Hadoop Yarn 四大模块详细介绍