大数据洞察：企业级Hadoop与流数据分析

需积分: 9 76 浏览量更新于2024-07-17 收藏 4.53MB PDF 举报

"Understanding Big Data (2012)" 是一本深入探讨大数据分析在企业级Hadoop和流数据处理中的应用的书籍。作者Paul C. Zikopoulos是IBM软件集团信息管理分部的技术专业人员总监，同时领导全球数据库竞争和大数据SWAT团队。他是一位获奖的作家和演讲者，具有超过18年的信息管理经验，撰写过350多篇杂志文章和14本关于数据库技术的书籍。本书主要知识点包括： 1. **大数据定义与特征**：大数据通常指数据量巨大、增长速度快、种类繁多且价值密度低的数据集合。这些数据无法用传统数据处理方法有效处理，需要借助新的技术和架构，如Hadoop。 2. **Hadoop框架**：Hadoop是一个开源的分布式计算框架，用于处理和存储大规模数据。它由HDFS（Hadoop Distributed File System）和MapReduce两大部分组成，提供高容错性和可扩展性。 3. **大数据分析**：企业级大数据分析涉及到对海量数据的收集、存储、管理和分析，以提取有价值的信息和洞察。这包括预测性分析、实时分析和数据挖掘等。 4. **流数据处理**：流数据是指持续不断产生的数据流，需要实时或近实时处理。这种处理通常由Apache Storm、Apache Flink等流处理平台完成，用于实时监控、报警和决策支持。 5. **企业级应用**：在企业环境中，大数据解决方案需要满足高可用性、安全性和合规性。书中可能探讨了如何将大数据技术应用于企业决策、客户行为分析、运营优化等方面。 6. **数据库技术**：Paul Zikopoulos的专业背景涉及DB2，因此书中可能涵盖了数据库在大数据环境中的角色，如NoSQL数据库、分布式数据库以及与Hadoop的集成。 7. **大数据挑战与风险**：数据的隐私、安全、质量和治理是大数据项目中必须面对的问题。书中可能会讨论如何管理和减轻这些风险。 8. **技能认证**：作者自身是DB2认证的高级技术专家和解决方案专家，读者可以期待书中对数据库认证和技能提升的建议。 9. **敏捷扩展**：书中的“风险免费敏捷扩展”可能涉及如何灵活地扩展大数据平台以适应业务需求的增长。 10. **案例研究**：作者的经验和行业洞察可能带来实际项目中的成功案例，帮助读者理解大数据解决方案在实际场景中的应用。《Understanding Big Data》这本书提供了对企业级大数据分析和Hadoop技术的全面理解，同时涵盖了流数据处理、数据库技术及其在企业实践中的应用。对于想要深入了解大数据领域的人来说，这是一个宝贵的资源。

Foreword xvii

these warehouses into data marts for faster decisions that are tightly associ-

ated with specific lines of business needs. These developments, over a short

period of ten years in the 1990s, made the IT department a key competitive

differentiator for every business venture. Thousands of applications were

born—some horizontal across industries, some specific to domains such as

purchasing, shipping, transportation, and more. Codenames such as ERP

(Enterprise Resource Planning), SCM (Supply Chain Management), and oth-

ers became commonplace.

By the late 1990s, inevitably, different portions of an organization used

different data management systems to store and search their critical data,

leading to federated database engines (under the IBM codename Garlic).

Then, in 2001, came the era of XML. The DB2 pureXML technology offers

sophisticated capabilities to store, process, and manage XML data in its na-

tive hierarchical format. Although XML allowed a flexible schema and ease

of portability as key advantages, the widespread use of e-mail, accumulation

of back office content, and other technologies led to the demand for content

management systems and the era of analyzing unstructured and semistruc-

tured data in enterprises was born. Today, the advent of the Internet, cou-

pled with complete democratization of content creation and distribution in

multiple formats, has led to the explosion of all types of data. Data is now not

only big, both in terms of volume and variety, but it has a velocity compo-

nent to it as well. The ability for us to glean the nuggets of information em-

bedded in such a cacophony of data, at precisely the time of need, makes it

very exciting. We are sitting at the cusp of another evolution, popularly

called as Big Data.

At IBM, our mission is to help our clients achieve their business objectives

through technological innovations, and we’ve being doing it for a century as

of 2011. During the last five decades, IBM has invented technologies and

delivered multiple platforms to meet the evolving data management chal-

lenges of our customers. IBM invented the relational database more than

30 years ago, which has become an industry standard with multiple products

provided by IBM alone (for example, DB2, Informix, Solid DB, and others).

Relational databases have further specialized into multidimensional data

warehouses, with highly parallel data servers, a breadth of dedicated

FM.indd 17 07/10/11 6:12 PM

xviii Foreword

appliances (such as Netezza or the Smart Analytics System), as well as analy-

sis and reporting tools (such as SPSS or Cognos).

Across industries and sectors (consumer goods, financial services, govern-

ment, insurance, telecommunications, and more), companies are assessing how

to manage the volume, variety, and velocity of their untapped information in

an effort to find ways to make better decisions about their business. This explo-

sion of data comes from a variety of data sources such as sensors, smart devices,

social media, billions of Internet and smartphone users, and more. This is data

that arrives in massive quantities in its earliest and most primitive form.

Organizations seeking to find a better way, which differentiates them

from their competitors, want to tap into the wealth of information hidden in

this explosion of data around them to improve their competitiveness, effi-

ciency, insight, profitability, and more. These organizations recognize the

value delivered by analyzing all their data (structured, semistructured, and

unstructured) coming from a myriad of internal and external sources. This is

the realm of “Big Data.” While many companies appreciate that the best Big

Data solutions work across functional groups touching many positions, few

corporations have figured out how to proceed. The challenge for the enter-

prise is to have a data platform that leverages these large volumes of data to

derive timely insight, while preserving their existing investments in Infor-

mation Management. In reality, the best Big Data solutions will also help

organizations to know their customer better than ever before.

To address these business needs, this book explores key case studies of

how people and companies have approached this modern problem. The

book diligently describes the challenges of harnessing Big Data and provides

examples of Big Data solutions that deliver tangible business benefits.

I would like to thank Paul, George, Tom, and Dirk for writing this book.

They are an outstanding group whose dedication to our clients is unmatched.

Behind them is the Big Data development team, who continually overcomes

the challenges of our decade. I get to work with an outstanding group of

FM.indd 18 07/10/11 6:12 PM

xxi

ACKNOWLEDGMENTS

Collectively, we want to thank the following people, without whom this

book would not have been possible: Shivakumar (Shiv) Vaithyanathan, Roger

Rea, Robert Uleman, James R. Giles, Kevin Foster, Ari Valtanen, Asha Marsh,

Nagui Halim, Tina Chen, Cindy Saracco, Vijay R. Bommireddipalli, Stewart

Tate, Gary Robinson, Rafael Coss, Anshul Dawra, Andrey Balmin, Manny

Corniel, Richard Hale, Bruce Brown, Mike Brule, Jing Wei Liu, Atsushi

Tsuchiya, Mark Samson, Douglas McGarrie, Wolfgang Nimfuehr, Richard

Hennessy, Daniel Dubriwny, our Research teams, and all the others in our

business who make personal sacrifices day in and day out to bring you the

IBM Big Data platform.

Rob Thomas and Anjul Bhambhri deserve a special mention because their

passion is contagious—thanks to both of you.

We especially want to give a heartfelt thanks to our terrific Distinguished

Engineer (DE), Steve Brodsky, and the two lead Senior Technical Staff

Members (STSMs) on BigInsights: Shankar Venkataraman, and Bert Van

der Linden; without their dedication and efforts, this book would not be

possible. IBM is an amazing place to work, and becomes unparalleled when

you get to work, day in and day out, beside the kind of brainpower these

guys have and their good natured willingness to share it and make us all

smarter. We would also be remiss not to thank Steven Sit, who at the last

minute came in to be our technical editor (and part-time researcher, though

we failed to tell him that when we approached him with the role).

We want to thank (although at times we cursed) Susan Visser and Linda

Currie for getting the book in place; an idea is an idea, but it takes people like

this to help get that idea off of a Buffalo wings–stained napkin and into your

hands without the mess. Our editing team—Sheena Uprety, Patty Mon, Paul

Tyler, and Lisa Theobald—all played a key role behind the scenes and we

want to extend our thanks for that. Thanks, to our McGraw-Hill guru, Paul

Carlstroem—there is a reason why we specifically wanted to work with you.

(By the way, the extra week came in handy!)

FM.indd 21 07/10/11 6:12 PM

剩余165页未读，继续阅读

shingocat

粉丝: 2
资源: 19

大数据洞察：企业级Hadoop与流数据分析

Big Data and Transport Understanding and assessing options.pdf

Python for analyze big financial data.pdf

Web Microanalysis of Big Image Data.pdf

Python for Finance.英文版(1st Edition.Yves Hilpisch)Analyze Big Financial Data.pdf

understanding-chief-data-officer.pdf

Big Data and Computational Intelligence in Networking-CRC(2018).pdf

Complete Guide to Open Source Big Data Stack-Apress(2018).pdf

Big Data in Omics and Imaging_Association Analysis-CRC (2018).pdf

Web Microanalysis of Big Image Data-Springer(2018).pdf

Web and Big Data_First International Joint Conference, Part I-Springer(2017).pdf

最新资源