Who Should Read This Book
This book is intended for Pig programmers, new and old. Those who have never used Pig will find
introductory material on how to run Pig and to get them started writing Pig Latin scripts. For seasoned
Pig users, this book covers almost every feature of Pig: different modes it can be run in, complete
coverage of the Pig Latin language, and how to extend Pig with your own user-defined functions
(UDFs). Even those who have been using Pig for a long time are likely to discover features they have
not used before.
Some knowledge of Hadoop will be useful for readers and Pig users. If you’re not already familiar with
it or want a quick refresher, “Pig on Hadoop” walks through a very simple example of a Hadoop job.
Small snippets of Java, Python, and SQL are used in parts of this book. Knowledge of these languages is
not required to use Pig, but knowledge of Python and Java will be necessary for some of the more
advanced features. Those with a SQL background may find “Comparing Query and Data Flow
Languages” to be a helpful starting point in understanding the similarities and differences between Pig
Latin and SQL.
What’s New in This Edition
The second edition covers Pig 0.10 through Pig 0.16, which is the latest version at the time of writing.
For features introduced before 0.10, we will not call out the initial version of the feature. For newer
features introduced after 0.10, we will point out the version in which the feature was introduced.
Pig runs on both Hadoop 1 and Hadoop 2 for all the versions covered in the book. To simplify our
discussion, we assume Hadoop 2 is the target platform and will point out the difference for Hadoop 1
whenever applicable in this edition.
The second edition has two new chapters: “Pig on Tez” (Chapter 11) and “Use Cases and Programming
Examples” (Chapter 13). Other chapters have also been updated with the latest additions to Pig and
information on existing features not covered in the first edition. These include but are not limited to:
▪ New data types (boolean, datetime, biginteger, bigdecimal) are introduced in Chapter 3.
▪ New UDFs are covered in various places, including support for leveraging Hive UDFs (Chapter 4)
and applying Bloom filters (Chapter 7).
▪ New Pig operators and constructs such as rank, cube, assert, nested foreach and nested
cross, and casting relations to scalars are presented in Chapter 5.
▪ New performance optimizations — map-side aggregation, schema tuples, the shared JAR cache,
auto local and direct fetch modes, etc. — are covered in Chapter 7.
▪ Scripting UDFs in JavaScript, JRuby, Groovy, and streaming Python are discussed in Chapter 9,
and embedding Pig in scripting languages is covered in Chapter 8 and Chapter 13 (“k-Means”).
We also describe the Pig progress notification listener in Chapter 8.
▪ We look at the new EvalFunc interface in Chapter 9, including the topics of compile-time
evaluation, shipping dependent JARs automatically, and variable-length inputs. The new
评论0