Spring Hadoop
1.0.0.RELEASE
Spring for Apache Hadoop
Reference Manual ii
Table of Contents
Preface ..................................................................................................................................... iv
I. Introduction ............................................................................................................................. 1
1. Requirements ................................................................................................................. 2
2. Additional Resources ...................................................................................................... 3
II. Spring and Hadoop ................................................................................................................ 4
3. Hadoop Configuration, MapReduce, and Distributed Cache .............................................. 5
3.1. Using the Spring for Apache Hadoop Namespace ................................................. 5
3.2. Configuring Hadoop ............................................................................................. 6
3.3. Creating a Hadoop Job ........................................................................................ 9
Creating a Hadoop Streaming Job .................................................................... 10
3.4. Running a Hadoop Job ...................................................................................... 10
Using the Hadoop Job tasklet ........................................................................... 11
3.5. Running a Hadoop Tool ..................................................................................... 11
Replacing Hadoop shell invocations with tool-runner ..................................... 13
Using the Hadoop Tool tasklet .......................................................................... 13
3.6. Running a Hadoop Jar ....................................................................................... 13
Using the Hadoop Jar tasklet ............................................................................ 15
3.7. Configuring the Hadoop DistributedCache ..................................................... 15
3.8. Map Reduce Generic Options ............................................................................ 16
4. Working with the Hadoop File System ........................................................................... 17
4.1. Configuring the file-system ................................................................................. 17
4.2. Scripting the Hadoop API ................................................................................... 18
Using scripts .................................................................................................... 20
4.3. Scripting implicit variables .................................................................................. 20
Running scripts ................................................................................................ 21
Using the Scripting tasklet ................................................................................ 21
4.4. File System Shell (FsShell) ................................................................................ 22
DistCp API ....................................................................................................... 23
5. Working with HBase ..................................................................................................... 24
5.1. Data Access Object (DAO) Support .................................................................... 24
6. Hive integration ............................................................................................................ 26
6.1. Starting a Hive Server ....................................................................................... 26
6.2. Using the Hive Thrift Client ................................................................................ 26
6.3. Using the Hive JDBC Client ............................................................................... 27
6.4. Running a Hive script or query ........................................................................... 27
Using the Hive tasklet ...................................................................................... 28
6.5. Interacting with the Hive API .............................................................................. 28
7. Pig support .................................................................................................................. 30
7.1. Running a Pig script .......................................................................................... 30
Using the Pig tasklet ........................................................................................ 31
7.2. Interacting with the Pig API ................................................................................ 31
8. Cascading integration ................................................................................................... 32
8.1. Using the Cascading tasklet ............................................................................... 35
8.2. Using Scalding .................................................................................................. 35
8.3. Spring-specific local Taps .................................................................................. 36
9. Using the runner classes .............................................................................................. 38
10. Security Support ......................................................................................................... 40
我的内容管理
收起
我的收益 登录查看自己的收益
我的积分
登录查看自己的积分
我的C币
登录后查看C币余额
我的收藏 
我的下载 
下载帮助 
评论1