简答题 在虚拟机里面编写WordCount示例程序

在虚拟机中编写 WordCount 示例程序通常是在Hadoop MapReduce框架下完成的，因为WordCount是一个经典的MapReduce任务，用于计算文本文件中单词的频率。以下是简单的步骤：

1. **安装Hadoop**：首先需要在你的虚拟机上安装Hadoop环境，包括HDFS（分布式文件系统）和YARN（资源管理器）。

2. **创建Mapper**：创建一个Mapper类，它负责读取输入数据行，解析出单词，并对每个单词调用`emit()`函数将键值对发送到Reducer。例如：

   public class WordCountMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
       private final static IntWritable one = new IntWritable(1);
       private Text word = new Text();

       public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
           String line = value.toString();
           StringTokenizer tokenizer = new StringTokenizer(line);
           while (tokenizer.hasMoreTokens()) {
               word.set(tokenizer.nextToken());
               context.write(word, one);
           }
       }
   }

3. **创建Reducer**：Reducer接收Mapper发送过来的所有相同键的值，然后进行累加。这里我们只需要简单地合并计数：

   public class WordCountReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
       private IntWritable result = new IntWritable();

       public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
           int sum = 0;
           for (IntWritable val : values) {
               sum += val.get();
           }
           result.set(sum);
           context.write(key, result);
       }
   }

4. **配置Job**：在主程序中配置JobConf，并指定Mapper、Reducer类以及输入和输出路径：

   Configuration conf = new Configuration();
   Job job = Job.getInstance(conf, "word count");
   job.setJarByClass(WordCount.class);
   job.setMapperClass(WordCountMapper.class);
   job.setCombinerClass(WordCountReducer.class); // 可选，优化减少网络传输
   job.setReducerClass(WordCountReducer.class);
   job.setOutputKeyClass(Text.class);
   job.setOutputValueClass(IntWritable.class);

   FileInputFormat.addInputPath(job, new Path("input_path"));
   FileOutputFormat.setOutputPath(job, new Path("output_path"));

5. **提交并运行**：最后，调用`job.waitForCompletion(true)`来启动作业。