mapreduce只输出key
时间: 2024-01-27 20:04:17 浏览: 35
如果使用MapReduce只输出key而不输出value的话,需要在Mapper中将value设置为null或者忽略value的输出。下面是一个简单的示例:
```java
public class MapOnlyJob extends Configured implements Tool {
public static class MapOnlyMapper extends Mapper<LongWritable, Text, Text, NullWritable> {
private Text outKey = new Text();
@Override
protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
// 将value设置为null,只输出key
outKey.set(value.toString().split(",")[0]);
context.write(outKey, NullWritable.get());
}
}
@Override
public int run(String[] args) throws Exception {
Configuration conf = getConf();
Job job = Job.getInstance(conf, "MapOnlyJob");
job.setJarByClass(MapOnlyJob.class);
job.setMapperClass(MapOnlyMapper.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(NullWritable.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(NullWritable.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
return job.waitForCompletion(true) ? 0 : 1;
}
public static void main(String[] args) throws Exception {
int res = ToolRunner.run(new Configuration(), new MapOnlyJob(), args);
System.exit(res);
}
}
```
在上面的示例中,Mapper的输出值类型被设置为NullWritable,表示忽略value的输出。在Mapper中,通过将value设置为null,只输出key。