mapreduve根据提示,在右侧编辑器补充代码,计算出每个手机号码的一年总流量。 main 方法已给出,其中 Job 和输入输出路径已配置完成,无需更改; map 和 reduce 的输入输出 key、value 已给出; 编程中直接写 map 与 reduce 过程的主要内容即可。
时间: 2024-02-25 19:58:12 浏览: 106
大数据学习(八):mapreduce编程案例-计算每个手机号访问网站请求流量、相应流量和流量总和根据手机号判断省份进行分类
以下是实现该功能的示例代码:
```java
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
public class PhoneFlow {
public static class Map extends Mapper<LongWritable, Text, Text, LongWritable> {
@Override
protected void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
String line = value.toString();
String[] fields = line.split("\\s+");
String phone = fields[0];
long upFlow = Long.parseLong(fields[1]);
long downFlow = Long.parseLong(fields[2]);
context.write(new Text(phone), new LongWritable(upFlow + downFlow));
}
}
public static class Reduce extends Reducer<Text, LongWritable, Text, LongWritable> {
@Override
protected void reduce(Text key, Iterable<LongWritable> values, Context context)
throws IOException, InterruptedException {
long sum = 0;
for (LongWritable value : values) {
sum += value.get();
}
context.write(key, new LongWritable(sum));
}
}
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "PhoneFlow");
job.setJarByClass(PhoneFlow.class);
job.setMapperClass(Map.class);
job.setReducerClass(Reduce.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(LongWritable.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}
```
在上述代码中,Map类继承自Mapper类,用于实现Map操作,将输入的每一行数据按照手机号码为键,总流量为值进行映射。Reduce类继承自Reducer类,用于实现Reduce操作,将相同手机号码的总流量进行合并,并输出结果。在main方法中,将Map和Reduce类设置为作业的Mapper和Reducer,设置输入输出路径和输出格式,最后启动作业并等待完成。
阅读全文