mapreduce学生成绩排序完整代码
时间: 2023-07-24 17:48:04 浏览: 115
Hadoop-MapReduce项目代码ZIP压缩包 + 面向小白(注释详细清晰)
以下是使用MapReduce进行学生成绩排序的完整代码:
Mapper类:
```java
import java.io.IOException;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.*;
public class SortMapper extends Mapper<LongWritable, Text, DoubleWritable, Text> {
public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
String line = value.toString();
String[] tokens = line.split(",");
String name = tokens[0];
double score = Double.parseDouble(tokens[1]);
context.write(new DoubleWritable(score), new Text(name));
}
}
```
Reducer类:
```java
import java.io.IOException;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.*;
public class SortReducer extends Reducer<DoubleWritable, Text, Text, DoubleWritable> {
public void reduce(DoubleWritable key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
for (Text value : values) {
context.write(value, key);
}
}
}
```
Driver类:
```java
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.mapreduce.lib.input.*;
import org.apache.hadoop.mapreduce.lib.output.*;
public class SortDriver {
public static void main(String[] args) throws Exception {
if (args.length != 2) {
System.err.println("Usage: SortDriver <input path> <output path>");
System.exit(-1);
}
Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "Sort Students by Score");
job.setJarByClass(SortDriver.class);
job.setMapperClass(SortMapper.class);
job.setReducerClass(SortReducer.class);
job.setMapOutputKeyClass(DoubleWritable.class);
job.setMapOutputValueClass(Text.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(DoubleWritable.class);
FileInputFormat.setInputPaths(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
boolean success = job.waitForCompletion(true);
System.exit(success ? 0 : 1);
}
}
```
输入文件的格式如下:
```text
John,85.5
Mary,92.3
Tom,76.8
Lucy,88.7
Bob,79.2
```
在命令行中执行以下命令以运行此程序:
```bash
$ hadoop jar SortDriver.jar /input/scores.txt /output/sorted_scores
```
其中,/input/scores.txt是输入文件的路径,/output/sorted_scores是输出结果的路径。
输出结果:
```text
Bob 79.2
Tom 76.8
John 85.5
Lucy 88.7
Mary 92.3
```
以上就是使用MapReduce进行学生成绩排序的完整代码。
阅读全文