flink统计每个人的总成绩样例打包执行代码
时间: 2024-03-16 12:48:04 浏览: 94
flink样例完整代码
以下是一个使用 Flink 统计每个人的总成绩的样例代码:
```java
import org.apache.flink.api.common.functions.AggregateFunction;
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;
public class ScoreTotal {
public static void main(String[] args) throws Exception {
ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
// 读取成绩数据,每行数据格式为:姓名 成绩
DataSet<String> scores = env.readTextFile("path/to/scores.txt");
// 使用 flatMap 将每行数据转换成 Tuple2<姓名, 成绩>
DataSet<Tuple2<String, Integer>> nameScores = scores.flatMap(new FlatMapFunction<String, Tuple2<String, Integer>>() {
@Override
public void flatMap(String value, Collector<Tuple2<String, Integer>> out) throws Exception {
String[] fields = value.split(" ");
out.collect(new Tuple2<>(fields[0], Integer.parseInt(fields[1])));
}
});
// 使用 AggregateFunction 统计每个人的总成绩
DataSet<Tuple2<String, Integer>> totalScores = nameScores
.groupBy(0) // 按姓名分组
.aggregate(new SumAggregateFunction()); // 使用自定义的 SumAggregateFunction 统计总成绩
// 打印每个人的总成绩
totalScores.print();
}
// 自定义的 AggregateFunction,用于计算总成绩
public static class SumAggregateFunction implements AggregateFunction<Tuple2<String, Integer>, Tuple2<String, Integer>, Tuple2<String, Integer>> {
@Override
public Tuple2<String, Integer> createAccumulator() {
return new Tuple2<>("", 0);
}
@Override
public Tuple2<String, Integer> add(Tuple2<String, Integer> value1, Tuple2<String, Integer> value2) {
return new Tuple2<>(value1.f0, value1.f1 + value2.f1);
}
@Override
public Tuple2<String, Integer> getResult(Tuple2<String, Integer> accumulator) {
return accumulator;
}
@Override
public Tuple2<String, Integer> merge(Tuple2<String, Integer> a, Tuple2<String, Integer> b) {
throw new UnsupportedOperationException();
}
}
}
```
你可以将你的成绩数据放在 `path/to/scores.txt` 文件中,每行一个人的成绩数据,格式为 `姓名 成绩`。运行以上代码后,会输出每个人的总成绩。
阅读全文