HADOOP ECLIPSE 统计这些人的共同好友 完整代码

以下是使用Hadoop和Eclipse开发的统计共同好友的完整代码：

Mapper类代码：

public class FriendsMapper extends Mapper<LongWritable, Text, Text, Text> {

    private Text outputKey = new Text();
    private Text outputValue = new Text();

    @Override
    protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {

        String line = value.toString();
        String[] split = line.split(":");

        String person = split[0];
        String[] friends = split[1].split(",");

        for (String friend : friends) {
            outputKey.set(getKey(person, friend));
            outputValue.set(person);
            context.write(outputKey, outputValue);
        }

    }

    private String getKey(String person, String friend) {
        if (person.compareTo(friend) < 0) {
            return person + ":" + friend;
        } else {
            return friend + ":" + person;
        }
    }

}

Reducer类代码：

public class FriendsReducer extends Reducer<Text, Text, Text, Text> {

    private Text outputValue = new Text();

    @Override
    protected void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException {

        StringBuilder builder = new StringBuilder();
        for (Text value : values) {
            builder.append(value.toString()).append(",");
        }

        outputValue.set(builder.toString());
        context.write(key, outputValue);

    }

}

Driver类代码：

public class FriendsDriver {

    public static void main(String[] args) throws Exception {

        Configuration conf = new Configuration();
        Job job = Job.getInstance(conf, "Friends");

        job.setJarByClass(FriendsDriver.class);
        job.setMapperClass(FriendsMapper.class);
        job.setReducerClass(FriendsReducer.class);

        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);

        FileInputFormat.addInputPath(job, new Path(args[0]));
        FileOutputFormat.setOutputPath(job, new Path(args[1]));

        System.exit(job.waitForCompletion(true) ? 0 : 1);

    }

}

在Eclipse中配置好Hadoop环境后，将以上代码打包成jar文件，并将其上传到Hadoop集群上。然后运行以下命令：

hadoop jar Friends.jar Friends /input /output

其中，/input是输入文件的路径，/output是输出文件的路径。运行后，即可在输出文件中获得每对共同好友的列表。