hadoop jar /usr/local/share/hadoop/hadoop-3.1.1/share/hadoop/tools/lib/hadoop-streaming-3.1.1.jar \ > -input /data/sk20/sk200245/testData.txt \ > -output /data/sk20/sk200245/output \ > -mapper mapTest.py \ > -reducer redTest.py \ > -file /home/sk200245/mapTest.py \ > -file /home/sk200245/redTest.py 运行时出错2023-05-29 18:16:23,282 WARN streaming.StreamJob: -file option is deprecated, please use generic option -files instead. packageJobJar: [/home/sk200245/mapTest.py, /home/sk200245/redTest.py] [] /tmp/streamjob6323488470150726374.jar tmpDir=null 2023-05-29 18:16:24,229 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2023-05-29 18:16:24,279 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2023-05-29 18:16:24,279 INFO impl.MetricsSystemImpl: JobTracker metrics system started 2023-05-29 18:16:24,291 WARN impl.MetricsSystemImpl: JobTracker metrics system already initialized! 2023-05-29 18:16:24,371 ERROR streaming.StreamJob: Error Launching job : Output directory hdfs://192.168.189.131:9000/data/sk20/sk200245/output already exists Streaming Command Failed!
时间: 2023-08-06 21:08:27 浏览: 189
winutils.exe/hadoop-common-2.2.0.rar
这个错误提示告诉我们输出目录已经存在,而Hadoop默认情况下不会覆盖现有的输出目录。为了解决此问题,您需要删除现有的输出目录或更改输出路径。
您可以使用以下命令删除现有的输出目录:
```
hadoop fs -rm -r /data/sk20/sk200245/output
```
或者,您可以更改输出路径:
```
hadoop jar /usr/local/share/hadoop/hadoop-3.1.1/share/hadoop/tools/lib/hadoop-streaming-3.1.1.jar \
-input /data/sk20/sk200245/testData.txt \
-output /data/sk20/sk200245/output_new \
-mapper mapTest.py \
-reducer redTest.py \
-files /home/sk200245/mapTest.py,/home/sk200245/redTest.py
```
请注意,我们使用了`-files`选项代替了`-file`选项,并更改了输出路径。
阅读全文