hadoop使用api上传文件
时间: 2023-11-24 22:06:51 浏览: 36
Hadoop提供了Java API来上传文件,以下是上传文件的步骤:
1. 创建Configuration对象并指定HDFS的URI和文件系统类型。
```
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://localhost:9000");
conf.set("fs.file.impl", "org.apache.hadoop.fs.LocalFileSystem");
```
2. 获取FileSystem对象。
```
FileSystem fs = FileSystem.get(conf);
```
3. 创建Path对象,指定要上传的文件路径和目标路径。
```
Path srcPath = new Path("/local/path/to/file.txt");
Path dstPath = new Path("/hdfs/path/to/destination");
```
4. 使用FileSystem对象的copyFromLocalFile方法上传文件。
```
fs.copyFromLocalFile(srcPath, dstPath);
```
完整代码如下:
```
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
public class HadoopFileUpload {
public static void main(String[] args) {
try {
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://localhost:9000");
conf.set("fs.file.impl", "org.apache.hadoop.fs.LocalFileSystem");
FileSystem fs = FileSystem.get(conf);
Path srcPath = new Path("/local/path/to/file.txt");
Path dstPath = new Path("/hdfs/path/to/destination");
fs.copyFromLocalFile(srcPath, dstPath);
System.out.println("File uploaded successfully");
} catch (Exception e) {
e.printStackTrace();
}
}
}
```
注意:在运行代码之前,请确保Hadoop已经启动,并且指定的目标路径存在。