在客户端的IDEA中创建项目hdfsAPI,创建并编辑源程序hdfsTest.java,完成以下功能:(1)判断HDFS是否存在/Test目录,如果不存在,则创建/Test目录。 (2)向HDFS上传文件(以自己姓名拼音+学号作为文件名,例如zhangsan20250001.txt) (3)下载(2)上传的文件到本地 (4)将(2)上传文件的内容输出到终端 (5)查看(2)上传文件的路径、读写权限、大小、创建时间等信息 (6)向(2)上传的文件追加内容 (7)重命名(2)上传的文件 (8)删除(2)上传的文件
时间: 2023-10-29 16:06:57 浏览: 91
好的,我会逐一回答你的问题。
首先,为了完成上述功能,需要安装Hadoop并启动HDFS服务,同时需要在IDEA中配置Hadoop的环境变量和相关配置。
接下来,我将按照你的要求逐一回答你的问题:
1. 判断HDFS是否存在/Test目录,如果不存在,则创建/Test目录。
可以使用以下代码实现:
```java
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
public class hdfsTest {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
Path path = new Path("/Test");
if (fs.exists(path)) {
System.out.println("目录已存在!");
} else {
fs.mkdirs(path);
System.out.println("目录创建成功!");
}
fs.close();
}
}
```
2. 向HDFS上传文件(以自己姓名拼音+学号作为文件名,例如zhangsan20250001.txt)
可以使用以下代码实现:
```java
import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
public class hdfsTest {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create("hdfs://localhost:9000"), conf);
String filename = "zhangsan20250001.txt";
Path path = new Path("/Test/" + filename);
InputStream in = hdfsTest.class.getClassLoader().getResourceAsStream(filename);
fs.create(path).write(read(in));
System.out.println("上传成功!");
fs.close();
}
public static byte[] read(InputStream in) throws IOException {
byte[] buf = new byte[1024];
int len;
ByteArrayOutputStream out = new ByteArrayOutputStream();
while ((len = in.read(buf)) != -1) {
out.write(buf, 0, len);
}
in.close();
out.close();
return out.toByteArray();
}
}
```
3. 下载(2)上传的文件到本地
可以使用以下代码实现:
```java
import java.io.FileOutputStream;
import java.io.InputStream;
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
public class hdfsTest {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create("hdfs://localhost:9000"), conf);
String filename = "zhangsan20250001.txt";
Path path = new Path("/Test/" + filename);
InputStream in = fs.open(path);
FileOutputStream out = new FileOutputStream(filename);
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) != -1) {
out.write(buf, 0, len);
}
in.close();
out.close();
fs.close();
System.out.println("下载成功!");
}
}
```
4. 将(2)上传文件的内容输出到终端
可以使用以下代码实现:
```java
import java.io.InputStream;
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
public class hdfsTest {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create("hdfs://localhost:9000"), conf);
String filename = "zhangsan20250001.txt";
Path path = new Path("/Test/" + filename);
InputStream in = fs.open(path);
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) != -1) {
System.out.print(new String(buf, 0, len));
}
in.close();
fs.close();
}
}
```
5. 查看(2)上传文件的路径、读写权限、大小、创建时间等信息
可以使用以下代码实现:
```java
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.FileStatus;
import java.net.URI;
import java.text.SimpleDateFormat;
import java.util.Date;
public class hdfsTest {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create("hdfs://localhost:9000"), conf);
String filename = "zhangsan20250001.txt";
Path path = new Path("/Test/" + filename);
FileStatus status = fs.getFileStatus(path);
System.out.println("路径:" + status.getPath());
System.out.println("读写权限:" + status.getPermission());
System.out.println("大小:" + status.getLen() + " bytes");
System.out.println("创建时间:" + formatTime(status.getModificationTime()));
fs.close();
}
public static String formatTime(long time) {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
return sdf.format(new Date(time));
}
}
```
6. 向(2)上传的文件追加内容
可以使用以下代码实现:
```java
import java.io.FileInputStream;
import java.io.InputStream;
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
public class hdfsTest {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create("hdfs://localhost:9000"), conf);
String filename = "zhangsan20250001.txt";
Path path = new Path("/Test/" + filename);
InputStream in = new FileInputStream(filename);
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) != -1) {
fs.append(path).write(buf, 0, len);
}
in.close();
fs.close();
System.out.println("追加成功!");
}
}
```
7. 重命名(2)上传的文件
可以使用以下代码实现:
```java
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import java.net.URI;
public class hdfsTest {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create("hdfs://localhost:9000"), conf);
String oldName = "zhangsan20250001.txt";
String newName = "lisi20250002.txt";
Path oldPath = new Path("/Test/" + oldName);
Path newPath = new Path("/Test/" + newName);
fs.rename(oldPath, newPath);
fs.close();
System.out.println("重命名成功!");
}
}
```
8. 删除(2)上传的文件
可以使用以下代码实现:
```java
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import java.net.URI;
public class hdfsTest {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create("hdfs://localhost:9000"), conf);
String filename = "lisi20250002.txt";
Path path = new Path("/Test/" + filename);
fs.delete(path, true);
fs.close();
System.out.println("删除成功!");
}
}
```
以上就是关于使用Java操作HDFS的全部代码,希望可以帮助到你。
阅读全文