springboot+hdfs+mysql实现文件上传下载

要实现Spring Boot与HDFS和MySQL的文件上传和下载，需要先配置Hadoop和MySQL环境。然后，需要添加相应的依赖项并编写以下代码：

1. 配置HDFS

在application.properties文件中添加以下配置：

# HDFS配置
hadoop.hdfs.path=hdfs://localhost:9000
hadoop.hdfs.username=hadoop

2. 配置MySQL

在application.properties文件中添加以下配置：

# MySQL配置
spring.datasource.url=jdbc:mysql://localhost:3306/test
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

3. 添加依赖项

在pom.xml文件中添加以下依赖项：

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-hdfs</artifactId>
    <version>3.2.1</version>
</dependency>
<dependency>
    <groupId>com.mysql.cj</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>8.0.23</version>
</dependency>

4. 编写上传和下载代码

上传代码：

@Service
public class HdfsService {
    @Value("${hadoop.hdfs.path}")
    private String hdfsPath;
    @Value("${hadoop.hdfs.username}")
    private String hdfsUsername;
    @Value("${spring.servlet.multipart.location}")
    private String uploadPath;
    @Autowired
    private FileSystem fileSystem;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    public void upload(MultipartFile file) throws IOException {
        String fileName = file.getOriginalFilename();
        String filePath = "/upload/" + fileName;
        Path path = new Path(hdfsPath + filePath);
        FSDataOutputStream outputStream = fileSystem.create(path);
        outputStream.write(file.getBytes());
        outputStream.close();
        jdbcTemplate.update("INSERT INTO file (name, path) VALUES (?, ?)", fileName, filePath);
    }
}

下载代码：

@Service
public class HdfsService {
    @Value("${hadoop.hdfs.path}")
    private String hdfsPath;
    @Value("${hadoop.hdfs.username}")
    private String hdfsUsername;
    @Value("${spring.servlet.multipart.location}")
    private String uploadPath;
    @Autowired
    private FileSystem fileSystem;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    public void download(HttpServletResponse response, String fileName) throws IOException {
        String filePath = jdbcTemplate.queryForObject("SELECT path FROM file WHERE name = ?", String.class, fileName);
        Path path = new Path(hdfsPath + filePath);
        FSDataInputStream inputStream = fileSystem.open(path);
        response.setContentType("application/octet-stream");
        response.setHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
        IOUtils.copy(inputStream, response.getOutputStream());
        response.flushBuffer();
    }
}

以上代码将文件存储在HDFS中，并将文件名和路径保存到MySQL中。下载时，从MySQL中查询文件路径并将文件流发送到响应中。注意，在这里我们使用了Apache Commons IO库的IOUtils类来将文件流复制到响应中。

同时，我们还需要在控制器中编写上传和下载的端点：

@RestController
public class FileController {
    @Autowired
    private HdfsService hdfsService;

    @PostMapping("/upload")
    public void upload(@RequestParam("file") MultipartFile file) throws IOException {
        hdfsService.upload(file);
    }

    @GetMapping("/download")
    public void download(HttpServletResponse response, @RequestParam("fileName") String fileName) throws IOException {
        hdfsService.download(response, fileName);
    }
}

现在，我们已经完成了Spring Boot与HDFS和MySQL的文件上传和下载。