HDFS压缩和解压缩

HDFS supports compression and decompression of files to reduce storage space and improve data transfer efficiency. It provides built-in compression codecs that can be used to compress and decompress files stored in HDFS.

To compress a file in HDFS, you can use the `hadoop fs -D` command with the `mapreduce.output.fileoutputformat.compress` and `mapreduce.output.fileoutputformat.compress.codec` properties. For example, to compress a file using the Snappy codec, you can use the following command:

hadoop fs -D mapreduce.output.fileoutputformat.compress=true \
-D mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec \
-cp <source_file> <destination_file>

To decompress a compressed file in HDFS, you don't need to explicitly decompress it. HDFS automatically decompresses the file when it is read by a MapReduce job or any other application that supports compression codecs.

Note that the compression and decompression are transparent to applications reading or writing data to HDFS. The data is automatically compressed or decompressed based on the configuration settings.