We've talked about data compression in memory access. There's no surprise that compression can be useful in disk access. Hadoop provides a wide range of options that you can compress the intermediate files, and the output file. This may have a significant impact on the execution time of a mapreduce job. Here's the related document1 and document2 from cloudera.
We've talked about data compression in memory access. There's no surprise that compression can be useful in disk access. Hadoop provides a wide range of options that you can compress the intermediate files, and the output file. This may have a significant impact on the execution time of a mapreduce job. Here's the related document1 and document2 from cloudera.