General Computing
windows compression 7-zip archiving
Updated Fri, 03 Jun 2022 17:13:33 GMT

Does compression into one large archive result in better compression than individual compression of folders?


I have several folders of around 8GB or so. Together these folders total around 60GB of data. I can compress these folders one of two ways: either individually, creating one compressed archive for each of them, or altogether into a single large compressed archive.

Generally speaking, assuming all the data to be compressed is of the same type and the compression algorithm used is the same (and that I also don't care about the time it would take to decompress the larger file), will either method result in better compression than another, or will the total sizes of the compressed files in the two scenarios tend to be equal?




Solution

Does compression into one large archive result in better compression than individual compression of folders? Not necessarily.

Only if the archive is using solid compression. A non-solid archive (like a Zip archive) compresses files individually. This enables you to easily decompress single files from the archive. It also allows you to add files to the archive without having to recompress everything.

With solid archives, all this is a lot harder: To decompress a file at the very end of the stream, everything has to be decompressed (though not necessarily written to disk). When adding a file, the algorithm also needs to go through everything.

There is a middle ground, however: Using solid blocks. Now the archiver doesnt have to process the entire file all the time but only some of the file.

In the 7-Zip GUI, its this option:

7-Zip Add dialog

Without taking into account the data being compressed, its really simple:

  • Non-solid: Fast interactive access, worst compression
  • Solid blocks: Somewhat efficient interactive access, better compression
  • Solid: No interactive access, best compression

Depending on the predicted access pattern, you should select a suitable variant.







External Links

External links referenced by this document: