archive_types
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
archive_types [2024/04/14 16:24] – [What's the difference between the ZSTD levels?] johnsanc | archive_types [2024/04/27 09:15] (current) – [Table] johnsanc | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== Archive Types ====== | ====== Archive Types ====== | ||
- | RomVault supports several different archive options for your ROM sets. Unlike most other ROM managers, RomVault focuses on consistency and normalizes archives. Everyone' | + | RomVault supports several different archive options for your ROM sets. Unlike most other ROM managers, RomVault focuses on consistency and normalizes archives. All archives that RomVault creates are deterministic. Everyone' |
^ Icon ^ Type ^ Compression | ^ Icon ^ Type ^ Compression | ||
Line 7: | Line 7: | ||
| {{ : | | {{ : | ||
| {{ : | | {{ : | ||
- | | {{ : | + | | {{ : |
- | | {{ : | + | | {{ : |
| {{ : | | {{ : | ||
| {{ : | | {{ : | ||
Line 14: | Line 14: | ||
==== Which archive type should I use? ==== | ==== Which archive type should I use? ==== | ||
- | It depends. Each archive type has pros and cons, but you don't need to choose just one. Choose an archive type that best fits your use case for a particular collection. For example, if you plan to extract discs from archives to load into an emulator, you may prefer ZSTD Zip due its fast decompression speed. That being said, ZSTD Zip is essentially a modernized TrrntZip and seems to be the most popular choice since it was introduced with RomVault 3.7.0. | + | It depends. Each archive type has pros and cons, but you don't need to choose just one. Choose an archive type that best fits your use case for a particular collection. For example, if you plan to extract discs from archives to load into an emulator, you may prefer ZSTD Zip due its fast decompression speed. That being said, ZSTD Zip is essentially a modernized TrrntZip and seems to be the next most popular choice since it was introduced with RomVault 3.7.0. |
Line 26: | Line 26: | ||
* Its designed to be highly tunable replacement for Deflate | * Its designed to be highly tunable replacement for Deflate | ||
* Its now well established and used for filesystem compression with ZFS and BTRFS | * Its now well established and used for filesystem compression with ZFS and BTRFS | ||
- | * Its gaining popularity and support for ZSTD Zip archives was added to MAME v0.262 in January 2024 | + | * Its gaining popularity and support for ZSTD Zip archives |
* It supports multi-threaded compression | * It supports multi-threaded compression | ||
- | * Its designed to be seekable with block sizes up to 128 KiB (compared to Deflate' | ||
* Its standard levels are 1-19, and ultra levels are 20-22 which require significantly more memory | * Its standard levels are 1-19, and ultra levels are 20-22 which require significantly more memory | ||
+ | * Its updated periodically, | ||
==== How does the compression ratio of ZSTD compare to Deflate and LZMA? ==== | ==== How does the compression ratio of ZSTD compare to Deflate and LZMA? ==== | ||
- | ZSTD level 19 has a compression ratio right in between Deflate level 9 and LZMA level 9. However, since ZSTD is multithreaded | + | ZSTD level 19 has a compression ratio right in between Deflate level 9 and LZMA level 9. However, since ZSTD is multi-threaded |
+ | ^ Format | ||
+ | | File | ||
+ | | TrrntZip | ||
+ | | ZSTD Zip | ||
+ | | Solid LZMA 7z | 2, | ||
+ | The diagram below shows the relationship of single-threaded compression speed and ratio. Most ZSTD use cases would involve multiple workers (threads) on modern hardware and compress much faster. The three callouts in the diagram represent the compression levels that RomVault uses for each method. | ||
+ | {{: | ||
==== Why is ZSTD slow when compressing archives with many smaller files? ==== | ==== Why is ZSTD slow when compressing archives with many smaller files? ==== | ||
RomVault' | RomVault' | ||
- | ==== Why does RomVault use ZSTD level 19? Why not level 22? Why can't I choose my own? ==== | + | ==== Why does RomVault use ZSTD level 19 instead of 22? Why can't I choose my own? ==== |
There are several reasons: | There are several reasons: | ||
* RomVault has always focused on consistency and normalization, | * RomVault has always focused on consistency and normalization, | ||
- | * Levels 20-22 are considered " | + | |
- | * The space savings | + | |
- | * Levels above 19 will often have significantly slower compression | + | * Levels |
+ | * Levels above 19 are not as performant in most multithreaded scenarios | ||
Line 53: | Line 61: | ||
* Window Log: The window size represented as an exponent, 2^X bytes. | * Window Log: The window size represented as an exponent, 2^X bytes. | ||
* Window Size: The size of the sliding window the algorithm uses. Larger window sizes can match patterns further away, but at the cost of higher memory utilization and speed. | * Window Size: The size of the sliding window the algorithm uses. Larger window sizes can match patterns further away, but at the cost of higher memory utilization and speed. | ||
- | * Job Size: The amount of a data allocated to a single worker (thread). | + | * Job Size: The amount of data allocated to a single worker (thread). |
* Strategy: The specific compression approach used. | * Strategy: The specific compression approach used. | ||
archive_types.1713137046.txt.gz · Last modified: 2024/04/14 16:24 by johnsanc