Conversation

JSON Baumgartner
@jasonbaumgartne
I just compressed one hour of verified Twitter data using these Zstandard options: zstd -v -22 --ultra --long TW_verified_2019-11-28-22 --zstd=wlog=30,hlog=26,clog=28,slog=26,mml=7,tlen=999,ovlog=9,lhlog=26,lblog=8,lmml=4096 Compression Ratio: 5.62% (337174007=> 18940872 bytes
4
27
@luca@vis.social
@luca
Replying to
@jasonbaumgartne
I need to test your options. The compression is better than anything I got. Only ppmd got close.
Quote Tweet
@luca@vis.social
@luca
Conclusion: PPMd gave me the best compression results for a 64MB JSON file. Small size (4MB) and super fast (7 seconds). Other algorithms that are available in 7zip work okay but compressed up to 2 times bigger. The default one (LZMA2) takes 30s and is 5MB. DEFLATE worst: 9MB.
Show this thread
cmix18 (not in 7zip) 3.70% PPMd 7z 6.28% PPMd zip 6.56% PPMd 7z 7.70% LZMA 7z 7.85% LZMA zip 7.85% LZMA2 xz 7.85% LZMA2 xz 7.85% LZMA2 7z 7.85% Zstandard ZSTD 8.43% LZMA2 7z 8.51% Brotli br 9.17% bzip2 bz2 9.18% bzip2 bz2 9.18% bzip2 7z 9.18% bzip2 zip 9.18% LZMA2 xz 9.40% LZMA2 7z 9.40% Zstandard ZSTD 10.53% Deflate64 zip 12.08% Deflate gz 12.51% Deflate zip 12.51% Deflate gz 13.10% Deflate zip 13.10% LZ5 lz5 13.55%