I just compressed one hour of verified Twitter data using these Zstandard options:
zstd -v -22 --ultra --long TW_verified_2019-11-28-22 --zstd=wlog=30,hlog=26,clog=28,slog=26,mml=7,tlen=999,ovlog=9,lhlog=26,lblog=8,lmml=4096
Compression Ratio: 5.62% (337174007=> 18940872 bytes
Conversation
Replying to
zstd is awesome especially the improvements in 1.4.4 - my benchmarks at twitter.com/centminmod/sta 😎
Quote Tweet
Compression algorithm benchmarks zstd 1.4.4 vs brotli vs gzip vs pigz vs pbzip2 vs pxz community.centminmod.com/threads/18669/ #compression #zstd #brotli #gzip #xz #bzip2 #lz4 #sysadmin #devops #linux #webpef #linux #pigz
2
1
Replying to
Yes! I have been very impressed with zstandard and each new version seems to improve compression ratios.
I'm really hoping that ZFS can incorporate zstandard compression sometime around the release of Ubuntu 20.04 LTS.