First test with 96 core server using zstd to compress 25,881,600 tweets.
time zstd -qcd tweets.zst | zstd -v -19 --long=31 -T96 > test.zst
5.07% (238960762511 => 12123745253 bytes)
real 34m31.667s
user 1599m22.198s
Conversation
This Tweet was deleted by the Tweet author. Learn more
Replying to
jason@instance-3:~$ cat /proc/cpuinfo | head -n 15
processor: 0
vendor_id: GenuineIntel
cpu family: 6
model: 85
model name: Intel(R) Xeon(R) CPU @ 2.00GHz
stepping: 3
microcode: 0x1
cpu MHz: 2000.176
cache size: 39424 KB
1
This Tweet was deleted by the Tweet author. Learn more
This Tweet was deleted by the Tweet author. Learn more
This Tweet was deleted by the Tweet author. Learn more