First test with 96 core server using zstd to compress 25,881,600 tweets.
time zstd -qcd tweets.zst | zstd -v -19 --long=31 -T96 > test.zst
5.07% (238960762511 => 12123745253 bytes)
real 34m31.667s
user 1599m22.198s
Conversation
This Tweet was deleted by the Tweet author. Learn more
Replying to
Google Cloud n1-highcpu-96 (96 vCPUs, 86.4 GB memory)
Translate Tweet
This Tweet was deleted by the Tweet author. Learn more
Replying to
I'm using pre-emptible which is ~$500 monthly ... about 3x that for standard.