Back on my "essays as commit messages" shithttps://github.com/rust-lang/crates.io/pull/2203 …
Show this thread
-
That's tough. But maybe you could use a more compact format in the db? Daily (version_id, date) adds ~200K identical dates every day. It could be (version_id, week_number) with a downloads[7] column, for example.
If we had a problem we were trying to solve, sure. But all our time sensitive queries are on version id and date, the current structure is perfect for that
-
-
I've meant that as a way to reduce data size, given that the sheer volume of the data is causing you pains. Normalized table has 4-byte counters with ~50 bytes of overhead each. Merging them to be one row per version per week (or longer) should considerably reduce table size.
-
The volume of data isn't, so much as how it's being accessed/laid out. Compressing rows would only delay the problem, and would likely slow down access patterns that are actually important to be fast. Only painful place right now is background jobs
- 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.