Miss Dada  🏳️‍⚧️

Saving you some math, our database is 3.6GB in total. Roughly 85% of that is counting downloads

CocoaPods had close to 3tb of download data before I had to shut that down bc $$$$

Yeah, we figure we'll eventually either squash old data, or just upload a backup of the table to S3 and then delete old partitions. Our web UI only ever accesses the last 90 days. Our API supports arbitrary dates (with a max range of 90 days at a time), but I have no stats on use

In the short term, our DB is 3.6GB, our max is 64GB, and we're not growing at any alarming rate. We'll end up upgrading to a tier that has 256GB storage to increase our connection limit before we hit storage limits I think

We very much are discussing this, because we definitely will eventually hit storage limits, and it makes no sense to increase our DB bill to keep rarely/never accessed historical download stats in the primary DB. But for the time being we're just ridiculously far off from that.

I'm actually really surprised cocoapods used so much space for download data. Was this a record per-download or something? (For reference, ours is stored as (version_id, date, downloads_that_day, downloads_added_to_other_caches, processed) or (i32, i32, i32, i32, bool)