I'm playing around with Knapsack Pro to automatically balance my RSpec tests across parallel CI jobs. It's pretty cool! knapsackpro.com
Looks like this could speed up my CI builds quite a bit
Conversation
Replying to
Seems to be a bit more balanced now, but it's broken some screenshot/PDF caching so now my test suite is a few minutes slower than it was with only 8 nodes. I'm going to need to rethink the caching stuff, maybe I'll use Redis on Google Cloud
1
I have some PDF integration tests that convert the pages into images and then do a visual diff to make sure nothing has changed. A good way to speed this up is to record the SHA256 hash of any PDFs that pass the visual check. If the SHA256 hash matches then I don't need to run it
1
My parallel test jobs were previously deterministic so the same test would always run on the same CI node, so I could cache everything normally. But now knapsack shifts tests around randomly, so I'm going to need a global cache server somewhere. So I'll probably use Redis
1
Oh it looks like Knapsack Pro actually did shave off 4 minutes, so now I'm down to 8 minutes. If I can fix the caching stuff and increase the parallel jobs, then I might even get down to ~3 minutes. That would be awesome
1
Another thing I'd like to try is to set up some seed data in the test database that I can re-use for all my tests, and then use database_cleaner with transactions. I'm currently just creating fixtures with FactoryBot. (Usually the same user and account, etc.)
1
I've got my CI builds down to 07:57. Not bad, but I hope I can do better
