Jon Pretty

I actually thought we should go the opposite direction and have tests in a different repository, evolving independently of the code they're testing. This would make it easier to check the behaviour of past as well as future versions of the code.

tbh that sounds like a tooling nightmare

I think only in the context of what we're currently familiar with. Trying to sell someone our current universe from my hypothetical alternative universe would probably make it seem a nightmare to them, too...

Fair point, so let's see: How would you handle compatibility between main and test repositories?

What do you mean by compatibility here?

It's difficult enough to keep compatibility across versions when you're programming against a public API. With tests, you often want to check internals.

Oh, you wouldn't, but the fact it's hard at the moment is sort of the point: If the tests aren't in sync with your main repo, then some of them would just fail (at compile time). You would update them if/when you care enough to do so, but no requirement for it to be synchronous.

That sounds like a recipe for deterioration.