continuous integration idea: make your buildbots run the testsuite twice, first without the non-test changes, second with them, to check that the tests, well, actually test something
17
24
171
Conversation
Replying to
This 👆 is why I'm adamant (unpopularly) that tests should be a separate repo, so you can run new versions of tests against arbitrary versions of the code being tested.
2
6
Replying to
this works only if there is an explicit API boundary between the code and the tests, but in that case, yes. I think it's not that your approach is unpopular, it's more that writing libc is not very popular :)
2
12
Replying to
Well, it's that stable APIs aren't popular. Sorry for omitting that part. I actually thought about mentioning it but that kind of information density on Twitter gets meh.
2
2
This also doesn't really work for unit testing the implementation details of high-level APIs. It can be really painful to do unit testing in a language / environment without good support for it. I tend to only write tests for the public API in languages with bad testing support.
1
1
I don't like the idea of making the code more complex with abstractions and dynamism to aid unit testing. Ends up meaning I write mostly integration tests. Unit testing pure functions in a language with decent support for it is nice, but other than that... it can be really awful.
I really like what Rust is doing with #[cfg(test)]/#[test]; more languages should be doing that
2
2
Yeah, it helps a lot with doing unit testing. Still end up in many situations where you would have to make abstractions to let you do unit testing. You could do it without runtime overhead using generics but it'd often be a mess, and just goes against how I want to do testing.
1