I've said this before, but: "Emissions test driven development" http://www.nytimes.com/2016/04/27/business/international/vw-presentation-in-06-showed-how-to-foil-emissions-tests.html?smid=tw-nytimes&smtyp=cur …
-
-
Replying to @qntm
More generally, this is what happens when you have adversarial TDD; when the tests and the code are written by people with conflicting goals
1 reply 4 retweets 3 likes -
Replying to @qntm
We tried adversarial TDD at work once, as an exercise. One person writes one test, the other writes the minimum of code to make it pass
2 replies 5 retweets 2 likes -
Replying to @qntm
We discovered that if e.g. the first test case expects a method to return 1, the first implementation of the method will return 1 every time
1 reply 4 retweets 1 like -
Replying to @qntm
If the second test case expects 2, the implementation will simply be revised to return 1 and 2 alternately.
1 reply 5 retweets 1 like -
Replying to @qntm
The tester had to write a *lot* of tests before the developer gave up and implemented the implementation that was really being tested for
1 reply 4 retweets 2 likes -
Replying to @qntm
And in principle, there is *no* point at which the developer genuinely has to give up. They can fake a working implementation forever.
1 reply 0 retweets 1 like -
Replying to @qntm
If the tester and the developer are not on the same side, it is VERY difficult for the tester to force out the implementation they want
1 reply 1 retweet 2 likes -
Replying to @qntm
This is at the unit test case level. At higher levels, fakery becomes harder, but is still possible if the developer is motivated
1 reply 0 retweets 1 like
The crucial point is this: it needs to be impossible to computationally distinguish an emissions test from regular driving
-
-
Replying to @qntm
If an engine passes your test, and later you wish it hadn't, harden your test. The engine manufacturer is not on your side
1 reply 0 retweets 2 likes -
Replying to @qntm
Here's the critical diagram. Coloured lines change VW engine behaviour if crossed. White line = emissions test http://lwn.net/Articles/670604/ …
1 reply 1 retweet 2 likes - 6 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.