"In the respective words of four experienced modellers, the code is “deeply riddled” with bugs, “a fairly arbitrary Heath Robinson machine”, has “huge blocks of code – bad practice” and is “quite possibly the worst production code I have ever seen”."https://twitter.com/DavidDavisMP/status/1259421989648904197 …
-
-
Replying to @mattwridley @afneil
In the words of
@ID_AA_Carmack (famously skilled programmer; made Doom and Quake), who assisted Microsoft in cleaning up the code for release: "Bugs were found and fixed, but generally in paths that weren't enabled or hit".3 replies 0 retweets 0 likes -
Replying to @jonrimmer @mattwridley and
"That was my fear — what if the code turned out to be a horror show, making all the simulations questionable? I can’t vouch for the actual algorithms, but the software engineering seems fine."
1 reply 0 retweets 0 likes -
Replying to @jonrimmer @mattwridley and
That speaks to the hobbyist standards of Carmack then. Have you looked yourself? Non-deterministic (same seed, same inputs = random outputs), zero unit, integration or regression tests. You can't tell what it's meant to do, or prove that it does it.https://github.com/mrc-ide/covid-sim/tree/master/src …
1 reply 0 retweets 0 likes -
Replying to @RogertheBorg1 @jonrimmer and
Go build the code and run it single threaded with the same seed. You get the same results. There was a regression check doing that during the brief time I worked on it. Nobody is going to defend this code as a paragon of style, but it is functional.
4 replies 1 retweet 3 likes -
Replying to @ID_AA_Carmack @RogertheBorg1 and
Getting seriously different results for different seeds isn't even a sign of anything being wrong. In the real world, too, epidemic spread depends a lot on random events. It only took one person to infect the Diamond Princess; that easily might not have happened.
1 reply 0 retweets 0 likes -
Replying to @NYarvin @ID_AA_Carmack and
Are you actually sane? Accidental randomness is a bug, not a feature. Why have a pseudo random seed if the result is non-deterministic anyway? I thank God every day that I work with people with higher standards than that.
1 reply 0 retweets 1 like -
Replying to @RogertheBorg1 @ID_AA_Carmack and
The proper way to do this is to do a lot of runs and report the average and standard deviation. (Which they tell you to do, but really the code should do it itself.) But with the transmissibility data being garbage, the output has to be garbage anyway.
1 reply 0 retweets 0 likes -
Replying to @NYarvin @ID_AA_Carmack and
The proper way to do this is to have a deterministic output for a given pseudo-random seed and inputs. Sure, run it with different seeds, but do you genuinely not understand that ICL's position is that it's non-deterministic for the same seed, or why that's a problem?
1 reply 0 retweets 0 likes
The extra non-determinism is only with the multithreaded version, which means it comes from OS scheduling. It's just calls to the random number generator happening in a different order, not some strange Heisenbug that renders the whole output questionable.
-
-
Replying to @NYarvin @ID_AA_Carmack and
"Only"? "Just"? Did you just wake up from a coma and think it's still 1983? How do they make those inconceivably huge numbers fit into just 8 bits?
0 replies 0 retweets 1 likeThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.