What is your favorite "best practice" when designing and implementing computational models? Mine is: As far as possible, isolate stochasticity.
-
Show this thread
-
For example, if you have a function that does sampling, then applies a transformation to the realizations, you should have *two* functions, even if the combination is short. It's worth the line count for testability.
1 reply 0 retweets 6 likesShow this thread -
Replying to @generativist
Interesting. Are there other ways to isolate stochasticity?
1 reply 0 retweets 1 like -
Replying to @kareem_carr
Well you're an R user (I forget which new term one), so I'm not sure. But, testing tools named something like "QuickCheck" are really useful. I wrote this one for Python a while back:https://github.com/jbn/pytest-caprng …
1 reply 0 retweets 1 like -
Replying to @generativist @kareem_carr
Basically, if you run a unit test on some stochastic code and it fails, my package freezes the PRNG state so that on the next run, it's identical. Making failing tests deterministically reproducible is very useful.
1 reply 0 retweets 2 likes -
Replying to @generativist
Interesting. I think most statistical programmers I know fix the random seed for tests. We learn to do it in class assignments so the grader can reproduce our stuff.
1 reply 0 retweets 2 likes -
Replying to @kareem_carr
Yea, I think that's the common training. My problem with that is that stochastic "fuzzing" is useful for uncovering bugs. Sometimes, your fixed seed generates realizations that are unfortunate w.r.t. a specific test. Quickcheck-style testing removes this risk...
1 reply 0 retweets 2 likes -
Replying to @generativist @kareem_carr
...but, if ill-designed, may introduce sporadic errors at a rate of p, which is a different risk -- you may learn to ignore those errors!
1 reply 0 retweets 2 likes
I don't think my view is orthodoxy at all though. It's essentially, "I'd rather constantly run stochastic-albeit-reproducible-on-failure tests to verify my expectations than occasionally run more expensive, fuller fuzzing tests." Mostly, it aligns with my dev habits well.
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.