A lot of criticism circles around things like the fragility of our algorithms (e. repro), and this is well founded. But we should also be more self-aware -- why were we so invested in that one algorithm in the first place after just a single study?
-
Show this thread
-
Both types of reproducibility are necessary and highly intertwined, but IMO s. repro is the most important, and proper s. repro takes patience, care, and aggregate effort.
1 reply 0 retweets 9 likesShow this thread -
So, to conclude: 1) State your scientific claims in your papers! 2) Proper science guarantees that good ideas stand the test of time. Code sharing alone does not. 3) A single paper should often only contribute little to the truth value of a claim.
1 reply 5 retweets 44 likesShow this thread -
4) We should not be shocked when we can't replicate results. 5) Let's be a bit more patient before declaring that the field is undergoing a "crisis". 6) And finally, let's all read a bit more philosophy of science :)
5 replies 4 retweets 57 likesShow this thread -
Replying to @santoroAI
Cool thread. I talked about this at ICML a little. "In this talk I will discuss the different types of replicability/reproducibility, focussing heavily on the two that involve software/computational models."https://figshare.com/articles/Varieties_of_Reproducibility_in_Empirical_and_Computational_Domains/6818018 …
1 reply 2 retweets 6 likes -
Replying to @o_guest @santoroAI
"However useful it is to be able to re-run code from the past, it is often secondary to doing good science because checking that the spec is generally correct — i.e., the theory is actually computationally captured — is more important to science."
1 reply 1 retweet 5 likes -
Replying to @o_guest @santoroAI
"Implementation-only details, for example, might need to be upgraded to theory-level if they turn out to be imperative to modelling a certain effect."
1 reply 1 retweet 4 likes -
Replying to @o_guest @santoroAI
"And vice versa, theory-level assumptions could be relaxed if it is found that other important aspects of the theory are nonetheless captured with a variety of implementations."
2 replies 1 retweet 3 likes -
Replying to @o_guest @santoroAI
Interestingly analogous to one of the principles of Model-Driven (IT) Architecture: "Raise the level of abstraction", back at the turn of the century. Intended to make designs usable across implementations and easier to understand. http://www.methodsandtools.com/archive/archive.php?id=5 …
1 reply 1 retweet 3 likes -
(Probably too detailed to read it all now but I was amused by the idea that there's a level of abstraction where the ideas converge
)1 reply 1 retweet 1 like
I'll check this out! Thanks!
-
-
Replying to @o_guest @santoroAI
LMK if you are interested in it, I could probably save some time for you by finding something less cluttered with then-current technology. Unfortunately I no longer have access to my teaching material.
1 reply 0 retweets 1 like -
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.