If replication is not an issue, then neither is preregistration. I imagine in pure modeling studies, this is the case. I assume peer review focuses on modeling constraints, validity and parsimony, rather than reliability of results. Open code is important for robustness though.
-
-
Replying to @the_Sage_BB @djnavarro
Olivia Guest | Ολίβια Γκεστ Retweeted Olivia Guest | Ολίβια Γκεστ
It's complicated. Open code & open source doesn't actually solve (all) the scientific problems with respect to modelling, sadly. More here if you are interested:https://twitter.com/o_guest/status/1058725725531004930 …
Olivia Guest | Ολίβια Γκεστ added,
1 reply 1 retweet 1 like -
Also check out
@ReScienceEds, a journal dedicated to publishing replications in modelling.1 reply 1 retweet 1 like -
Replying to @o_guest @the_Sage_BB and
Olivia Guest | Ολίβια Γκεστ Retweeted Olivia Guest | Ολίβια Γκεστ
To be clear, my high-level point is that just because replication in modelling is important doesn't mean prepreg can solve it by definition. Literally, the way we do science is different. See
@zerdeve's work, relevant quotes here:https://twitter.com/o_guest/status/1062771691552673793 …Olivia Guest | Ολίβια Γκεστ added,
Olivia Guest | Ολίβια Γκεστ @o_guest
TFW somebody really gets how you do science (AKA is also a modeller).
"We believe that a hypothesis-centric approach is too impoverished to provide the necessary resources for a formal theory of such open practices." @zerdeve https://arxiv.org/abs/1811.04525Show this thread1 reply 1 retweet 3 likes -
Replying to @o_guest @the_Sage_BB and
Generally speaking, isn't cross-validation to modeling what pre-registration can be for empirical work?
1 reply 0 retweets 1 like -
Replying to @Research_Tim @the_Sage_BB and
I think I know what you are trying to get at, so sort of: yes! Crossval evaluate cog/ML models. If you are interested, there is lots to read online. Crossval ofc doesn't really evaluate things on the theory level of analysis.
2 replies 0 retweets 1 like -
Replying to @o_guest @Research_Tim and
What aspect of crossval though do you see as being similar to prereg? Do you think they both serve the same role, of evaluating some aspect of an experiment or model?
1 reply 0 retweets 0 likes
For that, I agree. And I have said so before quite a few times on Twitter. For other potential parallels I'd have to hear more from you.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.