Could anyone point me towards a resource on best practices for registering predictive models prior to testing on independent validation sets? Or for pre-registering work on existing (but not yet accessed) datasets more generally? (@kirstie_j @o_guest?)
-
-
Replying to @AgnesNorbury @kirstie_j
Computational models? Never seen any coherent position on this. I'm not convinced pre-registration and Computational modelling are compatible per se more like parallels.
@BArslan_CogSci@bradpwyble@IrisVanRooij3 replies 1 retweet 3 likes -
Replying to @o_guest @AgnesNorbury and
I suppose one could upload a copy of a model to a pre-reg site to indicate the parameters that will be tested before you compare simulations to data. In this sense, the model would need to be finalized prior to peeking at the simulations.
1 reply 0 retweets 0 likes -
Replying to @bradpwyble @AgnesNorbury and
That's impossible, surely... How do you debug without running your code?
1 reply 0 retweets 0 likes -
Replying to @o_guest @AgnesNorbury and
Well you might have already developed the model on data set X, and tuned its parameters. Then you apply that same version of the model to data set Y.
1 reply 0 retweets 2 likes -
Replying to @bradpwyble @AgnesNorbury and
Ah, that kind of modelling. Got you.
1 reply 0 retweets 0 likes -
Replying to @o_guest @bradpwyble and
This isn't universal. I keep thinking back to my modelling without data. But you're right in this case it's possible to do what you said. Just resurfacing from a migraine so I think I'm not at 100%.
1 reply 0 retweets 1 like
Have you developed a model already @AgnesNorbury?
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.