Could anyone point me towards a resource on best practices for registering predictive models prior to testing on independent validation sets? Or for pre-registering work on existing (but not yet accessed) datasets more generally? (@kirstie_j @o_guest?)
-
-
Replying to @AgnesNorbury @kirstie_j
Computational models? Never seen any coherent position on this. I'm not convinced pre-registration and Computational modelling are compatible per se more like parallels.
@BArslan_CogSci@bradpwyble@IrisVanRooij3 replies 1 retweet 3 likes -
Replying to @o_guest @kirstie_j and
In this case, just wondering if it would be possible to fit a predictive model (e.g. symptom change from baseline imaging and clinical data), then register that somewhere before testing on independent data. Issue is both datasets already exists so hard to prove date of access...
1 reply 0 retweets 1 like -
Replying to @AgnesNorbury @kirstie_j and
A statistical model? Like a multiple regression?
1 reply 0 retweets 0 likes -
Replying to @o_guest @kirstie_j and
Yes, sorry, in this case likely some kind of regression (but also could be e.g. some kind of classification algorithm).
1 reply 0 retweets 1 like -
Replying to @AgnesNorbury @kirstie_j and
Ah, so this is surely not really what I'm an expert in at all. FWIW this should be exactly what standard prereg is — although not an expert myself!
1 reply 0 retweets 0 likes -
Replying to @o_guest @AgnesNorbury and
That "surely" is superfluous/a mistake.
1 reply 0 retweets 1 like -
Replying to @o_guest @kirstie_j and
Ah sorry, my fault for assuming you would probably know about anything model-related :-). Interesting to think about the computational case too though. Maybe this is covered under standard pre-reg under some kind of time-lock system.
3 replies 0 retweets 1 like -
Replying to @AgnesNorbury @o_guest and
It's a great idea with classification/machine learning so as to enable an honest approach to model fitting.
1 reply 0 retweets 0 likes
What about test, validation, and training sets? What isn't covered by using then properly?
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.