Could anyone point me towards a resource on best practices for registering predictive models prior to testing on independent validation sets? Or for pre-registering work on existing (but not yet accessed) datasets more generally? (@kirstie_j @o_guest?)
-
-
Ah sorry, my fault for assuming you would probably know about anything model-related :-). Interesting to think about the computational case too though. Maybe this is covered under standard pre-reg under some kind of time-lock system.
-
Ah, no worries! If you're asking me, I don't really think prereg makes sense for my work and I don't think computational modelling has the same type of reproducibility issues (it has others). But I can totally see why it's needed for (some) empirical work like this case.
-
IMO, Imodelling has enormous reproducibility issues in terms of over fitting to datasets. Models that work well on imagenet may not generalize to other image sets. If pre-regging reduces over fitting, then it may help.
-
I don't disagree with the first part and by all means use any method. I don't think prereg can address this, but please try, of course! Correctly using the testing, training, and validation sets might help, I think. But even then that might not either. It's a big issue.
-
Just to be clear... Here you're using modelling to men machine learning, right? They don't use modelling that way and neither do I, so just making sure.
-
Yes, ML. Modelling is one of those terms with many meanings.
-
Yes, but I wasn't talking about ML, just so we're clear, in the side thread but about the modelling I do... Which is why I mistakenly assumed I was tagged. Then I realised you mean ML and mentioned what I did in my replies to you.
-
What I believe about my kind of modelling is slightly different to what I believe about ML modelling but there are overlaps which I've discussed at great length.
- 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
