Jennifer Trueblood: ideas for improving cognitive modeling: make models available, report models that do not work as well, detailed evaluation of models #iccmpsyched
-
-
Replying to @mvugt
I like the idea of having both a pre-registration (w/the understanding that things will go wrong) and a post-registration (to document what went wrong and was done that didn't work before you got it to work).
#iccmpsyched1 reply 2 retweets 5 likes -
As Trueblood points out, this is essentially what already has to be done with funded research: a proposal and a final report.
1 reply 0 retweets 0 likes -
Ideally, we'd all do this all the time, I agree. But I think most journals have a slight bias towards having you report what works rather than what did not. I can only read what someone publishes but not the report they write for their funding agency.
1 reply 0 retweets 0 likes -
I agree—just pointing out that this isn’t too much extra work.
1 reply 0 retweets 1 like -
I'm not sure that I agree. There's a substantial difference between the kind of reporting one does at the end of a grant and publishing papers about what doesn't work. The two are written for different audiences.
1 reply 0 retweets 0 likes -
Fair enough. I guess it depends how thorough one’s final reports are.
1 reply 0 retweets 0 likes -
Replying to @dajmeyer @bradpwyble and
How do you preregister computational modelling work?
2 replies 0 retweets 0 likes -
Trueblood made very clear that not all work can or should be pre-registered and that exploratory work is essential. She presented this mostly in the context of pitching existing models against each other and/or using computational models as measurement tools.
2 replies 0 retweets 3 likes -
In those cases, she said, it makes sense to pre-register how you'll compare models + what the evaluation criteria are. She also said that we should recognize that these plans will go wrong + documenting what went wrong + what needed to be changed to make it work is important.
1 reply 0 retweets 1 like
Ah, preregistering the comparison. I see. Thanks.
-
-
It's a good idea but presumes the models are comparable of course.
0 replies 0 retweets 1 likeThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.