This reflects a continuing obsession with seeking “truth”. In your example we already have two rounds of learning: we learn something when the model misses the data, and also learn from how we have to modify the model to handle those data
-
-
Replying to @psy_farrell @bradpwyble and
Agreed. It’s about extracting information such that we learn more & more over time. We may aim for truth in the long, but cannot in anyway guarantuee it in the short run. Moreover, enforcing ‘truth’ in the short run IMO limits us to uncovering not more than simple/shallow truths.
1 reply 0 retweets 8 likes -
Replying to @IrisVanRooij @psy_farrell and
Yep, that's it exactly. Moreover, the idea that I could "verify" a complex model by running an experiment or two is incredible. There's no way to tick a box that says the model is right.
1 reply 0 retweets 5 likes -
Replying to @bradpwyble @IrisVanRooij and
But you are supposed to be able to disconfirm aspects of a model with experiment, right? I mean at each round one further possibility is killed off. At some point your reasonably close to “truth”, no?
1 reply 0 retweets 1 like -
Replying to @Alex_Danvers @bradpwyble and
I guess it just feels odd to me that folks would be frustrated with reviewers for “being obsessed with truth”
1 reply 0 retweets 1 like -
Replying to @Alex_Danvers @IrisVanRooij and
It's fine to pursue truth, just don't expect that you'll be able to reach it in a categorical sense (ever, much less in a single paper). Yes, you can disconfirm parts of a model, but they were asking for validation of the entire thing.
1 reply 0 retweets 2 likes -
Replying to @bradpwyble @Alex_Danvers and
This. One caveat though is that it can be appropriate to ask for more work if a new assumption seems arbitrary, or if authors are arguing for model X, rather than “model X with this assumption that makes things work but we aren’t sure about yet and will need further testing”
1 reply 0 retweets 2 likes -
Replying to @psy_farrell @Alex_Danvers and
Yes, good caveats. Also, don't expect the authors to play out 2+ cycles of develop/test within a single paper. It's creatively inefficient to try to squeeze that much work into a single publication time frame. Ideas need time to settle.
1 reply 0 retweets 1 like -
Replying to @bradpwyble @psy_farrell and
In response to your question
@Alex_Danvers: Psychologists tend to overestimate how quickly we can converge on even “approximate truth”. There exist mathematical proofs that this is generally impossible, even with perfect knowledge: https://onlinelibrary.wiley.com/doi/full/10.1111/j.1551-6709.2011.01182.x …1 reply 0 retweets 3 likes -
Replying to @IrisVanRooij @bradpwyble and
Iris van Rooij Retweeted Brad Wyble
A reform that forces psych scientists to achieve the unachievable is creating more problems than it solves, such as kill the necessary creativity of the research process?(as @amyhcriss
@VandekerckhoveJ &@bradpwyble also pointed out in parallel threads: https://twitter.com/bradpwyble/status/1086832738689302538?s=21 …)Iris van Rooij added,
Brad Wyble @bradpwyblere: pre-registration and model building. This is exactly the issue. Model building is incredibly difficult as it is, to do so according to guidelines you created for yourself before you even saw the data is both pointless, and harmful to the creative process. https://twitter.com/VandekerckhoveJ/status/1086020978189074432 …1 reply 0 retweets 3 likes
Olivia Guest | Ολίβια Γκεστ Retweeted Olivia Guest | Ολίβια Γκεστ
This thread too:https://twitter.com/o_guest/status/1085514960325799936 …
Olivia Guest | Ολίβια Γκεστ added,
This is a super simplistic description of science that tends towards confusing people. Truth isn't a scientific concept anyway... https://twitter.com/dstephenlindsay/status/1085455647200690176 …-
-
Replying to @o_guest @bradpwyble and
Thanks! I saw that connection and had wanted to cross post my link to
@richarddmorey’s commemt but given the lock, I cannot copy my tweets atm. So hi@richarddmorey Perhaps the above exchange is of interest to you too
1 reply 0 retweets 3 likes - 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.