(ok, I'm not completely confident about perfect information statement but feels like a good direction--it's drawing from why imperfect information games are so difficult,the connection I'm missing is how this is true outside a game setting. I think it is but cant find a path rn)
-
-
For example, I tried to replicate two models during my PhD and one doesn't replicate. The other has a mistake in a graph which is very cute as I think it was a bug in their original code and they state "we expected X but we got this graph instead", the correct graph shows X.
-
And yes,
@ReScienceEds try very hard to tackle this issue by publishing only replications of computational models (not just cognitive ones though). -
I have discussed this issue in a few different places. Here with
@NPRougier Guest, O. & Rougier, N. P. (2016). Dialogue: What is Computational Reproducibility?. IEEE CIS Newsletter on Cognitive and Developmental Systems. 13 (2). http://oliviaguest.com/doc/guest_rougier_2016.pdf … -
This blog post that
@chbergma invited me to write (thank you!): http://bootphon.blogspot.co.uk/2015/10/replication-in-computational-cognitive.html … -
This paper: Implementations are not specifications: Specification, replication and experimentation in computational cognitive modeling https://doi.org/10.1016/j.cogsys.2013.05.001 …
-
So there are more than a single issue here but I have been discussing this usually on my own and often without any support from others since the academic year of 2009/2010.
-
:( It's sad people don't see the importance of the points you've brought up. It's fascinating to see another field facing nearly the same issues as machine learning research and suggesting similar trajectories through them.
- 3 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.