We made this argument here ourselves a few years ago: Cooper, R. P. & Guest, O. (2014). Implementations are not specifications: specification, replication and experimentation in computational cognitive modeling. Cognitive Systems Research. http://dx.doi.org/10.1016/j.cogsys.2013.05.001 …
-
-
Replying to @o_guest @Neuro_Skeptic
Indeed, the abstract seems to argue for the same point. Wish we knew before! (Probably our justification is different, though).
1 reply 0 retweets 4 likes -
Replying to @MilekPl @Neuro_Skeptic
Kind of you to say! You cited another paper I'm an author on BTW. So yeah, what's your justification?
1 reply 0 retweets 3 likes -
Replying to @o_guest @Neuro_Skeptic
We argue that modeling papers should satisfy a specific norm: be complete wrt what it takes to evaluate the validity of a model. Of course, complete in the context of an ongoing distributed discussion, not self-contained in an absolute sense.
1 reply 1 retweet 5 likes -
Replying to @MilekPl @Neuro_Skeptic
I think that's the same as our point. And with respect to what
@Neuro_Skeptic tweeted you can easily argue: IN CERTAIN CASES releasing code can impair research as it can be used a crutch to avoid understanding the model.1 reply 2 retweets 7 likes -
If I am understanding you correctly what you call complete we call a "spec".
1 reply 0 retweets 2 likes -
So we can reduce the question in the OP to: "Could sharing [...] code harm reproducibility?" and answer with "Very much so, in certain specific cases which have and do arise quite frequently, sadly."
1 reply 3 retweets 6 likes -
But to be clear that does not mean we should not release code. It means we should but should warn scientists (as our two papers do) of what happens when we are not careful and just "mash buttons" on models/code with no deeper understanding.
1 reply 4 retweets 10 likes -
From p.43 "if a model is to be made freely available then it must come with a clear statement of the relation between the model and the theory behind the model. [I]t should not be necessary to scrutinize code in order to understand the model’s assumptions."http://dx.doi.org/10.1016/j.cogsys.2013.05.001 …
2 replies 4 retweets 13 likes -
Awww...why not...scrutinized code is such fun
1 reply 0 retweets 1 like
Keep reading to find out!
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.