"Reports of computational studies should remain selective and include all and only relevant bits of code." https://www.ncbi.nlm.nih.gov/pubmed/30377880?dopt=Abstract … Could sharing too much code harm reproducibility?pic.twitter.com/mAUF5pDjeT
You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
We argue that modeling papers should satisfy a specific norm: be complete wrt what it takes to evaluate the validity of a model. Of course, complete in the context of an ongoing distributed discussion, not self-contained in an absolute sense.
I think that's the same as our point. And with respect to what @Neuro_Skeptic tweeted you can easily argue: IN CERTAIN CASES releasing code can impair research as it can be used a crutch to avoid understanding the model.
If I am understanding you correctly what you call complete we call a "spec".
So we can reduce the question in the OP to: "Could sharing [...] code harm reproducibility?" and answer with "Very much so, in certain specific cases which have and do arise quite frequently, sadly."
But to be clear that does not mean we should not release code. It means we should but should warn scientists (as our two papers do) of what happens when we are not careful and just "mash buttons" on models/code with no deeper understanding.
From p.43 "if a model is to be made freely available then it must come with a clear statement of the relation between the model and the theory behind the model. [I]t should not be necessary to scrutinize code in order to understand the model’s assumptions."http://dx.doi.org/10.1016/j.cogsys.2013.05.001 …
I assume we agree on all this, right?
"Providing computationally naïve researchers with an implementation without also providing them with the means to discriminate between the consequences of theory-relevant or theory-irrelevant assumptions raises the possibility that such...
researchers will over-interpret the model’s behavior. This can work in two ways: being excessively impressed with some positive aspect of the model’s behavior that is not due to the theoretically critical elements of the model, or being unduly critical of some negative aspect...
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.