"In practice, for me to be able to replicate and verify your computational analysis and results, I will need to be able to see your source code, compile it myself, and potentially modify it."
Why? In this case to verify something you need to have the same output for a given input, this does not require any of the above (source code, compilation or modification), you just require a working version of the program/software. I like the open source model, but there are problems with this model along with the others. One major problem is that the authors are expected to support their software after others have gone in and modified it, which is unrealistic. I just think it is unrealistic to expect the source code just because someone wrote a paper (again, I like open source, but not everything is destined to be open source). cheers, tom Tom Peat Proteins Group Biomedical Program, CSIRO 343 Royal Parade Parkville, VIC, 3052 +613 9662 7304 +614 57 539 419 tom.p...@csiro.au ________________________________________ From: CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of Douglas Theobald [dtheob...@brandeis.edu] Sent: Wednesday, May 13, 2015 6:06 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] [RANT] Reject Papers describing non-open source software On May 12, 2015, at 3:19 PM, Robbie Joosten <robbie_joos...@hotmail.com> wrote: > > I strongly disagree with rejecting paper for any other reasons than > scientific ones. I agree, but … one of the foundations of science is independent replicability and verifiability. In practice, for me to be able to replicate and verify your computational analysis and results, I will need to be able to see your source code, compile it myself, and potentially modify it. These requirements in effect necessitate some sort of open source model, in the broadest sense of the term. To take one of your examples, the Ms-RSL license — I can’t effectively replicate and verify your results if I’m legally prohibited from compiling and modifying your source code, so the Ms-RSL is out. > A paper describing software should properly describe the > algorithms to ensure the reproducibility. *Should*. In practice, we all know (those programmers among us do, anyway) that descriptions of source code do not suffice. > The source should be available for > inspection to ensure the program does what was claimed, for all I care this > can be under the Ms-RSL license or just under good-old copyright. The > program should preferably be available free for academic users, but if the > paper is good you should be able to re-implement the tool if it is too > expensive or doesn't exactly do what you want so it isn't entirely > necessary. > Making the software open source (in an OSS sense) does not solve any > problems that a good description of the algorithms doesn't do well already. This is just wildly wrong. It’s basically impossible to ensure and verify that a “good" description of the algorithm actually corresponds to the source code without seeing, using, and modifying the source. To take an experimental analogy — my lab has endured several cases where we read a “good" published description of the subcloning and sequencing of some vector, only to find that the detailed published description is wrong when we are given the chance to analyze the vector ourselves. It happens all the time, and computer code is no different in this respect. > OSS does not guarantee long-term availability, a paper will like outlive the > software repository. OSS licenses (not the BSD license) can be so > restrictive that you end up having to re-implement the algorithms anyway. So > not having an OSS license should not be a reason to reject the paper about > the software. > > Cheers, > Robbie > >> -----Original Message----- >> From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of >> James Stroud >> Sent: Tuesday, May 12, 2015 20:40 >> To: CCP4BB@JISCMAIL.AC.UK >> Subject: Re: [ccp4bb] [RANT] Reject Papers describing non-open source >> software >> >> On May 12, 2015, at 12:29 PM, Roger Rowlett <rrowl...@colgate.edu> >> wrote: >> >>> Was the research publicly funded? If you receive funds from NSF, for > example, >> you are expected to share and "make widely available and usable" software >> and inventions created under a grant (section VI.D.4. of the Award and >> administration guide). I don't know how enforceable that clause is, > however. >> >> The funding shouldn't matter. I suggest that a publication that has the > purpose >> of describing non-open source software should be summarily rejected by >> referees. In other words, the power is in our hands, not the NSF's.
