Dear James, Indeed, others are thinking about openness in review of academic documents. In particular I know of a group who address particularly the idea that reviews, in this case of scientific manuscripts, might be signed by the reviewer. The group Biophysics Colab (https://sciety.org/groups/biophysics-colab/about), which is a run principally by and for researchers and is supported by eLife, are working to accomplish just this. They work to assure frank and open discussion that leads to improvement of the paper. They also seek to engage knowledgeable reviewers with diverse backgrounds who may ordinarily be overlooked by traditional publishers.
I poked around on their lists for a while and found this one, from a year ago; it's impressive: https://sciety.org/articles/activity/10.1101/2021.07.05.451181 One sees a substantial discussion back and forth with three reviewers, and then a final version of the paper. The "Read the full article" link at the top of that page leads one to the BioRxiv page of that preprint, https://doi.org/10.1101/2021.07.05.451181, where the title box reveals that it was finally published in Nature Communications. Several published versions of papers acknowledge suggestions from Biophysics Colab reviewers by name: https://www.nature.com/articles/s41467-022-30300-z#Ack1, and https://www.nature.com/articles/s41467-022-30406-4#Ack1. I've learned that the Biophysics Colab procedure is to assign one of their volunteers as a curator to papers they discover in BioRxiv, whose job is to interest the author in such a review, and then to recruit reviewers. Their organization is in a flexible early stage and they are eager for feedback about how to create the best service for the research community. Anyone interested in this might reach them at enquir...@sciencecolab.org. Perhaps this effort will follow the trajectory implied in this thread towards truly open review. Yours, Bob ================================================= Robert M. Sweet E-Dress: sw...@bnl.gov<mailto:sw...@bnl.gov> Scientific Advisor, CBMS: The Center for BioMolecular Structure at NSLS-II Photon Sciences, Brookhaven Nat'l Lab. Upton, NY 11973 U.S.A. Phone: 631 338 7302 (Mobile) ================================================= ________________________________________ From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of John R Helliwell <jrhelliw...@gmail.com> Sent: Thursday, June 23, 2022 9:18 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] open review? Dear James, This is an interesting question you have posed. The trend to open peer review reports in articles that we see more often today got a major kick off by the ASAPBio Workshop some years back https://asapbio.org/peer-review<https://urldefense.com/v3/__https://asapbio.org/peer-review__;!!P4SdNyxKAPE!GXPiBrQYlabJVYJN9TZjK9GIz8Onzw8rtQ40Sh6W4iSg1sXHMGNHqKzTKdaxFuv3AT_h4AbVyTedZgs5DrU$> . The workshop questions to participants included “would you sign your report?”. Earlier career researchers were not in favour due to fear of retribution by later career researchers whose submitted articles they would have to criticise. Your question however concerns “open review of research grant proposals?”. Different approaches have been tried, most famously perhaps the allocate funds randomly via a lottery. In trying to locate the weblink to that I found a more comprehensive overview of all sorts of methods and applied to a wide variety of types of grant proposals https://www.embopress.org/doi/full/10.15252/embr.201949472<https://urldefense.com/v3/__https://www.embopress.org/doi/full/10.15252/embr.201949472__;!!P4SdNyxKAPE!GXPiBrQYlabJVYJN9TZjK9GIz8Onzw8rtQ40Sh6W4iSg1sXHMGNHqKzTKdaxFuv3AT_h4AbVyTedsR8Y-uw$> Perhaps most interestingly higher risk ie adventurous proposals such as done by The Wellcome Trust some years back did not involve peer review at all as reviewers couldn’t be trusted to take anything other than a highly critical stance. A project manager decided on which got funded. On reading your message I have also twitter messaged ASAPBio for any further info of possible previous workshops that may have addressed open peer review of research grant proposals, mentioning your name as the originator of the question. If not, a workshop could be convened. (:-) I once was on an interviewing panel for a UK funder Advanced Fellowship. One applicant opened their interview with the statement “I am very sorry to say that when I joined a laboratory in country x, the laboratory Head asked to see my Advanced Fellowship proposal. He put all the staff in the lab to do the investigations. So, unfortunately their are no new ideas remaining.” Similarly the developing world’s researchers are very worried about the Global North sucking up their data, of all kinds, and doing the analyses that they would like to do themselves but more slowly. Open science will need very careful implementation. UNESCO are currently giving this a very serious go: https://www.unesco.org/en/natural-sciences/open-science<https://urldefense.com/v3/__https://www.unesco.org/en/natural-sciences/open-science__;!!P4SdNyxKAPE!GXPiBrQYlabJVYJN9TZjK9GIz8Onzw8rtQ40Sh6W4iSg1sXHMGNHqKzTKdaxFuv3AT_h4AbVyTedW5zi5cA$> . Greetings, John Emeritus Professor John R Helliwell DSc IUCr Representative to CODATA, IUCr Representative to UNESCO’s Open Science discussions. On 23 Jun 2022, at 02:08, James Holton <jmhol...@lbl.gov> wrote: Greetings all, I'd like to ask a question that I expect might generate some spirited discussion. We have seen recently a groundswell of support for openness and transparency in peer review. Not only are pre-prints popular, but we are also seeing reviewer comments getting published along with the papers themselves. Sometimes even signed by the reviewers, who would have traditionally remained anonymous. My question is: why don't we also do this for grant proposals? I know this is not the norm. However, after thinking about it, why wouldn't we want the process of how funding is awarded in science to be at least as transparent as the process of publishing the results? Not that the current process isn't transparent, but it could be more so. What if applications, and their reviewer comments, were made public? Perhaps after an embargo period? There could be great benefits here. New investigators especially, would have a much clearer picture of format, audience, context and convention. I expect unsuccessful applications might be even more valuable than successful ones. And yet, in reality, those old proposals and especially the comments almost never see the light of day. Monumental amounts of work goes into them, on both sides, but then get tucked away into the darkest corners of our hard drives. So, 2nd question is: would you do it? Would you upload your application into the public domain for all to see? What about the reviewer comments? If not, why not? Afraid people will steal your ideas? Well, once something is public, its pretty clear who got the idea first. 3rd question: what if the service were semi-private? and you got to get comments on your proposal before submitting it to your funding agency? Would that be helpful? What if in exchange for that service you had to review 2-3 other applications? Would that be worth it? Or, perhaps, I'm being far too naiive about all this. For all I know there are some rules against doing this I'm not aware of. Either way, I'm interested in what this community thinks. Please share your views! On- or off-list is fine. -James Holton MAD Scientist ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1<https://urldefense.com/v3/__https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1__;!!P4SdNyxKAPE!GXPiBrQYlabJVYJN9TZjK9GIz8Onzw8rtQ40Sh6W4iSg1sXHMGNHqKzTKdaxFuv3AT_h4AbVyTedGeWh8IY$> This message was issued to members of www.jiscmail.ac.uk/CCP4BB<https://urldefense.com/v3/__http://www.jiscmail.ac.uk/CCP4BB__;!!P4SdNyxKAPE!GXPiBrQYlabJVYJN9TZjK9GIz8Onzw8rtQ40Sh6W4iSg1sXHMGNHqKzTKdaxFuv3AT_h4AbVyTed-p6xJKg$>, a mailing list hosted by www.jiscmail.ac.uk<https://urldefense.com/v3/__http://www.jiscmail.ac.uk__;!!P4SdNyxKAPE!GXPiBrQYlabJVYJN9TZjK9GIz8Onzw8rtQ40Sh6W4iSg1sXHMGNHqKzTKdaxFuv3AT_h4AbVyTedEhdCJUA$>, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/<https://urldefense.com/v3/__https://www.jiscmail.ac.uk/policyandsecurity/__;!!P4SdNyxKAPE!GXPiBrQYlabJVYJN9TZjK9GIz8Onzw8rtQ40Sh6W4iSg1sXHMGNHqKzTKdaxFuv3AT_h4AbVyTedfURepeY$> ________________________________ To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1<https://urldefense.com/v3/__https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1__;!!P4SdNyxKAPE!GXPiBrQYlabJVYJN9TZjK9GIz8Onzw8rtQ40Sh6W4iSg1sXHMGNHqKzTKdaxFuv3AT_h4AbVyTedGeWh8IY$> ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
