Unfortunately, we have descended into a state in Science where once something is published we just accept it and move on. IN fact, the idea that "this has already been done" or that "once published its fact," is complete hogwash.
To suggest that a single publication is sufficient to warrant something a fact is baloney. The publication must be challenged by the scientific community, and confirmation should appear in publication. This does occur in some fields, but has been rather slacked off on in some areas of biology of late. For example, in 2003 I published a paper in ZOotaxa on the validity of the subspecies Acris crepitans blanchardi. My provided evidence that EITHER the subspecies was invalid and the species A. c. blanchardi was a single highly variable species, OR it was a multi-species complex. SO, right away a bunch of people started declaring the subspecies was canned and dumped it. Not more than 9 mo later, anotehr paper came out that suggested Acris crepitans and Acris blanchardi were two separate species. SO, immediately everyone was re-writing their papers with the species names. Several people are still working on the systematics of this frog, and I won't say more than the current taxonomic situation is probably wrong too. SOOOOO, in a matter of 1 year, we went from 1 species with two subspecies, to 1 species with no subspecies, to 2 species. However, none of these studies were complete. Together they tell us a lot. For the moment, two species is probably right, but people jump the gun to accept things JUST BECAUSE THEY ARE PUBLISHED> had we been patient, there would have been one change. So much of what happens is this way. We need to have patience before accepting published results as facts, give time for the results to be challenged so that we move in the right direction all at once rather than haphazardly bouncing back and forth. Who beards the cost? SCIENCE does. Its not the editor, the reviewer, the author, or the technician along or in part, its everyone who is involved, from the funder to the reader. If a reader is at odds with a study, they should be writing a response/rebuttal with demonstration of why they disagree. Not enough of this happens today. Nothing is sacred about publication other than the scientist took the time to complete his/her study. Its not done until its published! Peer and editor review only helps with eliminating the most critical mistakes. The better the peer reviewers are chosen, the more effective it is at that task. however, it was never intended to be a way to attack people under the shroud of anonymity, reject results for the benefit of our own internal conflicts of interest that we do not declare when asked, nor was it intended to make sure only the perfect study is published. In fact, there is no perfect study, researcher, reviewer, editor, or reader. So, it is our job, all of us, to contact authors with questions when we do not understand what we are reading, even as peer reviewers! Why not? heck, I just peer reviewed a paper and asked the author to send me a copy of one of their articles they cited! It really is not that big of a deal. However, some people are vindictive or even full of baloney and like to take out vengeance on others. This is probably why we really have anonymity. But, as I said, this is not a perfect system. things get published, errors are made, and mistakes are missed. Its called the human element, and it will always be there. That doesn't make the scientist a bad investigator, the study a bad study, or the results poorly deduced. It just means, if a human being did it, you can find something wrong with it. So we don't need to be buttheads and insult folks or whatever, just state what is wrong, why it is wrong and if you know how to fix it. The biggest problem is when you have a paper and you get responses like, "I don't know anything about this topic, but I'm going to criticize things I don't understand and make a fool out of myself to the author." This happens all the time. IF you are not qualified to review it, don't. Or, if you are qualified to review certain aspects but not others, then restrict your comments to the things you actually have a clue about. For example, my favorite peer review was my submission to a certain not-for-profit open-access journal published by a certain well known organization. I sent a paper to its One journal and got the review, signed by a Physicist who had no background in anything the paper was about. When a reviewer reviews a paper they have no expertise, heck no background in, the article has not been peer reviewed. It has been read by someone, possibly in an editorial capacity. As long ans we are sucker bait for every published paper, and peer review manuscripts we are completely unqualified to peer review, and editors allow this to happen, and readers allow papers to go without question, things will not work right. As long as we question everything, it will. On Sat, May 3, 2014 at 3:33 PM, David Schneider <[email protected]> wrote: > Dear ecologers, > > The question posed in the quote on fact-checking > from David Duffy is: > Who should undertake the cost of fact checking? > The quote gives 3 answers: warranty by the author, make the > data available, in-house by the publisher. > > In my experience, as reviewer and in an editorial capacity, > one can expect a reviewer to check the facts (results) against > interpretation. But it is often difficult for a reviewer to > fact check the results, simply because the data are of necessity > presented in summary fashion. Here are two examples. > If I see a confidence limit that includes zero, > where the data are counts, I can be fairly sure as a fact > checker that the wrong error distribution was used to > compute the ci. If I see a standard error or standard > deviation on count data, such that the variance is not at > least twice the mean, then I am suspicious, but I can't > check it, in the absence of the data. > > In my view the advent of online data archives are an important > step. I would also suggest that journals need to do more > than ask the author to warrant that they checked the facts. > I would suggest that a journal ask an author how they checked > the facts. > > That's my view. What is your take on who should undertake > the cost of fact checking? > > Best, > David S. > > > Quoting Malcolm McCallum <[email protected]>: > >> It is REALLY easy to screw up a figure, table or number set in a text >> if you have no one to review it before submission. IF you are a peer >> reviewer, this is one of the things you probably should be looking at. >> Do the numbers make sense? >> Peer review isn't there just to screen out garbage, its also there to >> assist authors. This is especially the case when an editor selects a >> reviewer specifically because of their exprtise in a particular area. >> I recall once as an editor that I sent a paper that involved some >> fancy modeling to a mathematical modeler to review the math. It was >> outside of what I did. She said he didn't know anything about the >> biology, and I told her that was easily covered by the other two >> reviewers, I just wanted to make sure the math was not tom-foolery. >> More of this needs to happen in peer review. I see a lot of papers >> that misuse different techniques. >> >> For example, I recall a paper published in one big ecology journal in >> which they used baysian statistics, and misinterpreted the sets. They >> said something had an effect, when the graph and stats clearly >> indicated there was no effect!!! So, the paper ended up widely >> covered in the news and people assumed it was what it said, when what >> it spent 4-5 pages discussing was complete rubbish. I've also seen >> interval analysis used where fuzzy sets should be used, and the misuse >> and over-use of monte carlo analysis is just over the top. >> >> Monte Carlo is only supposed to be used when you have a very great >> understanding of the system and very few assumptions and hopefully not >> a lot of unpredictable influences. This is actually not all that >> common in ecology and environmental work. yet, Monte Carlo is used >> and abused by simply "Assuming" things are that might not be. When >> you do this with MC you can get VERY wrong answers and there is >> virtually no way to check it. Fuzzy approaches are much more rubust >> in this regard as is interval analysis. But, you hardly see anyone who >> knows how to use these things, or people are caught 20-30 years >> out-of-date thinking they are controversial. >> >> The ideal way to do things is to use fuzzy sets to isolate your data >> sets to be used in monte carlo. That way, you reduce the odds of >> going completely off tangent. However, no one seems to do this either. >> it is pretty amazing because outside of ecology, the alternate >> methods are widely applied to many different situtaitons. heck, they >> even have fuzzy monte carlo and fuzzy neural networks now. But, that >> is an entire different topic. >> >> The point is, I think it is very reasonable for an editor to select a >> peer reviewer form an outside field to check up on methods and >> techiques that are outside of his/her expertise, especially if these >> are highly technical and particulary novel. A biologist is not always >> the best reviewer for some biology papers in such cases. >> >> On Wed, Apr 30, 2014 at 1:11 PM, David Duffy <[email protected]> wrote: >> > "To address this, the publishers of clinical journals must do more to >> > ensure that someone takes responsibility for the fact-checking. That could >> > involve asking authors to guarantee that they have checked figures, >> tables, >> > text and abstracts for internal consistency. Publishers could require >> > authors to make available suitably anonymized data on each patient as >> > metadata to the study, so that readers can trace the source of any >> > discrepancy that might creep through. Or the publishers could reach into >> > their pockets and provide more in-house resources to perform the necessary >> > checking. What is not acceptable is for the situation to continue as it >> is, >> > with responsibilities undefined and inexact publishing distorting clinical >> > messages." >> > >> > >> http://www.nature.com/news/false-positives-1.15119?WT.ec_id=NATURE-20140501 >> > >> > David Duffy >> > >> > Pacific Cooperative Studies Unit >> > Botany >> > University of Hawaii >> > 3190 Maile Way >> > Honolulu Hawaii 96822 USA >> > 1-808-956-8218 >> >> >> >> -- >> Malcolm L. McCallum, PHD, REP >> Department of Environmental Studies >> University of Illinois at Springfield >> >> Managing Editor, >> Herpetological Conservation and Biology >> >> “Nothing is more priceless and worthy of preservation than the rich >> array of animal life with which our country has been blessed. It is a >> many-faceted treasure, of value to scholars, scientists, and nature >> lovers alike, and it forms a vital part of the heritage we all share >> as Americans.” >> -President Richard Nixon upon signing the Endangered Species Act of >> 1973 into law. >> >> "Peer pressure is designed to contain anyone with a sense of drive" - >> Allan Nation >> >> 1880's: "There's lots of good fish in the sea" W.S. Gilbert >> 1990's: Many fish stocks depleted due to overfishing, habitat loss, >> and pollution. >> 2000: Marine reserves, ecosystem restoration, and pollution reduction >> MAY help restore populations. >> 2022: Soylent Green is People! >> >> The Seven Blunders of the World (Mohandas Gandhi) >> Wealth w/o work >> Pleasure w/o conscience >> Knowledge w/o character >> Commerce w/o morality >> Science w/o humanity >> Worship w/o sacrifice >> Politics w/o principle >> >> Confidentiality Notice: This e-mail message, including any >> attachments, is for the sole use of the intended recipient(s) and may >> contain confidential and privileged information. Any unauthorized >> review, use, disclosure or distribution is prohibited. If you are not >> the intended recipient, please contact the sender by reply e-mail and >> destroy all copies of the original message. >> -- Malcolm L. McCallum, PHD, REP Department of Environmental Studies University of Illinois at Springfield Managing Editor, Herpetological Conservation and Biology “Nothing is more priceless and worthy of preservation than the rich array of animal life with which our country has been blessed. It is a many-faceted treasure, of value to scholars, scientists, and nature lovers alike, and it forms a vital part of the heritage we all share as Americans.” -President Richard Nixon upon signing the Endangered Species Act of 1973 into law. "Peer pressure is designed to contain anyone with a sense of drive" - Allan Nation 1880's: "There's lots of good fish in the sea" W.S. Gilbert 1990's: Many fish stocks depleted due to overfishing, habitat loss, and pollution. 2000: Marine reserves, ecosystem restoration, and pollution reduction MAY help restore populations. 2022: Soylent Green is People! The Seven Blunders of the World (Mohandas Gandhi) Wealth w/o work Pleasure w/o conscience Knowledge w/o character Commerce w/o morality Science w/o humanity Worship w/o sacrifice Politics w/o principle Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
