That seems a sensible system, I can see how a 6 and a 7 were enough to reject one of my proposals, though I'd have appreciated a bit more feedback as to why.
But how strong did the divergence have to be if 5 and 8 didn't count as a "strong divergence"? WereSpielChequers > On 4 Feb 2016, at 11:55, Sebastian Wallroth <[email protected]> wrote: > > Hi Tomasz, > > this is what actually happend. Please refer to > https://wikimania2016.wikimedia.org/wiki/Critical_issues_presentations#Evaluation > > > : In order to achieve the greatest possible neutrality, the submissions > will > : be evaluated online using a *double-blind peer-review process*. This > : means that two evaluators will review the submission without knowing > : the name of its author. *If there are strong divergences among the two > : evaluations, at least one other review will be made.* > > Kind regards, > Sebastian > >> Am 04.02.2016 um 11:31 schrieb Tomasz Ganicz: >> Also - normally in Academia - if there are two strongly opposite >> reviews (one very positive, one very negative) a typical procedure is >> to send the submission to the third one. >> >> 2016-02-04 10:54 GMT+01:00 Tomasz Ganicz <[email protected] >> <mailto:[email protected]>>: >> >> Well I think that double blind peer review hardly make sense here, >> from the reviewer POV, as we are in fact small community and it >> is easy to guess who was a submiter in most cases. For example - >> if there is a submission about project X in country Y, which was >> funded by WMF grant - it is very easy to find out who was grantee >> and it is rather obvious that that person is a submitter :-) >> >> Also judging from the several reviewers comments which I saw >> already - they did not follow the very vague criteria which was >> posted here: >> >> >> https://wikimania2016.wikimedia.org/wiki/Critical_issues_presentations#Evaluation >> >> Normally - at least in Academia - reviewers are forced directly >> (by the review form) to address their opinion in relation to the >> criteria. The criteria were: >> >> " >> >> 1. problems and possible solutions in a specific field >> 2. proposals for others to replicate >> 3. issues (positive or negative) which have emerged from projects >> 4. issues you want to raise which you feel have not been >> discussed yet >> 5. issues which are at the centre of an online debate that you >> would like to address offline >> >> " >> >> 1-4 are IMHO relatively easy to evaluate - I would expect from the >> reviews to answer yes or no to them. 5 is a bit tricky as it >> depends strongly of what the reviewer think is "at the centre" - >> but I would expect that they at least explain in few words here >> what they think is "at the centre" or not :-) >> >> >> >> 2016-02-04 0:22 GMT+01:00 Dariusz Jemielniak <[email protected] >> <mailto:[email protected]>>: >> >> hi, >> >> I have some comments as a person from Academia (and not >> involved in Wikimania process in any way): >> >> 1. Short reviews are definitely not helping in addressing the >> frustration of rejection, yet are quite common in academic >> peer reviewing, especially for conferences. >> >> 2. Double blind peer review (not knowing who is reviewed, and >> not knowing who reviews) is a standard in Academia, although >> some perceive it as contributing to lack of responsibility >> (especially true in competitive journal submissions). >> >> 3. Two reviewers per submission is absolutely on par with the >> conference standards I'm used to. Sometimes there are three, >> but two is absolutely acceptable (although a third opinion >> should be used if the two disagree too much). >> >> 4. It could be useful to sensitize the reviewers that the main >> purpose of the review is to help the author to do better next >> time. >> >> 5. All this is volunteer work. We should be, generally, >> grateful to reviewers (but in the same time grateful to the >> contributors, too). >> >> best, >> >> dj >> >> On Wed, Feb 3, 2016 at 5:26 PM, Maarten Dammers >> <[email protected] <mailto:[email protected]>> wrote: >> >> What kind of ridiculous process is this? This is all I got: >> >> =============== >> >> ----------------------- REVIEW 1 --------------------- >> PAPER: 194 >> TITLE: GLAM+Wikidata >> AUTHORS: Sandra Fauconnier and Maarten Dammers >> >> OVERALL EVALUATION: 8 (Very good) >> >> ----------- REVIEW ----------- >> 8 >> >> >> ----------------------- REVIEW 2 --------------------- >> PAPER: 194 >> TITLE: GLAM+Wikidata >> AUTHORS: Sandra Fauconnier and Maarten Dammers >> >> OVERALL EVALUATION: 6 (Rather interesting) >> >> ----------- REVIEW ----------- >> 6 >> >> ============== >> >> So only two people reviewed this? Who are these people? Why is >> this secret? Last year I had 5 people reviewing my submission [1]. >> >> Maarten >> >> [1] https://wikimania2015.wikimedia.org/wiki/Submission_review/5 >> >> >> Op 3-2-2016 om 23:15 schreef Andy Mabbett: >>> >>> I've just received feedback on one of my pitches saying, >>> in part: >>> >>> "Bad boy Andy! This is supposed to be an anonymous review >>> process, so starting your abstract with your own name, is >>> not entirely fair." >>> >>> -- >>> Andy Mabbett >>> @pigsonthewing >>> http://pigsonthewing.org.uk >>> >>> >>> >>> _______________________________________________ >>> Wikimania-l mailing list >>> [email protected] >>> <mailto:[email protected]> >>> https://lists.wikimedia.org/mailman/listinfo/wikimania-l >> >> >> _______________________________________________ >> Wikimania-l mailing list >> [email protected] >> <mailto:[email protected]> >> https://lists.wikimedia.org/mailman/listinfo/wikimania-l >> >> >> >> >> -- >> >> __________________________ >> prof. dr hab. Dariusz Jemielniak >> kierownik katedry Zarządzania Międzynarodowego >> i grupy badawczej NeRDS >> Akademia Leona Koźmińskiego >> http://n <http://www.crow.alk.edu.pl/>wrds.kozminski.edu.pl >> <http://wrds.kozminski.edu.pl> >> >> członek Akademii Młodych Uczonych Polskiej Akademii Nauk >> członek Komitetu Polityki Naukowej MNiSW >> >> Wyszła pierwsza na świecie etnografia Wikipedii "Common >> Knowledge? An Ethnography of Wikipedia" (2014, Stanford >> University Press) mojego >> autorstwa http://www.sup.org/book.cgi?id=24010 >> >> Recenzje >> Forbes: http://www.forbes.com/fdc/welcome_mjx.shtml >> Pacific >> Standard: >> http://www.psmag.com/navigation/books-and-culture/killed-wikipedia-93777/ >> Motherboard: >> http://motherboard.vice.com/read/an-ethnography-of-wikipedia >> The >> Wikipedian: >> http://thewikipedian.net/2014/10/10/dariusz-jemielniak-common-knowledge >> >> _______________________________________________ >> Wikimania-l mailing list >> [email protected] >> <mailto:[email protected]> >> https://lists.wikimedia.org/mailman/listinfo/wikimania-l >> >> >> >> >> -- >> Tomek "Polimerek" Ganicz >> http://pl.wikimedia.org/wiki/User:Polimerek >> http://www.ganicz.pl/poli/ >> >> >> >> >> -- >> Tomek "Polimerek" Ganicz >> http://pl.wikimedia.org/wiki/User:Polimerek >> http://www.ganicz.pl/poli/ >> >> >> >> _______________________________________________ >> Wikimania-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/wikimania-l > > -- > Sebastian Wallroth > mobile +4917615154002 > http://about.me/real68er > PGP Key https://de.wikipedia.org/wiki/Benutzer:Sebastian_Wallroth/PGP > > > _______________________________________________ > Wikimania-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikimania-l _______________________________________________ Wikimania-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikimania-l
