Hi Tomasz, this is what actually happend. Please refer to https://wikimania2016.wikimedia.org/wiki/Critical_issues_presentations#Evaluation
: In order to achieve the greatest possible neutrality, the submissions will : be evaluated online using a *double-blind peer-review process*. This : means that two evaluators will review the submission without knowing : the name of its author. *If there are strong divergences among the two : evaluations, at least one other review will be made.* Kind regards, Sebastian Am 04.02.2016 um 11:31 schrieb Tomasz Ganicz: > Also - normally in Academia - if there are two strongly opposite > reviews (one very positive, one very negative) a typical procedure is > to send the submission to the third one. > > 2016-02-04 10:54 GMT+01:00 Tomasz Ganicz <[email protected] > <mailto:[email protected]>>: > > Well I think that double blind peer review hardly make sense here, > from the reviewer POV, as we are in fact small community and it > is easy to guess who was a submiter in most cases. For example - > if there is a submission about project X in country Y, which was > funded by WMF grant - it is very easy to find out who was grantee > and it is rather obvious that that person is a submitter :-) > > Also judging from the several reviewers comments which I saw > already - they did not follow the very vague criteria which was > posted here: > > > https://wikimania2016.wikimedia.org/wiki/Critical_issues_presentations#Evaluation > > Normally - at least in Academia - reviewers are forced directly > (by the review form) to address their opinion in relation to the > criteria. The criteria were: > > " > > 1. problems and possible solutions in a specific field > 2. proposals for others to replicate > 3. issues (positive or negative) which have emerged from projects > 4. issues you want to raise which you feel have not been > discussed yet > 5. issues which are at the centre of an online debate that you > would like to address offline > > " > > 1-4 are IMHO relatively easy to evaluate - I would expect from the > reviews to answer yes or no to them. 5 is a bit tricky as it > depends strongly of what the reviewer think is "at the centre" - > but I would expect that they at least explain in few words here > what they think is "at the centre" or not :-) > > > > 2016-02-04 0:22 GMT+01:00 Dariusz Jemielniak <[email protected] > <mailto:[email protected]>>: > > hi, > > I have some comments as a person from Academia (and not > involved in Wikimania process in any way): > > 1. Short reviews are definitely not helping in addressing the > frustration of rejection, yet are quite common in academic > peer reviewing, especially for conferences. > > 2. Double blind peer review (not knowing who is reviewed, and > not knowing who reviews) is a standard in Academia, although > some perceive it as contributing to lack of responsibility > (especially true in competitive journal submissions). > > 3. Two reviewers per submission is absolutely on par with the > conference standards I'm used to. Sometimes there are three, > but two is absolutely acceptable (although a third opinion > should be used if the two disagree too much). > > 4. It could be useful to sensitize the reviewers that the main > purpose of the review is to help the author to do better next > time. > > 5. All this is volunteer work. We should be, generally, > grateful to reviewers (but in the same time grateful to the > contributors, too). > > best, > > dj > > On Wed, Feb 3, 2016 at 5:26 PM, Maarten Dammers > <[email protected] <mailto:[email protected]>> wrote: > > What kind of ridiculous process is this? This is all I got: > > =============== > > ----------------------- REVIEW 1 --------------------- > PAPER: 194 > TITLE: GLAM+Wikidata > AUTHORS: Sandra Fauconnier and Maarten Dammers > > OVERALL EVALUATION: 8 (Very good) > > ----------- REVIEW ----------- > 8 > > > ----------------------- REVIEW 2 --------------------- > PAPER: 194 > TITLE: GLAM+Wikidata > AUTHORS: Sandra Fauconnier and Maarten Dammers > > OVERALL EVALUATION: 6 (Rather interesting) > > ----------- REVIEW ----------- > 6 > > ============== > > So only two people reviewed this? Who are these people? Why is > this secret? Last year I had 5 people reviewing my submission [1]. > > Maarten > > [1] https://wikimania2015.wikimedia.org/wiki/Submission_review/5 > > > Op 3-2-2016 om 23:15 schreef Andy Mabbett: >> >> I've just received feedback on one of my pitches saying, >> in part: >> >> "Bad boy Andy! This is supposed to be an anonymous review >> process, so starting your abstract with your own name, is >> not entirely fair." >> >> -- >> Andy Mabbett >> @pigsonthewing >> http://pigsonthewing.org.uk >> >> >> >> _______________________________________________ >> Wikimania-l mailing list >> [email protected] >> <mailto:[email protected]> >> https://lists.wikimedia.org/mailman/listinfo/wikimania-l > > > _______________________________________________ > Wikimania-l mailing list > [email protected] > <mailto:[email protected]> > https://lists.wikimedia.org/mailman/listinfo/wikimania-l > > > > > -- > > __________________________ > prof. dr hab. Dariusz Jemielniak > kierownik katedry Zarządzania Międzynarodowego > i grupy badawczej NeRDS > Akademia Leona Koźmińskiego > http://n <http://www.crow.alk.edu.pl/>wrds.kozminski.edu.pl > <http://wrds.kozminski.edu.pl> > > członek Akademii Młodych Uczonych Polskiej Akademii Nauk > członek Komitetu Polityki Naukowej MNiSW > > Wyszła pierwsza na świecie etnografia Wikipedii "Common > Knowledge? An Ethnography of Wikipedia" (2014, Stanford > University Press) mojego > autorstwa http://www.sup.org/book.cgi?id=24010 > > Recenzje > Forbes: http://www.forbes.com/fdc/welcome_mjx.shtml > Pacific > Standard: > http://www.psmag.com/navigation/books-and-culture/killed-wikipedia-93777/ > Motherboard: > http://motherboard.vice.com/read/an-ethnography-of-wikipedia > The > Wikipedian: > http://thewikipedian.net/2014/10/10/dariusz-jemielniak-common-knowledge > > _______________________________________________ > Wikimania-l mailing list > [email protected] > <mailto:[email protected]> > https://lists.wikimedia.org/mailman/listinfo/wikimania-l > > > > > -- > Tomek "Polimerek" Ganicz > http://pl.wikimedia.org/wiki/User:Polimerek > http://www.ganicz.pl/poli/ > > > > > -- > Tomek "Polimerek" Ganicz > http://pl.wikimedia.org/wiki/User:Polimerek > http://www.ganicz.pl/poli/ > > > > _______________________________________________ > Wikimania-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikimania-l -- Sebastian Wallroth mobile +4917615154002 http://about.me/real68er PGP Key https://de.wikipedia.org/wiki/Benutzer:Sebastian_Wallroth/PGP _______________________________________________ Wikimania-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikimania-l
