In a previous post I explained why Jens and me, as chairs of the ICSME industry track, decided to use double blind reviewing. In order to understand the impact of this, we made a survey and asked authors and reviewers for input. From the authors, 16 responded and from the reviewers we got 13 answers.

We asked participants how they felt about the results of double blind reviewing. Authors overwhelmingly agreed that double blind makes te process more fair:

Interestingly, reviewers mostly thought it made no difference, but still 46% thought it makes reviews more fair.

We worried a little bit that DB would make reviewing harder, which some reviewers agreed with (23%) but most thought it made no difference.

Here, we also asked an open question, and we did get some interesting answers! Some reviewers, like me, admitted that is would be hard to judge neutrally: “I like to believe that I can judge a paper without bias. But really, without DBR it would probably be harder for me to e.g. criticise a weak paper written by a person whose other works I admire.” and “the unseen pressure of reviewing a high profile author’s paper or an unknown author’s paper vanishes.”

Another reviewer thought DBR would produce better reviews: “not knowing who the authors were made it easier to treat each paper the same and to produce more consistent reviews.”

Authors seemed to feel the same way about the quality of the reviews: “It improves quality of reviews because reviewers must make a larger effort to understand the paper.”

All in all, both reviewers (69%) and authors (88%) agreed we should keep up DBR, and ICSME Industry 2017 will, and so will the research track!