Monday, October 25, 2004

give the newbies a break?

The review process for papers was an issue that came up in a town hall debate at the conference I am currently attending (User Interface Software and Technology [UIST]). In particular, this conference does not use a double blind process: reviewers have full knowledge of the authors of the papers they are reviewing. This is relatively unusual in the scientific community at large. One reason given for this approach is that it is easy to discern the authors of a paper from the work presented even if the paper is anonymized and so anonymization is an unecessary burden. But another reason some attendees gave is that when reviewing papers they used author information to access the quality of the work: if the authors were junior or were new to the field these attendees would cut the authors some slack. But if they were well known in the field, these attendees would hold the paper to a higher standard. This bias seems to go against the otherwise positivist approach the field takes. But is it indeed necessary to know the author of a scientific paper in order to judge its quality?

4 Comments:

Blogger Joseph Lorenzo Hall said...

First, unless I'm crazy, no journal in physics, astrophysics, chemisitry or geophysics uses any kind of blinding process (single, double, whatever). Further, I don't know of any movement in these areas to go to anonymous reviewing... it's just not seen as a problem.

Second, I find authorship to be a central part of what I'm reading. For example, I know any paper that my undergrad. advisor writes will be dyno-fuckin-mite, because she's a bad-ass. However, I can see in a decision-making context (grants, reviews, etc.) it might be best to have the initial review stage be blind to really weed the crap out of the system... however, it is a pain in the ass to anonymize a paper and un-anonymizing it after it is accepted could result in something substantially different than what was submitted and accepted.

4:50 PM  
Blogger yardi said...

This reminds me of the Sokal article we read earlier in the semester. I bet the journal would not have accepted the paper if they did a blind review on it. But on the assumption that most scientific authors aren't being deceptive, I would always want to know who the author is. It's fair to expect some context and frame of reference, even if that does necessarily cause bias.
On a similar note, most undergrad professors read papers knowing the author's identity, but I had a couple who preferred to read it blindly first. And TA's or peer reviews were always read blindly. I guess in some cases it is more important to remove all biases than to have context.

11:26 PM  
Blogger yardi said...

This comment has been removed by a blog administrator.

11:30 PM  
Blogger Geoff Nunberg said...

I appreciate the reasons for anonymous refereeing of papers in the abstract, but I don't know that it makes for higher quality contributions to conferences and proceedings. I recall one year when I was refereeing abstracts for the Linguistic Society annual meetings and noted that the committee had turned down a couple of papers by noted linguists who would surely have had something interesting to say about certain topics. I suggested that the Society should institute something like a "Salon des Refus├ęs" where a committee would go over de-anonymized rejected abstracts and select a few that looked interesting. Nothing came of the suggestion, though -- maybe it seemed too elitist.

11:49 AM  

Post a Comment

<< Home