When I am on a conference program committee or editor of a journal, here is how I choose a sub-reviewer who I ask to read and evaluate the submission.
First, I carefully read the part of the introduction that talks about "related work" and about "techniques", looking up every reference. Then, I quickly look through the body of the paper, checking whether some paper is referenced inside the proofs. Then I know that that paper is probably important and the authors of that papers would probably have a well-informed opinion on the submission.
Once I have my list of relevant or very relevant references, I restrict attention to recent papers, discarding anything older than 10 years old, because I assume that, absent other information, the authors of those papers are probably no longer interested in the subject. I look at redundancy, giving higher priority to authors who have written several of the relevant cited papers. I look at conflicts of interest, giving lower priority to people who have recently co-authored papers with the authors of the submission. Then, I give higher priority to authors of papers that have been published in similar venues, because I assume that they will be better able to evaluate the quality threshold. Finally, I try to pick a mix of junior people, who have more time and may be more interested in the submission, and senior people, who may have more perspective and more of a long view on the results submitted.
To avoid awkward situations, in most cases I do the following steps as well:
ReplyDelete1. Remove anyone mentioned in the Acknowledgments section from the list.
2. Remove fellow PC members from the list.
3. Remove the authors of the paper from the list. :-)
Oh, right, good point. I have accidentally sent a paper to fellow PC members (or received a paper from them) but luckily I have yet to send a paper to one of the authors to review!
ReplyDeleteI think I do the stuff you way you do but much faster,
ReplyDeleteor perhaps you just wrote down carefully what you do
instinctively. GOOD ADVICE in any case.
My first year out of grad school my adviser was on a program committee and was assigned a paper of mine.
He (rightly) asked it to be reassigned which it was.
It was then rejected. Oh well.
Great advice! But may be hard to follow to the dot when each PC member has too many papers to review, as it seems to be the case in SODA this year. Our paper was sent to one of the authors for review!! Once the PC member realized the mistake, (s)he apologized for the slip via email, giving the inordinately large number of papers to handle as the reason.
ReplyDelete