Reviewer Best Practices

To the Reviewer,
Thank you for being willing to review for ICCC. The reviews you provide contribute directly to improving the quality of CC research, to reinforcing the fabric of our community, and to encouraging the efforts of individual researchers. These efforts are essential to the success of our community and the conference. In the interest of helping you be successful in this role, we are providing the following suggested guidelines to assist you.

 

Double-Blind Review

The double-blind review process means that submitting authors do not know the identities of reviewers and that reviewers do not know the identities of submitting authors. This process helps to ensure that reviews and recommendations for acceptance/rejection are unbiased and based solely on the merits of the submitted work. This process also helps to encourage honest reviews and feedback from reviewers. The EasyChair system is built to facilitate double-blind reviews. Authors should anonymize manuscripts prior to submission. This entails removing or replacing author names and affiliations, as well as any self-citations or URLs that would clearly disclose their identity. Likewise, reviewers should avoid communication that would compromise the integrity of the double-blind review process.
 

Confidentiality

Ideas and results presented in papers under review are to be kept strictly confidential. Likewise, all communication that takes place as part of the review process should also be kept confidential. Reviewers also agree not to distribute submitted papers, results, or ideas to anyone without explicit permission from program chairs.
 

Appendices and Supplementary Information

Reviews should be based on the main content of the paper. Reviewers are not required to review or take into account appendices or supplementary information when completing their reviews.

 

Preferred Organization of the Review

  1. Summarize the aim or contribution of the paper in your own words. This helps program chairs and authors know that the reviewer understood the paper.
  2. Clearly state your decision to accept or reject the paper.
  3. Briefly summarize the primary reasons that justify the reviewer’s scoring of the paper.
  4. Provide additional supporting arguments for your decision to accept/reject the paper in a specific, clear, and constructive manner, and other feedback on ways in which the paper can be improved. These may include:
    • Support claims (e.g., about lack of novelty, clarity, or grammaticality) with evidence and specific examples
    • Pointing out minor fixes, including typos and grammatical errors, is also appropriate and encouraged

 

Areas of Review Focus

Reviews should focus on the strengths and weaknesses of the submitted manuscript in terms of:

  • the appropriateness of the manuscript content to the goals and interests of conference;
  • the significance of the problem being addressed;
  • how well-situated and contextualized the work is to other related work;
  • the cohesiveness and accuracy of language, in communicating ideas, methods, and results;
  • how well authors substantiate claims made in the manuscript;
  • effectiveness and clarity of figures and tables.

 

Best Practices

The following are good practices for reviewing:

  • Focus on the scientific contents and contributions of the paper
  • Do more than a superficial reading of the paper
  • Seek to balance critical feedback with positive reinforcement
  • Use the review as an opportunity to help make the paper better
  • Be assertive rather than passive or neutral
  • Use unambiguous language
  • Be kind, honest, and optimistic
  • Use positive language where possible (e.g., “the paper needs to do X” vs. “the paper doesn’t do X”)
  • Focus review on merits of the paper rather than reviewer’s sentiments or ideology

The following are practices that are not appropriate for reviewing:

  • Harassment, bullying, derogatory or toxic language
  • Discrimination based on age, gender, sexual orientation, religion, race, medical condition, or disability
  • Statements that make assumptions about authors’ demographics (e.g., “the paper needs to be reviewed by a native English speaker”)

 

Confidential Comments to the Program Committee

The confidential comments section of the review can be useful for many things, for example: suggesting a paper be accepted for oral versus poster presentation; raise concerns about plagiarism or similarity to other works; or disclosing conflicts of interest. This section also provides an opportunity for the reviewer to use bluntness where needed in communicating to the PC without offending the authors.
 

Some excellent resources on how to be a good reviewer