Reviewer Best Practices

Dear Reviewer,

Thank you for being willing to review for ICCC. The reviews you provide contribute directly to improving the quality of CC research, to reinforcing the fabric of our community, and to encouraging the efforts of individual researchers. These efforts are essential to the success of our community and the conference. In the interest of helping you be successful in this role, we are providing the following suggested guidelines to assist you.

 

ICCC-specific Guidelines

Computational Creativity VS Generative AI

Due to the rise in popularity of Generative AI, we would like to emphasize the differences and similarities between Computational Creativity and Generative AI. It is important that ICCC submissions are evaluated in the context of Computational Creativity, rather than focusing exclusively on Generative AI priorities.

While Generative AI concerns machine learning methodology for generation, often emphasizing large models, Computational Creativity comes from a broader context. Having been developed for over a decade and a half, Computational Creativity (or CC) is a discipline with its roots in Artificial Intelligence, Cognitive Science, Engineering, Design, Psychology and Philosophy that explores the potential for computers to be autonomous creators in their own right. ICCC is an annual conference that welcomes papers on different aspects of CC, on systems that exhibit varying degrees of creative autonomy, on systems that act as creative partners for humans, on frameworks that offer greater clarity or computational felicity for thinking about machine (and human) creativity, on methodologies for building or evaluating CC systems, on approaches to teaching CC in schools and universities or to promoting societal uptake of CC as a field and as a technology, and so on. Generation of creative artifacts makes up just one facet of computational creativity.

Full Paper Types
  • Technical papers: These are papers posing and addressing hypotheses about aspects of creative behaviour in computational systems. The emphasis here is on using solid experimentation, computational models, formal proof, and/or argumentation that clearly demonstrates advancement in the state of the art or current thinking in CC research. Strong evaluation of approaches through comparative, statistical, social, or other means is essential.
  • System or Resource description papers: These are papers describing the building and deployment of a creative system or resource to produce artefacts of potential cultural value in one or more domains. The emphasis here is on presenting engineering achievement, technical difficulties encountered and overcome, techniques employed, reusable resources built, and general findings about how to get computational systems to produce valuable results. Presentation of results from the system or resource is expected. While full evaluation of the approaches employed is not essential if the technical achievement is very high, some evaluation is expected to show the contribution to CC of this work.
  • Study papers: The emphasis here is on presenting enlightening novel perspectives related to the building, assessment, or deployment of systems ranging from autonomously creative systems to creativity support tools. Such perspectives can be presented through a variety of approaches including ethnographic studies, thought experiments, comparisons with studies of human creativity, and surveys. The contribution of the paper to CC should be made clear in every case.
  • Cultural application papers: These are papers presenting the use of creative software in a cultural setting, for example via art exhibitions/books, concerts/recordings/scores, poetry or story readings/anthologies, cookery nights/books, results for scientific journals or scientific practice, released games/game jam entries, and so on. The emphasis here is on a clear description of the role of the system in the given context, the results of the system in the setting, technical details of inclusion of the system, and evaluative feedback from the experience garnered from public audiences, critics, experts, stakeholders, and other interested parties.
  • Position papers: These are papers presenting an opinion on some aspect of the culture of CC research, including discussions of future directions, past triumphs or mistakes, and issues of the day. The emphasis here is on carefully arguing a position; highlighting or exposing previously hidden or misunderstood issues or ideas; and providing thought leadership for the field, either in a general fashion or in a specific setting. While opinions need not be substantiated through formalization or experimentation, any justification of a point of view will need to draw on a thorough knowledge of the field of CC and of overlapping areas, and provide relevant motivations and arguments.
Short Papers: Difference between long and short papers

Long papers are expected to be significant scientific contributions, complete with evaluation where appropriate. On the other hand, short papers are intended to share new directions and ideas, spark debate, or otherwise enrich the conference and provide value to the community, without the same evaluation and rigour requirements of long papers. They are not merely long papers with fewer pages. To this end, different review criteria will be applied to long and short papers.

General Aspects

Double-Blind Review: The double-blind review process means that submitting authors do not know the identities of reviewers and that reviewers do not know the identities of submitting authors. This process helps to ensure that reviews and recommendations for acceptance/rejection are unbiased and based solely on the merits of the submitted work. This process also helps to encourage honest reviews and feedback from reviewers. The EasyChair system is built to facilitate double-blind reviews. Authors should anonymize manuscripts prior to submission. This entails removing or replacing author names and affiliations, as well as any self-citations or URLs that would clearly disclose their identity. Likewise, reviewers should avoid communication that would compromise the integrity of the double-blind review process.

Confidentiality: Ideas and results presented in papers under review are to be kept strictly confidential. Likewise, all communication that takes place as part of the review process should also be kept confidential. Reviewers also agree not to distribute submitted papers, results, or ideas to anyone without explicit permission from program chairs.

Appendices and Supplementary Information: Reviews should be based on the main content of the paper. Reviewers are not required to review or take into account appendices or supplementary information when completing their reviews.

 

Writing a review

Guidelines to help you write a good review.

Preferred Organization of the Review
  1. Summarize the aim or contribution of the paper in your own words. This helps program chairs and authors know that the reviewer understood the paper.
  2. Clearly state your decision to accept or reject the paper.
  3. Briefly summarize the primary reasons that justify the reviewer’s scoring of the paper.
  4. Provide additional supporting arguments for your decision to accept/reject the paper in a specific, clear, and constructive manner, and other feedback on ways in which the paper can be improved. These may include:
    • Support claims (e.g., about lack of novelty, clarity, or grammaticality) with evidence and specific examples
    • Pointing out minor fixes, including typos and grammatical errors, is also appropriate and encouraged
Areas of Review Focus

Reviews should focus on the strengths and weaknesses of the submitted manuscript in terms of:

  • the appropriateness of the manuscript content to the goals and interests of conference;
  • the significance of the problem being addressed;
  • how well-situated and contextualized the work is to other related work;
  • the cohesiveness and accuracy of language, in communicating ideas, methods, and results;
  • how well authors substantiate claims made in the manuscript;
  • effectiveness and clarity of figures and tables.
Best Practices

The following are good practices for reviewing:

  • Focus on the scientific contents and contributions of the paper
  • Do more than a superficial reading of the paper
  • Seek to balance critical feedback with positive reinforcement
  • Use the review as an opportunity to help make the paper better
  • Be assertive rather than passive or neutral
  • Use unambiguous language
  • Be kind, honest, and optimistic
  • Use positive language where possible (e.g., “the paper needs to do X” vs. “the paper doesn’t do X”)
  • Focus review on merits of the paper rather than reviewer’s sentiments or ideology

The following are practices that are not appropriate for reviewing:

  • Harassment, bullying, derogatory or toxic language
  • Discrimination based on age, gender, sexual orientation, religion, race, medical condition, or disability
  • Statements that make assumptions about authors’ demographics (e.g., “the paper needs to be reviewed by a native English speaker”)
Confidential Comments to the Program Committee

The confidential comments section of the review can be useful for many things, for example: suggesting a paper be accepted for oral versus poster presentation; raise concerns about plagiarism or similarity to other works; or disclosing conflicts of interest. This section also provides an opportunity for the reviewer to use bluntness where needed in communicating to the PC without offending the authors.

 

Other resources

Some excellent resources on how to be a good reviewer: