three years of papers from ICER. We describe the process of assessing the inter-rater
reliability of the system and fine-tuning it along the way. Our analysis of the ICER papers
confirms that ICER is a research-intensive conference. It also indicates that the research is
quite narrowly focused, with the majority of the papers set in the context of programming
courses. In addition we find that ICER has a high proportion of papers involving more than …