Determining the Quality of Assessment Items in Collaborations: Aspects to Discuss to Reach Agreement Developed by the Australian Medical Assessment Collaboration

The Australian Medical Assessment Collaboration (AMAC) project, funded by the Office of Learning and Teaching, seeks to provide an infrastructure and a road map to support collaboration between Australian medical schools in matters of assessment. This may not seem very new perhaps, because there are already several collaborations taking place in Australia, and, typically, they relate to joint item banks, (such as the IDEAL consortium), or joint test administration, (such as the International Foundation of Medicine tests). The AMAC project seeks to build on these existing collaborations in two ways: first, by tying these initiatives together and thus bundling the combined expertise and experiences in road maps, draft agreements and suggestions for governance structures; and, second, by combining joint examination item production and test administration into one. This should enable continuous meaningful quality comparisons between medical schools, with a view on continuous quality improvement. One contentious issue in similar collaborations concerns differences in perceptions of the quality of test material. Often there are diverse views on what makes a test item high quality or not. This disagreement in views is a serious breakdown risk for collaborations when it cannot be reconciled (Schuwirth, Bosman, Henning, Rinkel & Wenink, 2010). Unfortunately, the determination of ‘quality’ is an inexact science, and the medical education literature does not provide clear-cut answers to questions concerning quality. The role of this document is therefore to provide a framework for quality to help participants make perceptions more explicit and by this, support assessment collaborations.