Using Log Stream Data and Item Response Theory to Understand Proficiency in Collaboration

The nature of skills such as collaboration is complex, given that there are internal processes at play and inferences need to be made to interpret explicit behaviours observed from intentionally designed assessment tasks. The theoretical concept of the paths presented in this chapter is to a priori define the processes being sought in the process stream data and interpret their substantive meaning in relation to the skill being measured. This chapter centres on a learning analytical approach to reduce complex skills into simpler but observable components that can be represented as aspects of the target construct and scaled to understand the growth of such skills. The method outlined in this chapter presents an approach that transforms process stream data of collaboration into formative assessment data that can be analysed using Item Response Theory. The application of this model in turn allows the interpretation of the data as levels of proficiency that we can use to map or monitor progress in collaborative skills.