Scoring Design Decisions

Jilliam N. Joe; Catherine A. Mcclellan; Steven L. Holtzman; Catherine McClellan

doi:10.1002/9781119210856.ch13

Scoring Design Decisions

Jilliam N. Joe, Catherine A. Mcclellan, Steven L. Holtzman, Catherine McClellan

Australian Council for Educational Research (ACER)

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Abstract

This chapter presents two studies that address scoring design decisions associated with observation length and observation instrument structure. MET project data were used in both studies. In Study 1, the amount of time that is necessary to observe and score classroom teaching practice is examined. The results indicate that a score based on the first thirty minutes of the lesson has a strong relationship with the full lesson score when using CLASS, FfT, MQI, PLATO, and QST instruments. In Study 2, differences in inter-rater reliability when the observation instrument requires the observer to focus on a group of traits during a thirty-minute observation, and when an observer scores all traits on the instrument, is examined. The chapter concludes by discussing how these results may be applied in practice. The studies were conducted to examine aspects of classroom observation that impact cognitive complexity, the scoring rubric, and time on task.

Original language	English
Title of host publication	Designing Teacher Evaluation Systems: New Guidance from the Measures of Effective Teaching Project
DOIs	https://doi.org/10.1002/9781119210856.ch13
Publication status	Published - 16 Sept 2015

Keywords

MET Scoring Design
classroom observations
cognitive complexity
inter‐rater reliability
observation length
scoring design
special studies
time on task

Disciplines

Educational Assessment, Evaluation, and Research
Computer Sciences

Access to Document

10.1002/9781119210856.ch13Licence: Unspecified

https://onlinelibrary.wiley.com/doi/abs/10.1002/9781119210856.ch13

Cite this

@inbook{130a717c49c04e31bd12f7014b1abc52,

title = "Scoring Design Decisions",

abstract = " This chapter presents two studies that address scoring design decisions associated with observation length and observation instrument structure. MET project data were used in both studies. In Study 1, the amount of time that is necessary to observe and score classroom teaching practice is examined. The results indicate that a score based on the first thirty minutes of the lesson has a strong relationship with the full lesson score when using CLASS, FfT, MQI, PLATO, and QST instruments. In Study 2, differences in inter-rater reliability when the observation instrument requires the observer to focus on a group of traits during a thirty-minute observation, and when an observer scores all traits on the instrument, is examined. The chapter concludes by discussing how these results may be applied in practice. The studies were conducted to examine aspects of classroom observation that impact cognitive complexity, the scoring rubric, and time on task. ",

keywords = "MET Scoring Design, classroom observations, cognitive complexity, inter‐rater reliability, observation length, scoring design, special studies, time on task",

author = "Joe, {Jilliam N.} and Mcclellan, {Catherine A.} and Holtzman, {Steven L.} and Catherine McClellan",

note = "Please check your email for instructions on resetting your password. If you do not receive an email within 10 minutes, your email address may not be registered, and you may need to create a new Wiley Online Library account.",

year = "2015",

month = sep,

day = "16",

doi = "10.1002/9781119210856.ch13",

language = "English",

booktitle = "Designing Teacher Evaluation Systems: New Guidance from the Measures of Effective Teaching Project",

}

TY - CHAP

T1 - Scoring Design Decisions

AU - Joe, Jilliam N.

AU - Mcclellan, Catherine A.

AU - Holtzman, Steven L.

AU - McClellan, Catherine

N1 - Please check your email for instructions on resetting your password. If you do not receive an email within 10 minutes, your email address may not be registered, and you may need to create a new Wiley Online Library account.

PY - 2015/9/16

Y1 - 2015/9/16

N2 - This chapter presents two studies that address scoring design decisions associated with observation length and observation instrument structure. MET project data were used in both studies. In Study 1, the amount of time that is necessary to observe and score classroom teaching practice is examined. The results indicate that a score based on the first thirty minutes of the lesson has a strong relationship with the full lesson score when using CLASS, FfT, MQI, PLATO, and QST instruments. In Study 2, differences in inter-rater reliability when the observation instrument requires the observer to focus on a group of traits during a thirty-minute observation, and when an observer scores all traits on the instrument, is examined. The chapter concludes by discussing how these results may be applied in practice. The studies were conducted to examine aspects of classroom observation that impact cognitive complexity, the scoring rubric, and time on task.

AB - This chapter presents two studies that address scoring design decisions associated with observation length and observation instrument structure. MET project data were used in both studies. In Study 1, the amount of time that is necessary to observe and score classroom teaching practice is examined. The results indicate that a score based on the first thirty minutes of the lesson has a strong relationship with the full lesson score when using CLASS, FfT, MQI, PLATO, and QST instruments. In Study 2, differences in inter-rater reliability when the observation instrument requires the observer to focus on a group of traits during a thirty-minute observation, and when an observer scores all traits on the instrument, is examined. The chapter concludes by discussing how these results may be applied in practice. The studies were conducted to examine aspects of classroom observation that impact cognitive complexity, the scoring rubric, and time on task.

KW - MET Scoring Design

KW - classroom observations

KW - cognitive complexity

KW - inter‐rater reliability

KW - observation length

KW - scoring design

KW - special studies

KW - time on task

U2 - 10.1002/9781119210856.ch13

DO - 10.1002/9781119210856.ch13

M3 - Chapter

BT - Designing Teacher Evaluation Systems: New Guidance from the Measures of Effective Teaching Project

ER -