North Carolina Improves Calibration of their Evaluators with Observation Engine

The North Carolina Department of Public Instruction (NCDPI) is using Observation Engine to improve the calibration of their teacher evaluators.  And it works!   In the pilot of their Observation Calibration Training (OCT) resources during the 2014–15 school year, NCDPI worked with administrators fOCT logorom 32 LEAs. Educators from BloomBoard organized training webinars for which Observation Engine was accessed through their online tools.  After completing an initial calibration assessment the administrators were given access to a suite of observation practice exercises designed to help them improve their observation skills. These exercises were created by McREL International using Observation Engine’s Lessons feature. At the end of the pilot, administrators were given a second calibration assessment. Empirical Education conducted a formal analysis of scoring data as well as survey and focus groups feedback. The analysis showed that there was statistically significant scoring improvement from the first calibration assessment to the second. In addition, the improvement in scoring was directly related to the number of exercises each observer completed-–the more Lessons observers completed, the more their scoring improved. This is the first formal study showing that Observation Engine Lessons improves scoring for teacher evaluators. Survey and focus group feedback documented positive feedback on the experience as well as insightful suggestions for improving the OCT in the future. To access the full report from the NCDPI pilot, click here.

During the 2015–16 school year, NCDPI is rolling out the OCT to administrators across the state. This implementation will focus on establishing a collaborative element to the experience, encouraging administrators to engage in conversations around the videos, the rubric, scoring, and classroom observation in general. The goal is to enhance the experience for participants and help foster a supportive community resource for teacher evaluators.

As NCDPI is moving forward with this year’s roll-out, other Observation Engine clients will also continue their use of the tool. These include: The Northeast Florida Education Consortium, Cooperative Counsel for Oklahoma School Administration, Tulsa Public Schools, Pittsburgh Public Schools, Texas Region 18 ESC, and New Mexico Public Education Department.


Proposal coordinator and marketing manager for Empirical Education Inc.

Posted in News Tagged with: , , , , , , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *