Calibrating and Collaborative Scoring of Student Work

Video Transcription

Welcome, I'm Trisha Fyfe, and in today's video lesson I'm going to be exploring the topic of calibrating and collaborative scoring of student work. As we learn about this topic we will work towards several learning objectives and together we'll answer the following two questions in this video lesson. What is the purpose of calibrating students work as a collaborative process? And, what does the calibration protocol look like?

Let's start by talking about the question why should we calibrate? Calibration of student's work has many benefits, both for us as teachers and for our students as well. When we calibrate student work as a collaborative process we use rubrics. This creates a consistent and reliable system for scoring. This is the purpose of calibration. Because of the consistency, scorers are able to apply the rubrics in the same way and the scores are the same, regardless of who is scoring the work.

It's also important to have protocol in place to capture the various scorers input. This increases the accuracy of scoring, and also helps the team follow time restrictions and norms. A great resource for calibration is Rhode Island's Department of Education, or the RIDE website, R-I-D-E. We'll look at the calibration protocol for Rhode Island's Department of Education.

In this sample protocol the purpose is to calibrate the scoring of student work and to consider the instructional implications of the prompt or task, student work and rubric. How do you plan and prepare? In this sample protocol, time would take approximately two to three hours depending on the number of pieces of students work. And the group size would be four to eight individuals. The materials for each person would be a prompt or the task, a rubric, student work of course, a score sheet and a score sheet for the recorder.

And the roles would include a facilitator, time keeper and recorder. The process for calibration in this protocol would be first, the facilitator reviews the protocol process with the group and describes the context of the task. Then we get to the examination part, where the members silently examine the prompt, including any text or graphics. Examine the student work and the rubric, making sure they pay attention to the differences in performance descriptors for each level and the score sheet.

The third would be to clarify questions. What questions do the group members have about the materials or the process? This would be the time to ask those.

The fourth step in the calibration process for this protocol would be to read and score. Using the rubric, group members independently and silently read the student work and rank them as high, average, or low, based on their overall impression. The work is scored and the scores are recorded on the score sheet.

The fifth task would be to share the scores. One at a time, team members share their score for each of the rubric categories, without an explanation, and the recorder completes the group's score sheet as a whole. The sixth task would be the discussion. The facilitator invites the groups to consider where the differences in scores occurred and why. Why are they scored differently for each rubric area? Particularly, the highest and lowest scores.

Group members would explain and justify scores, pointing to maybe specific language in the rubric or evidence in the work, and then they would discuss each piece of student work making sure that all of the issues were resolved. Centered on either the meaning of the rubric or merit and the validity of the evidence in the student work, until a consensus was reached.

The very last step in this process, this protocol sample, would be the debrief. They would discuss the following questions after the calibration. What did we notice that scoring student work and using rubric? What would be the next steps for instructing the student? What revisions should be made to the task and instructions? And what are the implications for instructional practice?

So let's now talk about the connections between PLCs, or professional learning communities, and calibration. According to RIDE, or Rhode Island Department of Education, the purpose of all academic assessment is twofold. First, we need to ask what do students know? As well, we need to ask what can students do? Assessments help us gain accurate information about both of these areas.

We see increases in accuracy and results and consistency when we collaborate throughout this entire process. We need to keep in mind the importance of mutual respect and equal participation. These norms, as well as the use of protocol, like the sample we just looked at, help keep us on track as a group, limiting conflicts and establishing guidelines for our conversations. We're able to build a culture and skills that will help us successfully calibrate our student's work.

Let's talk about how technology can help us in the process. Here's an example. First, you could use a Google Drive to upload student work samples into a shared folder. It's important to note it would be important to make sure that there were no student identifiers on this work. Google Docs could be used, the comment function specifically, to allow fellow teachers to make comments. And a form could be built using the protocol and the teachers could score the work.

The summary tool could be used to review the analytics. The final step would be to collaborate together and have dialogue after the protocol is followed completely.

So let's talk about what we learned today. We looked at the following two questions. What is the purpose of calibrating students work as a collaborative process? We also looked at what does the calibration protocol look like? We talked about what calibration of student work is, the process of getting together with a team and using a rubric, protocol, and plenty of mutual respect and equal participation, to grade student work in a consistent and fair way.

We looked at Rhode Island's Department of Education protocol for calibration and also discussed technology that might be beneficial to use when calibrating. Now that you're more familiar with these concepts, let's reflect.

What are the benefits and challenges to using this process to score students work? Can you think of other tools that would help you with this process?

Thanks for joining me today to discuss the lesson calibrating and collaborative scoring of student's work. I hope you found value in these ideas and concepts and you're able to apply them to your own teaching. To dive a little deeper and learn how to apply this information, be sure to check out the additional resources section associated with this video. This is where you'll find links targeted toward helping you discover more ways to apply this course material.