Intro to the Diagnostic Classification Model
DCM or Diagnostic Classification Model is a powerful way to understand students' mastery of subjects. Watch our intro to DCM video to learn more!
[MUSIC PLAYING] DR. JONATHAN TEMPLIN: My research is in the field of psychometrics. That's the field that provides us test scores and what we know about them to better understand what people understand about a subject area. So the field of psychometrics is really vast. And it has evolved a lot. It kind of evolved into is this area called Diagnostic Classification Models.
These are sort of classification-based models where, instead of providing a score for a person, we go and tell them which group they're in. Are they in the high-performing or mastery group? Are they in a low performing or non-mastery group? We came to find out that it's a whole lot easier, when it comes to a reliability sense, to classify people into groups rather than to provide a continuous test score. It turns out, it takes far fewer items to give a reliable measure, a consistent measure, of that classification based. We tried to quantify this. And what we showed is, to get a minimum level of reliability, a minimum-- the index itself goes from zero to one, minimum acceptable, is usually we think of this point 0.
8, at least in the research literature. For a basic Item Response Theory model, or IRT model, it would take something along the lines of 35 items. All right, so that's a lot of information to give you one score. For a DCM, it took about seven items, seven to eight items, to give you that same level of information. So that is one fifth the length of the test.
That's one fifth of the time and the theory. The algorithms that we use now are fairly advanced. They work well with multidimensional data. They work well with small sample sizes, things that you would find a lot in the classroom settings, particularly formative settings, where assessments seem to have the most impact. RICH WILSON: We had been using an assessing platform for a number of years.
And we would collect the data. And then we would do very little with it other than judge classrooms, judge teachers, judge buildings. But we could-- there was no action to it. You know, any larger test that you give, any state assessment that you give, the data has a shelf life. And sometimes, it's too big and not specific enough to be actionable for teachers.
So what the DCM has afforded us is that opportunity to give a valid and reliable assessment that set the standards level. The shortness of them means that we can react to them immediately. We don't have to wait for somebody else to report the data to us. We get it immediately. What I really think what it's done is it's allowed our footprint of assessing to be back in the classroom.
We don't have to borrow time from another class to take a larger assessment. We could do other things in class besides taking the assessment. So you're talking about shifting mindsets away from what students have completed or not completed to what they've learned and not learned. We have wanted to create and maintain an atmosphere where we are not over-testing our students. One of the things that we looked at is how we can triangulate our data in an effective way.
One of the hangups that districts had in building this process themselves is the lack of confidence that teachers have in building sound, reliable assessment. As a fifth-grade teacher on a PLC team, that's real. We do not get enough training in our pre-service education to be assessment experts. The DCM is the go-to model. Teachers can have the confidence that this is providing them the feedback they need.
It is classifying the students correctly. But because of the time that the information comes back, I just taught a standard. I have an assessment that measures that standard. And I know that that is reliable. I can now trust that, if my formative practice is spot on, it's good, I'm asking the right questions.
I'm getting at the heart of it while I'm teaching. If it's off, if my results are not saying the same thing as the DCM assessment is, I need to tweak a few things. DR. JONATHAN TEMPLIN: And what MasterViews provides is the ability to track what students have learned as they're learning it in real time throughout the year. If we can accurately track where a student is going and what they're currently learning, not only do we get the information that we can use in instruction, but in the big-picture sense, that integrate assessment, the need for it, is now eliminated.
Because we know what students know already. Rather than pile up a big assessment with big sticks and lots of stress at the end of the year, my approach, or my philosophy, is, let's understand what student know throughout the school year and then forget that thing at the end, and all the prep that goes into it. Recover that time. And keep teaching them that. DR.
ZACH CONRAD: We have forgotten that teachers and students need this data the most, not to perform well on end-of-level assessments, but so that we can change instructional practice in the moment. We know that the formative process and formative assessments are what move the needle on student learning. That's really where assessments need to be, supporting teaching and learning, providing the feedback for teachers and students. RICH WILSON: The DCM that is so easy to use, it's quick, it's within the same structure of any other assessment I would give. It does it feel different to the kids.
We have gotten teachers to the point where they want to assess. They want to find out where their students are. But they don't want to just leave it up to chance where they're going. They want to be able to influence the outcome even more.
These are sort of classification-based models where, instead of providing a score for a person, we go and tell them which group they're in. Are they in the high-performing or mastery group? Are they in a low performing or non-mastery group? We came to find out that it's a whole lot easier, when it comes to a reliability sense, to classify people into groups rather than to provide a continuous test score. It turns out, it takes far fewer items to give a reliable measure, a consistent measure, of that classification based. We tried to quantify this. And what we showed is, to get a minimum level of reliability, a minimum-- the index itself goes from zero to one, minimum acceptable, is usually we think of this point 0.
8, at least in the research literature. For a basic Item Response Theory model, or IRT model, it would take something along the lines of 35 items. All right, so that's a lot of information to give you one score. For a DCM, it took about seven items, seven to eight items, to give you that same level of information. So that is one fifth the length of the test.
That's one fifth of the time and the theory. The algorithms that we use now are fairly advanced. They work well with multidimensional data. They work well with small sample sizes, things that you would find a lot in the classroom settings, particularly formative settings, where assessments seem to have the most impact. RICH WILSON: We had been using an assessing platform for a number of years.
And we would collect the data. And then we would do very little with it other than judge classrooms, judge teachers, judge buildings. But we could-- there was no action to it. You know, any larger test that you give, any state assessment that you give, the data has a shelf life. And sometimes, it's too big and not specific enough to be actionable for teachers.
So what the DCM has afforded us is that opportunity to give a valid and reliable assessment that set the standards level. The shortness of them means that we can react to them immediately. We don't have to wait for somebody else to report the data to us. We get it immediately. What I really think what it's done is it's allowed our footprint of assessing to be back in the classroom.
We don't have to borrow time from another class to take a larger assessment. We could do other things in class besides taking the assessment. So you're talking about shifting mindsets away from what students have completed or not completed to what they've learned and not learned. We have wanted to create and maintain an atmosphere where we are not over-testing our students. One of the things that we looked at is how we can triangulate our data in an effective way.
One of the hangups that districts had in building this process themselves is the lack of confidence that teachers have in building sound, reliable assessment. As a fifth-grade teacher on a PLC team, that's real. We do not get enough training in our pre-service education to be assessment experts. The DCM is the go-to model. Teachers can have the confidence that this is providing them the feedback they need.
It is classifying the students correctly. But because of the time that the information comes back, I just taught a standard. I have an assessment that measures that standard. And I know that that is reliable. I can now trust that, if my formative practice is spot on, it's good, I'm asking the right questions.
I'm getting at the heart of it while I'm teaching. If it's off, if my results are not saying the same thing as the DCM assessment is, I need to tweak a few things. DR. JONATHAN TEMPLIN: And what MasterViews provides is the ability to track what students have learned as they're learning it in real time throughout the year. If we can accurately track where a student is going and what they're currently learning, not only do we get the information that we can use in instruction, but in the big-picture sense, that integrate assessment, the need for it, is now eliminated.
Because we know what students know already. Rather than pile up a big assessment with big sticks and lots of stress at the end of the year, my approach, or my philosophy, is, let's understand what student know throughout the school year and then forget that thing at the end, and all the prep that goes into it. Recover that time. And keep teaching them that. DR.
ZACH CONRAD: We have forgotten that teachers and students need this data the most, not to perform well on end-of-level assessments, but so that we can change instructional practice in the moment. We know that the formative process and formative assessments are what move the needle on student learning. That's really where assessments need to be, supporting teaching and learning, providing the feedback for teachers and students. RICH WILSON: The DCM that is so easy to use, it's quick, it's within the same structure of any other assessment I would give. It does it feel different to the kids.
We have gotten teachers to the point where they want to assess. They want to find out where their students are. But they don't want to just leave it up to chance where they're going. They want to be able to influence the outcome even more.