In my last post I talked about a variety of data sheets for tracking challenging and replacement behaviors. One of those was the use of rating scales. I love rating scales! I love them for a variety of reasons.
- They are easy to use
- They can self-graph easily
- When done right, they can give a reliable, quantitative method for assessing broader issues, like severity
Having said that…when done wrong they don’t really give us what we need…just like with any type of data collection. So here are 5 ways to make sure we get them right!
Rating scales can be an easy, efficient way to take #data…here’s 5 ways to get them right! #datamonster
1. Clear Rating Scale Anchors
You can solve this issue easily by choosing a rating scale that has already been developed with anchors. However, there aren’t many options out there that truly measure behavior. The one included in my Behavioral Data Sheets (shown below) is based on the research I did for my dissertation and was validated with a group of teachers. The teachers were able to clearly differentiate and agree on the ratings of the severity of behavior using the scale (see below).
If you create your own scale, then you just want to keep some things in mind to make sure your anchors are clear.
- Make it observable–only describe the things you can actually observe. Anger, upset, frustrated would only work if the student is able to rate himself and understand the levels of his feelings. If an observer is rating, rate what he actually did. For instance, you might classify the severity of behavior based on how much it might injure someone (e.g., mild-limited or no injury; moderate-might injure someone else or self, etc.). Or you might determine that crying is a less severe behavior than hitting, so crying gets classified as a 2, but if hitting occurs it automatically starts at a 3.
- Add descriptors–descriptors allow you to expand on the meaning of one word ratings. For instance, what does severe mean? Does it mean injury actually occurred? Or does it mean that it might have occurred if left unchecked?
- Don’t limit yourself to severity–You can use rating scales for duration, frequency, or almost any other measure that goes from high to low. Just be specific. If you are rating duration, is 5 minutes a 3 and 10 minutes a 5? What does it translate into approximately. Remember that a rating scale isn’t an exact measure, but it should give you a good, communicable measure of the behavior over time.
2. Behaviors You Observe
Behaviors You Observe
Make sure that your rating behaviors pass the “dead man’s test.” The Dead Man’s test is a test to determine if something is really behavior–if a dead man CAN’T do it, it qualifies as a behavior.
Here’s an example. Off-task behavior…really isn’t something you can observe. You actually are talking about the absence of actual behavior. Because a dead man can be off-task (he’s lying there, not doing his work). So, you’re better off making your scale measures things you can actually observe to measure. Use on-task behavior instead of off-task, for instance. Inattention is another one–instead, focus on rating how attentive he was. But make sure to follow #1 above and describe what attentive means (e.g., looking at the teacher, following directions, needed no more than 1 reminder, needed frequent reminders, etc.).
Also, don’t forget that you can use them to rate positive behaviors as well–like on-task behavior. You can rate whether they made frequent requests for things they needed or wanted. Whether they asked for help 0, 1, 2, or 3 times.
3. Test It Out
Recognize that if you are using a new scale you haven’t used before, that you are going to need to test it out. Have all of your staff watch videos or observe in the classroom and rate the behaviors on the scale. See if they agree on the ratings. If not, make changes based on that feedback.
Testing it out and doing periodic checks for reliability are important when you use rating scales for #data. #specialeducation
4. Have a Primary Rater w/ Periodic Checks
Consider having 1 (or possibly 2) raters who rate the behaviors everyday. It makes it more likely that the ratings will be consistent if the person observes the behaviors regularly.
For instance, it’s likely that I will rate a behavior as less severe than most teachers because I’ve spent so many years working with students with very severe behaviors. So if I rated one day and someone else with less exposure to severe behavior rated another day, it may look like my day was better when in fact they were the same. My perception was different.
Clearly if you do 1-3 from above, you are less likely to have this problem. But it still is helpful to have some consistency.
With that said, however, it’s also important to do periodic checks with an “outside” rater. It might be that a paraprofessional takes the data each day and the teacher is the outside rater who checks once a week or every other week. Or perhaps it’s the behavior specialist or even an administrator.
Having an regular outside check prevents what is called observer drift. Over time raters are likely to drift in their observations. Behavior that seemed really severe at the beginning starts not looking so bad. Putting checks in place helps to keep them to the anchors on the scale.
5. Rate Shorter Time Periods
Finally, you are more likely to have accurate ratings if you don’t wait too long between the observation of the behavior and when the rating is taken. So, it’s easier to get an accurate rating on the duration of the behavior right after it ends than 5 hours later when it’s more difficult to remember.
Similarly, it’s easier to rate severity of individual episodes of behavior than to try to rate the whole day. Rating the whole day means you have to account for the good, the bad and the ugly behaviors throughout the day.
So, for complex students with different levels of behavior that is likely to happen throughout the day, I will have staff rate the behavior right after the episode ends. Then we’ll average those ratings. You can see an example of this in the data sheet below.
You could also rate the behavior during each activity in the scheduled day. Or rate the behavior in the morning and the afternoon.
The smaller your time ranges are, the less confusion there often is about what to rate. If you do rate for a whole day, like in the picture below, I recommend having raters rate the worst behavior during the day as the rating. This avoids forgetting individual episodes or trying to come up with an average on the fly.
I love using rating scales, but want to make sure that our estimates are reliable for making changes to the behavior plan. These 5 things will keep you out of trouble and make them successful in your classroom.
Until next time,