Getting a valid measurement on complex language skills is challenging. Rubrics are a great tool for measuring multiple skills with multiple levels of complexity. They can also be valuable motivators for clients to monitor their own progress.
Rubrics to the Rescue! discusses the when, how, who, and what of using rubrics in language therapy.
— Useful Links —
Music: Simple Gifts performed by Ted Yoder, used with permission
Hello, and welcome to episode 63 of The Speech Umbrella podcast. I’m your host, Denise Stratton and today’s episode is called Rubrics to the Rescue. When it comes to measuring progress with some skills, it’s very hard to show actual progress without using something like a rubric. Now, if you’re not familiar with rubrics, hang in there because we will get to the specifics in just a bit.
I’m so excited about what you’ll walk away with today after listening to this podcast, especially if you haven’t been using rubrics before. And this is because I am so pumped up about helping clients help themselves.
The questions I want to answer today are
Before I dive into those questions I’m going to describe a rubric as best I can. I grew up in the dark ages, pre Rubric era. I don’t know how many of you are familiar with them. A Rubrik is a scoring guide, used to evaluate performance, a product or a project. It has three parts, performance criteria, rating, scale, and indicators.
It basically looks like a grid or a table. Down the left side the criteria describe the key elements of student work or the product. The top of the rating scale identifies levels of performance. Under each section of the rating scale, the indicators provide examples or concrete descriptors for each level of performance.
So criteria for a narrative might include referring to characters, sentence accuracy, and describing the setting. Those would be three separate criteria. The rating scale defines how well they did
You might be looking at things like frequency, precision, or accuracy, and they’re usually numbered 4, 3, 2, 1 with four being the best performance.
The indicators describe in detail what is expected for each criteria at each level. For me, this is the hardest part of creating a rubric. This is where most of my time is spent tweaking a rubric for a specific client.
Here’s an example of a rubric I made for retelling or summarizing a story. You’ll notice I have the performance descriptions written in first person.
My first criteria is: introduced the story
Under levels of performance in the four column it says I named and described the main character. I named the setting and described what the character was doing or wanted at the beginning.
Under the three column, it says I named and or described the main character And the setting.
Under the two column. It says, I talked about the character or setting, but not both.
And under the one column, it says, I didn’t write about the character or the setting in my first few sentences.
My reasons for this criteria is because so many of our clients don’t set up the story. They don’t give the setting and they don’t tell what the character’s situation was at the beginning. they just launch into the story and the listener is thinking what? What’s going on?
If I were using this rubric for written work I might also include criteria for word and sentence accuracy, and just change up the wording on the first criteria to reflect writing rather than speaking. Here’s an example:
Under the four column it reads all of my sentences make sense because there are no missing words or words out of order and my past tense verbs are correct.
Under the three column it reads all my sentences make sense because there are no missing words and all the words are in order.
And under the two column it says, most of my sentences are correct, but I left out one to three words or got them in the wrong order.
Finally under the one column it reads: my story doesn’t make sense because more than three words are missing, or are out of order.
Notice that only the highest criteria requires correct verb tense. That’s because I’ve found it’s one of the last things to get remediated in written work, and usually the meaning of their stories are still intact even with verb tense errors. I’m giving priority to meaning over form, although I do want them to eventually get the form correct too. A rubric is such a great way to show someone how well they are doing, even when they still have errors.
Why rubrics?
Now I hope everyone has a clear picture of rubric because we’re going dive into the questions I posed at the beginning. One: why rubrics?
When it comes to measuring complex language skills, there is a lot going on. Story telling can become very complex for instance.
I like to compare a client to telling a story to juggling. There’s a ball for referring to characters, a ball for introducing the story, and another one for including all parts of the story. That’s the story grammar. Keep on adding balls for constructing sentences and for using specific vocabulary. Let’s not forget to add balls for length of story and complications. Is your head spinning yet? Some of our clients’ heads are for sure!
When a client can say the most basic of sentences, the who, and what of a story, They’re already juggling two balls. Add information about where, and now they are managing three balls. They need to describe the problem —add another ball. And so it goes, but here’s the thing. When a juggler drops a ball, it’s not because she has a problem with that particular ball. It’s because she added one too many balls. Someone who is competent with a certain level of narrative can lose it all when one too many balls are in the air and all the balls come crashing down.
The first reason for rubrics is to help us move away from specific parts of language like character names and pronouns, specific vocabulary, conjunctions, and so forth to measuring how well our client can keep all the balls in the air. At a certain level of complexity that’s really the problem. It’s one of integration. It’s not about whether they know how to do all these skills separately. It’s about how well they can put them all together.
And the second reason to use rubrics is to help clients begin to monitor themselves. If you’ve listened to many of my podcasts, you know how big I am into self-monitoring. A rubric is a really, really great way to help a client begin to monitor themselves and to be thoughtful about their speech or about their writing. The reason my rubrics are written in first person is because I share the rubric with them and we score it together.
What clients?
Because rubrics are so good at taking a multi-dimension look at language, I use them a lot with language clients who are integrating a number of skills in spoken or written language. For example, they might be telling stories with complications, they might be comparing and contrasting facts or they might be summarizing facts or stories. All of those would lend themselves well to using rubrics. When you notice the cognitive load is getting so high that your clients starts dropping balls—balls you KNOW they KNOW—then a rubric could help them organize their delivery.
I do use them with a few other clients too, like fluency clients. But mainly here, we’re going to talk about language, written and spoken language, because I’ve got whole other podcast on fluency and rubrics in the wings.
I generally use rubrics with upper elementary and junior high clients. There’s just so much information on a rubric. It can be overwhelming for someone who’s younger. But you can simplify them and only have a couple of rows, nor do you have to use 4 levels of performance. I used a rubric with a 10 year old once, and it was effective.
Those are just some of the things I keep in mind as I look at clients who might need rubrics and I’m sure I still have many more ways to explore using rubrics.
Being Effective
How to use rubrics in the most effective way is almost the same question as what makes a good rubric? A good rubric needs to effective both for my prep time and my client’s benefit. It needs to check both those boxes. Creating a rubric from scratch whenever I need it is NOT my idea of being effective, so I started searching online.
When I first decided to do this podcast on rubrics, I actually thought it was going to have a different format. I decided I was going to explore and try out three different websites that offered rubrics and do a review of all of them and what my recommendations were. Well what really happened is I explored two websites the first one was really no good, it wasn’t flexible, the second website met all my needs and I thought well okay why go on exploring when I have what I need here.
The website I use is called Rubi Star. It’s R U B I S T A R dot then the numeral four. teachers.org.
Rubistar4 teachers was designed by the US Department of Education, and I think they’ve done a pretty good job. It’s nice to see our tax dollars at work!
They have several sample rubrics you can use as a jumping off point, so there is no need to reinvent the wheel.
Here’s the three main reasons that I like RubiStar for teachers. They have a really nice collection of templates in a wide variety of topics. I mostly end up using the ones under oral presentation and story writing.
Even though these templates are created for general classroom use and not special education use, a lot of the language is really customizable to what I need and I have taken a lot of their language and used it.
I like how you can save a template under one name and then duplicate it and then change it so that you can adapt it for another client. It’s really, really easy to do that. And also other users can make their rubrics public, so we’re able to look at what other people have designed and borrow their rubrics if they have made it public. Rubistar4teachers saves me TONS of time. Once you get a few rubrics customized you might find you only need minor changes to use them with other clients; that’s been my experience.
If you want three levels of performance rather than four you can do that, but you’ll be working from your own template rather than adapting one. You can have as many or as few criteria as you want.
That’s being effective from a prep time standpoint,
Let’s move on to what makes a good rubric. Totally my opinion here, but from my experience, this is what I have found to work and not work. You don’t want too many criteria to begin with. It’s always easier to add criteria than to take it away. So for example, if your client is not ready to have their spelling and punctuation included in the rubric, leave it off because they’re just going to get discouraged If they have to think about so many things.
Another thing that makes a really good rubric is one where you are able to think of clear and distinct differences between the score of the four, the three, the two, and the one.
It’s really important that our clients understand why the score of two is better than one. What I mean by that is that they understand what they can do differently with their writing or their language that makes the two better than the one and the three better than the two and the four best of all.
This leads me to my next point, is that the rubric is so easy for them to understand that they can score themselves. One of the huge values of a rubric is when clients begin to monitor their own language or their own writing or their own speech, because they clearly understand what makes it better. I also leave room for them to grow on the rubric. Sometimes the four column that I have describes a pretty high standard, while a three is adequate. Say if they were to get threes on every criteria. That is still pretty good, but then they’ve got room to grow and think about making it even better.
And finally a really good rubric is the one that works for that client. So you can measure what you need to measure, and they can see how they need to improve.
Here’s an example of a rubric I used for a very particular language skill.
I had a client on the autism spectrum who could tell a great story except for one thing: he wouldn’t indicate who was speaking, which is called role shifting. He wouldn’t say “he said” or “she asked”, instead he would use different voices to indicate when different characters were speaking, which is great if you’re in a play, but really confusing for the listener of a story. He had this roadblock in his mind about role shifting.
Then rubrics changed things for him. I used this wording for role shifting almost word for word, that was on the RubiStar template.
Under the four column, it says shifts roles smoothly almost all the time. It is very easy for the audience to tell which character is speaking. Under the third column, it says shifts roles pretty well, is usually easy for the audience to tell which characters speaking. Under the second column, it says, tried to do role shifting, but wasn’t very successful. It was often hard for the audience to tell which character was speaking. And under the one column it says, did not attempt to role shift. It was very difficult for the audience to tell which character was speaking.
Because this client loved the gamification of the rubric, the earning of points, he let down the roadblock and met his goal of using role shifting.
Sometimes you have to speak it a little bit. It’s taken me sometimes two or three sessions to tweak the rubric so it is at the point where it’s really measuring what i want it to measure and the client can really use it to monitor themselves. that’s when I know I’ve got a really good rubric.
Real examples
Now for the fun part! Get ready to hear the amazing difference in quality from a pre and post rubric task.
I have a client who’s been working with me a long time. She’s been working on language, reading, writing, and phonological awareness. And she has slowly wended her way through telling and writing a simple story. A simple story just has those five parts: the character, the problem, the feeling, the action and the resolution.
Then she moved on to telling and writing a story with a complication, then on to telling and writing about parts of longer and more complex stories, which often means there’s just more than one complication. And she was doing really well at all these and she was using conjunctions and mental verbs really well. So I thought she was ready to fly solo and retell an entire story with multiple complications.
After we watched the Simon’s Cat video, Light Lunch, I asked her to retell it. And this is her retell.
There was this, so it was Christmas or almost Christmas and he was hungry and he decided to eat some snacks, but the cat came over and said meow, He was hungry. So he was pointing at his mouth, cause that means he’s hungry. And then Simon did not let him, cause he’s going to get sick. And when he decided to go away too, he could eat his food.
And he went and then the cat crept up a little bit and came up and then eat the crackers. He licked the little thingys, sausages. And then when he crunched the pickle, he just throw it up on the ceiling, into the lamp. And then when he eat the cracker, he was like, and he spit it out. And then when he eats it all, he decided to get some big water, dip it with his paw. And then when he says food and then when he came over, when he hide behind the couch and he’s like, Hmm. And then he liked mew walked away and then he ate the sausages and then a pickle fall on him. He wondered why was a pickle on my head?
Well, as you can tell, that was a bit of a train wreck. Not only did her story not make sense. Her verb tenses regressed big time and that’s something she’s been working on a long time and had been doing very well at. So what happened? A lot of balls dropped, right?
One of the things I realized after listening to her retell is that she was trying to include every single little detail in the story. And this was a story with multiple complications. So I did tell her that she didn’t need to describe everything, just use a few sentences.
Now I didn’t actually tell her to make a summary, but she naturally defaulted to creating a summary after I told her to limit the number of her sentences. And it was awesome because we had never talked about summaries. Summaries require a high level of language organization.
The other thing I told her was I told her, you cannot use the word then. Then was her kryptonite, it just got her going into run-on sentences. I actually put that in the rubric—using a variety of conjunctions, but no “then”. The rubric also had describing the character and setting and using mental verbs, and describing the problem.
Listen to her post rubric of Light Lunch and hear how she used ALL of those skills. One day Simon decided to eat Christmas lunch. But when Simon’s cat was hungry, he decided to trick him into leaving the room. While Simon was getting cat food, Simon’s cat jumped on the couch and ate Simon’s food
Talk about rubrics coming to the rescue! Can you believe the difference in those two retells? It knocked my socks off.
I think it’s a perfect example here of my tagline: when you master the simple, the complex takes care of itself, because when she learned how to master some of the simpler things that were in the rubric—the summaries took care of themselves.
If you have clients who are juggling multiple language skills and dropping a few balls, consider using rubrics. Check out RubiStar4teachers and see what they have for you. I’ve made all of my rubrics public and you can search for them by putting in my email [email protected].
It’s time to wrap up this podcast. Thank you so much for listening. Come back next time to hear about tongue ties and all things related to tongue ties.
I am a therapist and entrepreneur, clinic owner, published author, and creator of speech therapy materials.
The Speech Umbrella is run by Denise Stratton, Reach out with any questions.