Effective Instruction: Ongoing Assessment

Effective Instruction: Ongoing Assessment Disabilities Dialogue

Tamarack O'Donnell: Hello, everyone. Thank you so much for being with us today and taking time out of your very busy schedules to learn about ongoing assessment and how it benefits children with special needs and teachers. We have a whole team of people here with you today onboard making sure we provide you the best support that we can. I'm Tamarack O'Donnell; I go by Tam.

Kristin Ainslie: I'm Kristin Ainslie.

Tamarack: And we're curriculum specialists joining you today from the National Center for Quality Teaching and Learning headquarters.

Kristin: That's right.

Tamarack: We also have Susan Stewart joining us from California. Hi, Susan. Susan Stewart: Hi, everyone.

Tamarack: Dawn Williams is online from Chicago. And our guest speaker, of course, Dr. Mary McLean, is with us from Florida.

Kristin: So even though this content is primarily geared towards disabilities coordinators to support your work with teachers and children with disabilities, it's a reminder that all are welcome who are interested in this topic today. So welcome. Even if you're not a disabilities coordinator, we're happy you're here.

Dawn Williams: Amanda Bryans, who joins us on these webinars, and she's from the Office of Head Start. She's one of the federal project officers there, and we're always very happy to have her with us. She can answer any questions that might be more policy related. And if she doesn't get to it here, then she can get to it afterwards, but she's just really supportive of these and is very glad that we're doing them here for disabilities coordinators.

Kristin: Yes, it is our pleasure to introduce Dr. Mary McLean. She is professor at University of Florida in early childhood studies. She chairs the DEC Recommended Practices Commission. She works as a consultant for NCQTL; we are thrilled about that. And she is an expert on ongoing assessment in the field of early childhood, and we are so fortunate that we have you joining us today. Hi, Mary.

Mary McLean: Hello. Hi, I'm delighted to be here.

Tamarack: Thank you so much. So, Mary, just to get this kind of kicked off, can you set the stage for us and just tell us, what is ongoing assessment, and why is it so important?

Mary: Sure. Ongoing assessment is a system of assessment that involves continual observation and documentation in typical environments, in routines and activities that children typically engage in, and

has a very important purpose. And the purpose of ongoing assessment is to inform teaching and instruction. So it is a system that helps teachers teach.

Kristin: That's great. So thinking about that and then thinking about children with special needs, Mary, what is unique about collecting data and assessment for children with special needs?

Mary: Well, the best part is that in Head Start, that assessment of children with special needs can be done in the typical environment with peers that don't have special needs. And so what is special about it is that of course children who have an IEP come with that IEP. The IEP has goals that have been identified for each child, unique to each child, and those goals guide intervention. So the teaching that we do for children with disabilities are based on the goals, but there are usually smaller steps than we are using with children that don't have disabilities, more repetition, and we'll be following them closer.

Tamarack: Great. So you mentioned thinking about a goal in smaller steps for a child with disabilities. How would a disabilities coordinator help a teacher make sense of a really broad annual goal that's written on a child's IEP?

Mary: Right. I think that's a great question. Because they're written for 12 months from the time that they're written, they tend to be large so that you don't have lots and lots of goals, but what you have is rather large chunks of behavior. And it's helpful to break them down into steps. So you can see on the slide that we've got an annual goal that we're going to be working for, but what we need to do is break it down into smaller steps, which we call learning targets. So as you can see, there's the first learning target that we would decide. And that first learning target is very important, because it needs to be just a little bit more advanced than where the child is right now. So we'll work on a learning target that's close to where the child is now, but more advanced. When they reach that first learning target, then we'll move to the next learning target. And then when they reach that target, we'll move to the next one. When they reach that one, we'll move to the next one, and then we will have accomplished the annual goal. So it's a means of, again, breaking down a goal into smaller steps that are more manageable, easier to teach.

Tamarack: And, Mary, are there different ways that you can break a goal down? I'm thinking, you're talking about breaking a skill down into different steps. What are the different ways that you could break a goal down?

Mary: Right. One thing is we can -- we can have smaller amounts. I think we had a slide that showed that. We can provide help. And we can do it step by step. If a child has a task that can be broken into steps -- for instance, brushing teeth can be broken down into picking up the toothbrush, picking up the toothpaste, and so forth. So there are a variety of ways that we can actually break down a goal. What's most important is that we have a logical approach and that that approach fits the needs of the child that we're working with. So we want to spend some time thinking about this and being careful to have a goal broken down in a way that, again, makes the most sense in relation to the child that we're teaching.

Tamarack: Right. So do you have an example you could share with us?

Mary: Sure. On the screen, what you see is "Carly will name..." and then in the target is "the numerals 1 to 5." So we're interested in teaching her to name all five of those numerals, but we're going to start with just the numeral 1, because right now she doesn't name any of the numerals -- the numbers -- consistently. So we're going to start just with 1. That would be until she does that well. Then we'll add

    So then she'll be working on the numerals 1 and 2. Then we'll add 3, so she'll be working on 1, 2, and

    And then we'll add 4, so she'll be working on 1, 2, 3, and 4. And then finally we'll add 5, and we will have accomplished teaching her all of the numerals, but probably in a way that would make it easier for her to learn, because instead of giving her all five to begin with, we've broken it down and turned it into smaller steps for her.

Tamarack: And I imagine, Mary, that when the teacher or the disabilities coordinator who's helping a teacher think about this, the place that you start is where the child's current level of ability is.

Mary: Right, right, just a little more advanced than what the child is doing right now, right.

Tamarack: So that example you just gave us looked like you broke the skill down into different steps. Do you have another example you could share with us?

Mary: Sure. So what you see on the slide right now is Sonya will request objects from peers using twoword combinations. So, for example, maybe we want her to say "want baby" or "more blocks" when she's interacting with peers. And one useful way to do this might be to actually break down the task according to the type of prompts that we'll be giving her. So, for instance, you see the first step might be -- and we're assuming that she doesn't use two-word combinations for requests very consistently at this point. So we might provide her a model at first, and we'll actually model for her "want baby" in that situation where it makes sense, and she would then imitate it, and then she would get the baby. So when she does that consistently with a model, then we might move to the next step, which is, when provided with a verbal prompt, which is a little bit less of a prompt, and that might be that we just say the first word for her, "want..." and expect her to come across with both of the words in combination. And then, once she's doing that with just a verbal prompt, then we might not use prompts at all. So then we're expecting her to be able to produce those two-word combinations without assistance.

Kristin: That's really -- I love how those models of broken down twice, different ways to break down a goal. So I'm thinking, Mary, how many Head Start teachers have really seen a real IEP, real IEP goals, and would really understand what to do with all that paperwork? I think this is a great opportunity for disabilities coordinators to really do some teaching around the actual IEP, right? So how can a disabilities coordinator help a teacher to kind of figure out how to measure progress?

Mary: Right. Well, I think that the way that the goal is broken down is hugely important. And one of the things that's usually stressed is the learning targets that you develop when you're breaking down the goal need to be observable and measurable. And observable means that you can see it and you can hear it. For example, if I say I am going to -- the learning target is going to be that the child understands that four is more than two, I can't really see that, right, and so that makes it hard to measure because I can't see it. Or maybe we want to hear what the child is saying so that it's observable. So we focus on making sure that the learning targets are written in a way that they can be seen or heard and that

they're measurable; we can count them or we can time them, but we've got some way of knowing whether or not the child actually did the learning target. A classic example here is "pays attention." Many times I've seen goals for children written as "the child will pay attention for 15 minutes" or "the child will pay attention for 20 minutes." But it's really hard -- you can't see that or observe it. We don't know if the child's paying attention. And so it's terms like that that might in fact be in the goal that we need to translate into something we can see or hear and that is measurable, so we'll know, we'll know when the child's got it and when we should be able to then move on to the next step.

Tamarack: Right. It sounds so logical and obvious, and yet I'm not sure how many times a goal is actually written that concretely where a teacher could simply take the goal and know exactly how to implement that in the classroom. So this seems like another really important step that a disabilities coordinator could help a teacher with is, okay, now we've broken it down, how do you understand how this goal is actually going to be taught in the classroom, and what exact behaviors are you looking for to demonstrate progress?

Kristin: Have that conversation about meeting the goal. Mary: Well, I think -- go ahead.

Tamarack: Well, I was just wondering if you had another example of taking a goal from a more broad idea to something more specific.

Mary: I do. And something that you said is really important, because we need to think about the goal in relation to this particular classroom that we're in as well. So we'll want to be thinking about what things are happening: what activities are in the classroom, what routines are in the classroom. And that lets us really take the goal and divide it up in a way that will make it easiest for teachers to see it, to actually teach it, and certainly to measure it. So here's an example: Neveah will demonstrate functional use of books. That's very common for us to talk about the functional use of books, but what does it mean? How do you see it? How do you hear it? How do you measure it? So we've broken it down here, and we've actually changed the words. We've decided that we'll turn it into three steps.

The first is that Neveah will hold books correctly, so not upside down, but right side up. And then, when she's doing that, then we'll work on holding the books correctly and turning the pages one at a time. And then when she's done that, we'll add, she'll hold the book correctly, turn the pages one at a time, and touch or verbalize at pictures that are in the book. And the combination of those three steps will essentially give us the functional use of books that's in her annual goal.

Kristin: That's great. So it's really okay to sort of translate it, keeping the meaning of the goal, into something that works for your routines in your classroom. I love that. That's great.

Mary: And I think disabilities coordinators can be very, very helpful in that process. Kristin: Absolutely.

Tamarack: Right, I'm sure teachers could use some insight on --

Kristin: I know I could have, yeah. That's great.

Tamarack: Okay, so, Mary, we've broken our goal into smaller steps. We're teaching behaviors and observing behaviors that can be measured. How do we go about collecting data on this?

Mary: Well, the good news is that Head Start already has a system in place for ongoing assessment for all children. And so if you're implementing that in your classroom, then you're halfway there at least, probably more, in terms of what you need to do to gather the data that you'll need for children with IEPs. So the -- what we have in place already includes a system that -- and we have training materials, as you know, available on ECLKC, the 15-minute in-services relative to ongoing

assessment. And those materials look at the use of checklists, the use of work samples, the use of videos and anecdotal records, and also gathering information from families. All of those are also important for children with IEPs. But we might also want to think about some other ways of collecting data, and that would include frequency, so counting the number of times that something happens. Or accuracy, which is the percent correct. How many times did the child do a particular behavior correctly? Or what kind of support did we need to provide, recording the level of support that we needed to provide for a child. So there's at least three more possibilities that can be embedded within the day and within the existing assessment system that most classrooms already have in place. So we can look a little bit more closely, if you like, at these three. The frequency count, that just means how often a behavior occurs. We're literally counting throughout the day when we see a behavior. And usually we do that because we want to either increase that behavior or we maybe want to decrease that behavior. So, for example, we might be looking at the number of times that a child gets out of his seat when he's supposed to be sitting down. And we can count that. We can see that and we can count it.

Tamarack: A tally.

Mary: Right, just tally, a simple tally. The most important thing to remember is you need to write that down because -- or somehow record that number, because you will not remember it. We are humans. Teachers are humans.

Tamarack: And trying to keep track of so many different things throughout the day.

Mary: That's right, that's right. So the secret to this, in addition to knowing what to look for and counting it, is to have a system that works for the teacher of recording it, recording it and saving it, so that then I have the data to go back to, to help me inform my teaching.

Tamarack: I remember using golf counters as a teacher. Kristin: Absolutely, absolutely.

Tamarack: All sorts of different creative things. So do you have an example of a goal that's written where you might be taking frequency data?

Mary: Yes, there's an example. And the example is Sam will raise his hand during classroom instruction to respond to a group question or initiate a question or request. And then we've added something that's very important to this whole system, and that is a criterion. And the criterion is, you can see there, says from zero times -- so right now he's not doing it at all; he's not raising his hand at all -- to a minimum of three times, and that would be in an hour. So we want to see him raise his hand at least three times in an hour for the first step that we're working on. And so you can see below there, we've got some data across a couple of weeks. And I think the question -- I have a question, and that is, has Sam met this criterion?

Tamarack: Right. Looking like --

Kristin: A lot of people think maybe he has met criteria. Yeah, what do you think, Mary? Do you think he has met it?

Mary: Not quite, not quite. But you can certainly see how he seems to be improving, right? He went from zero, none the first day, none the second day, and then he starts getting ones. And when -- we only have data through October 12th, and we can see that he did it twice then. So we're seeing improvement, but we set our criterion at three, so he's not quite there yet. We probably want to keep going a little bit longer and keep teaching and keep focusing on that to see if we can get him to three times.

Kristin: Okay, that sounds good.

Tamarack: Thanks to all of you listeners who participated in that. So we've talked about frequency, Mary. What about -- I know accuracy was another example that you gave.

Mary: Right, right. So remember, accuracy is the number of correct over the number that were not correct -- or I'm sorry, over the total number. And we need accuracy when it's important that the child be responding correctly. So some examples would be we want to be sure that the child is labeling things correctly. Or if we're trying to teach him to communicate no, that he's shaking his head for no. Or if we're teaching colors, that he's pointing to the correct color when we ask him to point to red or point to blue. So accuracy lets us know, it's not just how many times did he do it correctly, but how many times out of all the times that we asked did he do it correctly. So we end up with a percentage, a percentage correct.

Tamarack: Right. So do you have an example that you could share again with us, Mary?

Mary: I do. I have an example about Miguel. Miguel will jump up by pushing off the ground with both feet at the same time. And so just to clarify that, I'm sure you've seen children who can't yet jump, and so one leg goes up and then the other leg goes up. But we're working on motor skills with this child, and so we want him to get both feet off the ground. And we have decided that when he can do that for 80 percent of the times that he jumps, for two days, then we'll say he's got this step, and now we can move on to whatever comes next. And so right below there, you see the data that's been collected... for Miguel.

Tamarack: Mary, can I ask you something quickly?

Mary: Sure, please.

Tamarack: I'm looking at the 80 percent, and I'm wondering if a teacher could maybe benefit from somebody stepping in to help them think about how many opportunities you want to provide in order to reach that 80 percent. For example, it's realistic that when we go out to recess that he'll have five opportunities to practice this skill. We want to see that at least four of those times it's accurate.

Mary: Right. I think that's a very important point. The criteria that you set should not only make you think when he does it this well, then I think he's got it, and then he's ready to move on, but it also needs to mesh with what opportunities you'll have during the day to see the behavior. Because, remember, this is observation in typical routines and activities. And so we'll want to think about exactly what you said. How many times does it seem like we would be able to see this? And this teacher has decided that five times a day she'll be able to observe when he jumps, whether he jumps with both feet off the ground.

Tamarack: Okay. So with this example, we would love you, as our listeners, to show us again a show of hands if you think that Miguel has mastered this skill and is ready to move on. It looks like they're looking for him to do this on four out of five occasions across two days. So go ahead and raise your hand only if you think that Miguel has mastered this skill and is ready to move on. I'm seeing a lot of hands.

Kristin: A lot of hands.

Tamarack: So, Mary, what do you think? A lot of people think that the skill has been mastered. Kristin: Right, looking at May 11th and May 12th there.

Mary: Yes, that's right. Those people are correct. So if you look at May 11th, he's got four out of five, which is 80 percent. And by May 12th, he's got five out of five. So, again, what we see here is nice progress for this child, which is not only great that he's learning the skill, but it's great for the teacher, because she can see how well he's doing and that he's, you know, he's there. He's gotten to the point that we thought that's good enough, and now we can move on to the next target for him.

Kristin: Right. Miguel's doing well, yeah.

Tamarack: So we've covered frequency and accuracy. How about the level of prompt that you might provide?

Mary: Yep, so we can record data that way as well. And what you're doing here is recording the prompt that was used or was needed to get the correct response from the child, or the desired response from the child. Over time, what you want, of course, you'll remember from the example we had with the learning target about using two words, you want to be providing less and less support. But sometimes when you first teach a skill to a child, you need to start with a good deal of support, and then we fade that support away over time as the child gets better and better. It's actually a very nice way of teaching a number of different skills for children. So we want to demonstrate decreasing need for support.

And here are examples of the possible types of support that we might provide. The full physical prompt, we call it, is hand over hand. So sometimes when we're first teaching a skill -- and I think probably tooth brushing might be a good example here again. You've got your hand over the child's hand and you're actually moving them through the movements that they need to make to brush teeth. Partial physical prompt means we're not hand over hand anymore, but we may have a hand on the elbow. We're somehow still guiding the child physically to be able to produce a particular behavior.

And then less than that -- so full physical prompt is the most amount of prompting that you can provide, verbal prompt is a little bit less than full physical or partial physical, and now we're giving the child a cue of some kind of what they need to do as a prompt. We might use a gesture as a prompt, and that would mean maybe we point to what it is the child should be pointing to or doing or saying something about. And then a picture prompt. So I think probably a number of people have used picture systems with children, and we might give a child a picture prompt that lets them know what comes next, what they need to do, and so forth. So that's what we would record. We would record the FP or the PP or the V or the G or the P for picture as a means of knowing how much prompting did I have to do for this particular child to get the behavior that's desired.

Kristin: That's great. That's nice to see it broken down like that.

Tamarack: Yeah. And all things that teachers are probably already doing in the classroom, now it's just a matter of deciding intentionally what level to use and actually recording it so that you're keeping track of whether or not you're fading that out.

Kristin: That's right, a teacher may easily be able to say, well, yes, he or she can do this with a little bit of help, and then we're just going to put maybe a different word to that and kind of specify what kind of help that was a little bit more.

Tamarack: Break it down.

Mary: Yeah, and it helps to make instruction systematic. So the child -- many children can learn better that way, when instruction is systematic, as opposed to sometimes I do one thing, and other times I do another thing. So that would be very helpful.

Kristin: Good.

Tamarack: So how about another example.

Mary: Okay. Jeron will respond to peer requests for items at snack time with only a verbal prompt from the teacher. And you'll see the criterion here is four out of five opportunities for three days. So this time we've said we need to really be sure that he has this skill, and we think five occurrences make sense in terms of what we can see, so we're going to say four out of five opportunities for three days.

Kristin: With the verbal prompts, right. Okay, so we're going to have another little pop quiz here for you. How many of you think that Jeron has met criteria for needing just a verbal prompt from the teacher on four out of five opportunities for three days? Go ahead and raise your hand.

Tamarack: A quarter?

Kristin: Yeah, about a third or a quarter. What do you think, Mary? How's Jeron doing?

Mary: Well, he's doing really well. Again, you can look at the difference there. It's quite good. He went from full physical prompt on August 10th, and now, by August 20th and 21st, he's got four out of five for both of those days. He only had three out of four for the 19th, so this teacher might want to decide, let's try another day and see if he can get four out of five, and then we would have across three days.

Kristin: Right, this is exciting, because he's made huge progress. Tamarack: Getting great information. You just need that final push.

Kristin: Exactly. I think that's just so helpful for teachers and for disabilities coordinators to help -- how do you think that -- how can disabilities coordinators really help teachers find a data collection system that works for their team and for them in the classroom?

Mary: Yeah. Well, one of the nice things, again, is that we're -- we're going to be embedding this in ongoing activities. And the answer of how do I find a system, I think, is that probably everybody will do it just a little bit differently. What works for one teacher won't necessarily work for another teacher or won't work as well for another teacher, and others who are in the classroom as well. So we're going to look across activities, but we're going to search for something that makes the most sense for the people who are in the classroom and actually collecting the data.

Kristin: Right, for those who are actually doing it, right, what works for them might be different.

Mary: Right. But happily, we have, you will remember from the last webinar, the discussion about the teaching loop. And the teaching loop is extremely helpful in helping us think about data collection because -- and we'll walk through the teaching loop just to remind you about that. It starts with providing a cue for the child. And then if we don't get the behavior, we're going to give some assistance until we get the response that we're looking for. And then we're going to give feedback.

We're going to give the child -- we're going to reinforce that behavior by giving them some feedback, telling them, "That was great, you did a good job," and that's a loop. And one of the things that was mentioned in the last webinar that's really true, is this is really fast. It doesn't take much time. Each time you do a loop, that's one behavior. So you will simply make a tally or write down a P or a G or whatever the level of prompt was. But if you're already using teaching loops, then to collect data on that is simply a matter of being able to figure out how can I tally or record this behavior so that I'll have it and I'll be able to look at it and make sense out of it in relation to the teaching that I'm doing.

Kristin: Great, great.

Tamarack: All right, so can you maybe talk about a few different examples maybe that you've seen in your work with teachers, systems that might have been helpful?

Mary: Sure. So there's -- I think we mentioned this just briefly below -- or earlier. There's a lot of different ways that this can be done, and what you see before you are just some of those examples. Be

creative is a great -- is a great suggestion because, again, it needs to work. What we know about the system that you set up is if it doesn't work, you won't use it, the teacher won't use it. So it needs to be a system that -- we need to find the system that works for us. And so you see examples there. You see a clipboard; you see index cards on a ring. You see labels, and we'll look at labels in the next slide too. You see notebooks -- the notebooks are really important because you've got to put the data somewhere so that you'll have it. And then we've got some tape, and you can put tape on your clothes and tally on your clothes -- on the tape on your clothes, if you like.

Tamarack: That's one that I actually do quite a bit.

Kristin: Yeah, I remember doing it, too. I just put it right on my pants and carried a pen and just --

Mary: Right. And then you could take it right off of there and put it in the notebook so that you have it. So these ideas are all easy. It's a matter of what will work in the classroom.

Dawn: Hey, Mary, this is Dawn. We had a couple questions I wanted to jump in and ask you from the audience. So one question was, how do you suggest therapists or special education teachers working with a child in a Head Start classroom be involved in the assessment process?

Mary: Well, that's a great question, and we're going to see a little bit more about this in a minute. But as other people come into your -- so we know that it needs to work for the people who are in the classroom, the teacher and assistant teacher, but also, as a specialist comes in, a speech language pathologist or maybe even a physical therapist, they can also be involved in the same system. And so we want to make it easy for everybody to be able to observe the targets and simply record what the child has done and put them somewhere. But, yeah, I think you'll very much want to involve other people who are working with the child. They'll want to know what the child is working on and how that progress is going. So it's information for the whole team to consider as we have particular targets to work on.

Tamarack: Right, and the more data, the better, right? The more information that you can get about how the child is demonstrating a skill and who they're demonstrating it with, and is it generalizing, is probably --

Mary: Exactly.

Kristin: And how great for the special education teacher to be able to say, when the physical therapist comes in once a week or once every two weeks, I can't remember the little boy, I think it was Miguel, he jumped with two feet three days in a row, and I just made a tally mark, and giving that - providing that for the specialist who comes in will help benefit...

Mary: Right. So not only are they then aware of what you're working on, which is of course very important, but they can tally, too. They can help you keep track of the behavior when they see it as well.

Tamarack: Dawn, was there another question you wanted to ask before we move on? I think we're good.

Dawn: Yeah, actually, this was in regards to earlier when we were talking about frequency and the example of the little boy that was raising his hand in the group and you wanted him to answer questions three times in one hour. So the question was, would you really be working one hour with the child in a group to be able to check that kind of data?

Mary: Right. That seems like a long time, doesn't it? But it may be more than one activity. Again, that teacher wrote that for her classroom, what made sense in her classroom. So maybe it was -- maybe it was center time and a variety of activities. But that's a good question. Might not work in some classrooms, but it might work in others.

Kristin: Or getting creative over that hour over a few different activities. Maybe we're going to have all the children raise their hand and ask a question as they leave the circle to go to another activity, or maybe there's a transition activity that could be worked into that hour. Raise your hand if you have a pet, that kind of thing.

Tamarack: Yeah, raise your hand if you want some glitter to use on your art project.

Mary: Sure. Good examples of embedding instruction, right, that is important for that child.

Tamarack: Great. So can you kind of go into a little more detail about an easy system that might work in a classroom?

Mary: Sure. And this is one that I think is really easy and a good idea. So these are just labels, and you can see that the teacher has labeled them already with the child's name and the particular area that she's looking at. You could put particular targets on there as well, learning targets. They could be on a clipboard. And so during the day, then you simply write when you see -- when you have something to put down, data to put down. But then again the stickum comes right off and it can go right into a child's file or folder so that you've got the data organized. There also are -- there are now apps for iPhones and tablets that are very promising. There's one called Tally. Actually, there's a lot of them that have "tally" in their name. The one that was recommended to me is just called Tally. It costs 99 cents. And you can put it on your iPhone, you can put it on your tablet. When you touch the tablet, it records one. So each time you touch it, it will record one of something. Then when you swipe it, it will add them up. And apparently you can then download, and you could even email them to people if you want to from the app. So I think we're going to see probably more of that kind of thing in the future. But how easy to carry around your phone, right, or a tablet to be able to tally as well.

Tamarack: Which most people are anyways, especially if you're collecting video in the classroom as a means of gathering information about a child. I think there's many ways that technology nowadays can be used to inform whether or not a child is making progress. There's audio -- you can record audio. So if the child -- if you're working on the two-word "move this," you can record it, and then you're not really having to keep track of it in your mind or even write it down, but then at the end of the day, if you go back and view it, you have it right there.

Kristin: That's right, and I'm thinking I'd be much more on the low-tech side of things. I like this label, where some teachers might be a little more tech-savvy. But what about -- thinking about these labels, thinking about all of these systems, it's a lot to keep track of. How do you recommend that a disabilities coordinator can help teachers plan for this and plan for their systems?

Mary: Well, that's where the activity matrix comes in and helps us out. I think some people probably are already familiar with the activity matrix, which is a way of -- in this case, we've got three children that we are -- we have, as you see, some targeted, some targets for each of those three children. And then you'll see on the left-hand side we've got the routine for the day for the classroom. And this is a very simple way of helping us plan when we're going to -- what are good opportunities to work on particular targets, right? So talking about people or objects in view is easy at free play. You can put more than one time for a particular target. So we see "labels an emotion." The teacher has decided there are three times that that might be useful to focus on. But you can also keep your data right on the activity matrix. And the activity matrix can be -- oh, I've seen some phenomenally creative ways of having an activity matrix. It can be a poster in the classroom with stickums on it, so it makes it really easy to change. And if you're putting data on your activity matrix cells, then you just take the stickum off and put it in the folder, and you have transferred your data. So a really nice way that a lot of people can find will help them. I had a student once tell me, she was actually a teacher, but she was taking a course from me and said that learning about the activity matrix was worth the tuition she had paid for the course, because it -- she found it to be so useful in her --

Tamarack: It's everything in one place, right? Mary: Yep.

Kristin: I love seeing those tally marks on the matrix. I think that's ingenious.

Tamarack: And I've actually seen classrooms where they have it up on a whiteboard and then you can take a photo of it at the end of the day. So just everyone's putting marks up. It's quite large, so everyone can see it. Obviously respecting the privacy of the children's names. And then even with the sticky labels that you showed, the address labels, those can be printed off, for those that are a little more tech-savvy, you can print off the matrix eventually on those stickies and then pull them over.

Kristin: There's so many ways.

Mary: So I'm thinking we should develop an app that is a matrix, right? Wouldn't that be the next -- Kristin: Yes!

Tamarack: Yes!

Mary: The next thing? Right. Well, and this matrix, this just shows you, again, you can put whatever you need on your matrix. This has when assessment will be on what targets for all children in the second column in, and then also for Matthew and for Carla, who are children with IEPs. So we've got some additional things that we're recording for them, and they have their own column on the matrix. But this matrix also -- like this matrix works for a whole week, and you can see that sometimes the days are specified. Tuesday and Thursday are good for identifying printed names. And it even specifies who will be responsible for recording that data and what kind of data will it be. So there really is -- it's a tool that can work in a lot of different ways.

Tamarack: So I'm hearing a theme, Mary, as we're talking about ongoing assessment, and the word "individualizing" keeps coming to mind. The same thing that we want to do for children is also important as a disabilities coordinator to help a teacher individualize how they are going to be able to take data in their classroom and make it work and motivate -- motivational to actually do it in the classroom.

Mary: Right. It has to be accessible. So the information, not only do we have to write it down somewhere, but we have to make it relatively easily accessible to us to inform teaching. So if I take data and then I can't find it, or I take it and I don't know which is which child's data, that's not going to be helpful. Because at the end of the day, we've got a relatively small amount of time to look at the data and make some important decisions about it. And so organization is key, and that's what this slide just shows is a variety of ways that will help teachers be organized to make that data accessible to them, because we know you won't remember it.

Tamarack: Right, and you've put in all this effort to get a system in place, to get all your team members on board. You're taking this lovely data, you have this great information, but it's not useful unless you're actually looking at it.

Mary: Right, right. And I think the next slide shows us a further step, and that is, how can we look at our data that probably is in the form of tally marks and how can we visualize it in a way that will help us understand what's happening? Now, we did pretty well with the three that we looked at where we just had tallies and we even had the names of some prompts, but visualizing it can be helpful over time. If we had three weeks or four weeks, it would be harder probably to look at. So there's different ways to visualize. What you see there is a scatter plot. We also have line graphs. And line graphs are my favorite. We're going to look at a few more line graphs. But I also want to say that line graphs are really easy to make on Excel. Very, very simple. It's like three or four steps, and you've got a line graph that shows you whether the child is increasing in relation to the skill or whether the behavior is decreasing. And line graphs make sense to most people. Almost everybody understands what that means when that line is going up or what it means when the line is going down. So you can share with people, and it gives us an, "Oh, look," this is what's happening relative to this particular skill. And there's a bar graph, of course, and pie graph, I think, pie chart. So there's a variety of ways that we can look at the data. And it's helpful for us. So we're going to look at some data here, I think. You have to use data. It doesn't make any sense to have a system that you've created in your classroom to be gathering the data unless you make it work for you and the child in terms of making the most effective instruction that you can get.

Kristin: Right, so the data's going to help you make your decision, right?

Tamarack: This is what it's all about, right? This is why we're collecting this information.

Kristin: Right, am I going to move forward, are we going to stay where we are, or do we need to maybe even reteach some, go back a little bit? So those are the conclusions teachers are making all the time.

Mary: Right. And it's very helpful to have something in writing that shows you how the child did, and so that you're not just trying to remember whether -- we have research that shows us that we can't remember. And so we need to have a system of looking at the data that can help us make these important decisions about how do we adjust our teaching. Because at the end of the day, that's what it's all about. It's about having the most effective teaching that we can have because we've got data guiding us in the decisions that we're making. So there's basically three big decisions that occur. One is that the child is making progress, and probably we're going to decide that's good, and so we're going to keep doing what we're doing because it's having an impact. The child might not be making progress, and then we need to change something. That's equally as important. If the child is not making progress, we need to do something different so that we can get progress. And we may have a situation where the goal has been accomplished, and then, this is equally as important, we have to move on to something else. It's good that the child made -- achieved the goal, but what's next? We need to keep going in our instruction.

Kristin: That's right, absolutely, absolutely. So I'm just looking at our time here and thinking about, Mary, let's think about we're going to just pop over to the slide that is... let's see here, thinking about... Okay, the numbers, am I doing it, is this working, and where do we go from here?

Mary: Okay. What you're seeing here is basically what we just talked about, the three possibilities. This one, if we can just pause a minute here, this child is not making progress. And that's very important, too. If you look at the next slide, it relates to what do we do in this case? And we need to change, we need to change what we're doing if the child is not making progress. Now, this is the least favorite that people like. They like the child that's making progress and the child that's accomplished the goal. This is harder, but it's also very, very important. If a child isn't making progress, we need to make some changes until we see that progress coming. Most important question here is have we been implementing the instruction we said we were going to implement? And sometimes we haven't been for one reason or another. So if the child isn't making progress, the first thing to check is whether or not we actually were doing what we planned to do to help that child learn the skill.

Tamarack: Right. Were we teaching?

Kristin: Were we teaching? Yeah. And what level of support were we giving?

Mary: Right. If we have been implementing the instruction, we might need -- and he's not making progress, we might need to change the learning target. Maybe it's too advanced. Maybe it needs to be a little bit different somehow. And this isn't fun -- as fun either. Teachers -- I had a teacher tell me once, "But if I change it, that means I didn't get it right the first time." And it's okay if you didn't get it right the first time. The important thing is you realized that, and now we're going to try to get it right. So changing is not a bad thing.

Kristin: That's right, that's right. Very experienced teachers often have a child who doesn't make progress. It's not a reflection on your teaching. But making those decisions and changing what you're doing, that's really the sign of a teacher who's going to be able to move that child forward.

Tamarack: The teachers that are most dedicated to child's progress that are taking the time to document this progress, and if it's not happening, make those adjustments. So what are the three big questions we want to be asking ourselves as teachers?

Mary: So the three questions, when you have a situation where the child is not making progress, is am I doing it, am I implementing the teaching that I planned to do? If yes, then is it working? If the child is making progress, that's great. If not, then do I need to make changes and to revise the instructional plan for that child?

Kristin: Okay, so those three questions, I see that posted -- actually, I'm visualizing those questions could be posted in a classroom just to kind of remind us: are we doing it? Is the child making progress? And what changes need to be made? So I love that.

Tamarack: So, Mary, you've shared so much information with us today. You showed us how to step back and look at the goal in steps and take data, summarize it, look at it, and then use it to make instructional decisions so that you're making an impact on a child's learning.

Kristin: Yeah, it's been just a wealth of information. What is a resource that you want to -- do you have any resources that you want to share or...?

Mary: Well, I think there's a number of resources that are up on ECLKC that teachers should -- and disabilities coordinators will want to look at relative to just about every aspect of what we've talked about. I also wanted to give credit for some of what's in this to -- this presentation to the Embedded Instruction for Early Learning Project here at the University of Florida, who carried out a research study, a three-year research study, on these very things not too long ago.

Tamarack: Great. Well, I'm sure that disabilities coordinators are walking away with some really useful ideas and ways to bring data collection into the classroom with the teachers they're supporting. Mary, thank you so much for your time.

Mary: Thank you. This was fun, a lot of fun.

Tamarack: Great. Thanks, Mary. And join us next month, when we'll be talking about working with dual language learners, children with disabilities who also are dual language learners. Thanks for joining us today.

Kristin: Thanks, everyone. Bye-bye.