Khan Academy Ed Talks with Professor Thomas Guskey, PhD

Hello and welcome to Ed Talks with Khan Academy. I'm Kristin Disarro, the Chief Learning Officer at Khan Academy, and today I am looking forward to talking with Dr. Thomas Guskey about many things learning-related, but particularly grades, grading, and reporting systems.

Before we get started, I have a couple things to remind you of. First, Khan Academy is a non-profit organization, and we are able to do our work thanks to donations from folks like you. If you go to khanacademy.org/donate, you can help us be able to continue to do the work that we do. Thank you for doing that.

Second, I want to recognize some of our corporate sponsors who have particularly helped us in the COVID-19 crisis, including AT&T, General Motors, and VASL. Finally, if you want to hear this conversation or other conversations that Sal or I have had with folks in the education industry or outside of talking about education-related things, you can find us on your podcast app at Homeroom with Sal, the podcast.

So with that, I am glad to welcome Dr. Thomas Guskey. He has written over 25 award-winning books, but his latest is "Get Set Go: Creating Successful Grading and Reporting Systems." So we'll be talking about that. He's also a professor emeritus in the College of Education at the University of Kentucky, where he served as department chair, head of the educational psychology area committee, and president of the faculty council. He began his career in education as a middle school teacher and earned his doctorate at the University of Chicago under the direction of Professor Benjamin Bloom and served as an administrator in Chicago Public Schools before becoming the first director of the Center for the Improvement of Teaching and Learning at the National Research Center.

So welcome! Thank you for joining us today.

Thank you, Kristin. I've been a long admirer of the work of the Khan Academy, so this is quite an honor for me.

Thank you! Oh, it is an honor for us to have you. I'd like to get started by talking to folks a little bit about their career paths. So, as you think about how you ended up being a professor and researcher studying the things you study, how did you get there? What did you say when you were a kid that you wanted to be when you grew up, and how did that evolve?

Well, I had a very different career path than most people in education. My undergraduate major in college was actually physics and electrical engineering. But I graduated from college with my degree in that area to discover that I had no talent in that field at all. It was a terrible disappointment to my father, who is an engineer. I was at a party one night, which I did a lot at that time, talking to a friend who told me about a teaching job he had been looking for. It was in a private school middle school where they needed a seventh and eighth grade math teacher who would also coach. This just happened to be the third week of August.

So, because my father always claimed that the only reason I was hired is I passed a mirror test—they put here my nose had fogged—and so I got the job. But I started teaching middle school, and I just fell in love with it. I know a lot of people struggle with that age of children, but I thought they were terrific. So as a contingency of me being hired, I had to gain my certification, so I began taking education classes. Then I gained my certification and then, after several years, decided that I really wanted to go back to graduate school to learn how to become a better teacher.

So I began in Boston and then ended up at the University of Chicago, where my advisor and the chair of my doctoral dissertation committee was Benjamin Bloom. It was from that that I really became interested in the whole area of educational measurement and evaluation, learned about the taxonomy, learned about instruction and mastery learning, and then began working with Chicago Public Schools. I did that for several years and decided after that I really wanted to go into a university position. So I accepted a position here at the University of Kentucky in the Department of Educational School and Counseling Psychology, where I spent most of my academic career.

Fantastic! So, as I alluded to, you have written a lot of books and articles and have an impressive library. If you had to pick out some of the areas that you think you say, "Hey, I have a lot to say about this topic," what are some of those areas that you feel like are important and that you have some important things to talk about?

Well, my very first book was on implementing mastery learning. This was an extension of what I had learned with Benjamin Bloom. He was the person to develop the whole concept of mastery learning. Then, when I began working with Chicago Public Schools, we were leading a team of people there really to implement Bloom's ideas into the public school system in Chicago. So that was really my first interest, the instructional programs and the idea of using assessments as learning tools.

But my dissertation was actually in the area of change, and teacher change in particular, so that led me into a profound interest in the nature of the change process, but teacher learning and professional learning experiences for teachers. I developed a book and an evaluation model for professional learning, looking at how we can translate professional learning for educators into learning for students. Then from that, I got interested in sort of using assessments as learning tools, as we had through mastery learning. That took in the area of assessment, and then when we gather that information, we have to report it and communicate it to parents and families, and that brought me into the theory of grading. And that's really what I've been focusing on for the last several years—grading issues and how to really enhance that understanding of what we know about the grading process and how we can improve it to enhance the communication between schools and homes.

That totally makes sense, and I see how things built on each other to get to where you are today. So before we get to the grading piece, I'd love to talk a little bit more about mastery learning, since it is also a foundational piece in terms of how we think about it at Khan Academy. So for those who are watching who don't know, Dr. Benjamin Bloom is considered one of the pioneers, and then Dr. Guskey has taken many of those lessons and applied them and shared them with the world as well. How do you—when you were first talking to teachers or even now when you're talking to teachers who are just learning about mastery learning—how do you introduce it to them in terms of what are the main key points and key elements of it?

Well, I always introduce mastery learning to teachers as a way to solve the two biggest problems that teachers face. The first being the whole notion of motivation. You know, how do I motivate students to learn things that they may not consider important in their lives or don't seem as relevant? The idea is that the key to motivation and what we know seems to be the foundation of building motivation is success. I mean, students persist in activities in which they're successful, and they avoid with passion those things in which they're not successful or believe they cannot be. I mean, I always use the example of video games. I mean, the biggest selling video game of all time is Pac-Man. I mean, nothing has ever outsold Pac-Man! It's like a 17-18 billion dollar industry, and the purpose of Pac-Man is basically to gobble up dots.

Now what makes that interesting? What makes that relevant to anybody's life? Nothing! And then some people say, "Well, maybe it's behavioral. Maybe it's based on the rewards you get." But that's not true. I mean, in Pac-Man, what do you get if you gobble up all the dots? You get more dots! That's a great word! That's like saying, "Look, do these ten problems right, and I'll give you ten more to do!" But what Pac-Man did is it showed people that they could be successful at this task and capitalized on that success every time they'd improve and make the challenge a little bit greater. And so you can succeed at that.

So that's the same that we have to carry through in education. We have to find ways we can help students succeed and see that they can be successful as learners. But that ties to the second major problem, and that problem is the diversity that teachers face in group-based instruction. I mean, it would be great to think about individualized instructional programs, but we will never be able to get to a situation in our country where we have a class size of one. You know, it's always going to be group-based. And because of that diversity in there, no matter what approach a teacher chooses, it works for some; it doesn't work for others. And so teachers have to find a way to provide for the diversity of needs in a group-based environment. And those are exactly the two conditions under which Benjamin Bloom developed the ideas of mastery learning. What could we do in a group-based environment to help more students learn the excellent things we have to teach and gain the many positive benefits of that success?

So I think it's an approach as a way of solving those two critical problems that teachers face and giving them a solution to an issue with which they've always been familiar and probably have always struggled.

I continue to hear those as probably the two biggest questions that we get from teachers: How do I help differentiate for all the different levels I have in my class, and how do I motivate students?

So that absolutely makes sense in terms of the problems to be solved and where those were that it continues to be some of the issues that teachers face. When you were first starting, I don't want to make you feel old, but you've been doing this for a while.

Yes, that's true!

What were some of the conditions then, and how do you see things either changing or the same in the world in the school environments now?

Well, it certainly has changed a lot. I mean, I have the advantage of now, with social media, many of my former students—middle school students—are now my friends on Facebook, and they write to me about their lives and what's going on and the positive ways in which education has changed, especially through the use of technology. It's just remarkable the things that we're able to do today.

But the social conditions are really different, and teachers are, I think, very sensitive to that and aware of the things that our students bring to school and how we have to help them use school as an environment in which they can feel safe, and in the safety of that environment, really become successful in learning. And I think that's where Benjamin Bloom made a remarkable difference—that he said that learning in any subject area is infinite and there's no limit to what you can learn in any subject area at all. But a curriculum is finite. And when we identify a curriculum, what we do is we take that entire domain of learning in any academic discipline or subject area and we specify, "These are the things we think all students should be able to learn really well."

And then once we do that, our job as educators becomes doing everything within our power to ensure that all students, not just some, but really all, learn that excellently. And so where Bloom really differed in his orientation compared to others is that at that time when he developed these ideas in the late 1960s and early 1970s, people were talking about equality of educational opportunity and doing things to make sure that all kids had similar opportunities when they entered school.

And Bloom saw that as just an impossible sort of thing, that no matter what we did, our society was going to dictate that kids would come to school dramatically different. What we needed to do was not concern ourselves so much with the quality of opportunity, but equity in terms of the outcomes and what could we do to ensure that all students, regardless of the conditions they brought with them to school, were able to learn excellently and so they would master that curriculum, gain the benefits of learning success, and be able to carry that over to what they then would have in their lives.

That absolutely makes sense as we think about equity and equality, and those are certainly themes that we're seeing playing out in schools today, has been sailing as they were then. I think they continue to be.

As we think about that equality, I know one of the struggles that teachers have in classrooms is thinking about when students have those gaps that need to be filled in their foundational knowledge, and they also have the pressure—teachers have the pressure—of meeting those grade-level standards, and I have to cover all this material this year. How do you help teachers think about that tension?

Well, I think one of the things that I try to emphasize in my work is that we're never going to be able to overcome all those gaps that students might have when they come in. I think it's really folly to try to look back and think about all that you might have missed. What's much more productive is to look ahead and with each learning unit, try to identify the maybe two or three things that students need to have in order to succeed in this next unit. They don't—you’re not going to remedy like three years of learning deficits in the first six months of school! But what you can do is look ahead to that first instructional unit and say, "In order for the students to really be successful, to start at a level playing field, here are the two or three things they really need to learn." These are the things they really need to have—this would establish a baseline for them; this would be the prerequisite for it.

And then what we do is we teach those two or three things to a level of mastery, and then everybody's starting on that level playing field! When you get everybody starting in a level playing field, then you can guarantee success in that first unit. And typically, success in the first unit is prerequisite for success in the second unit. Although in the second unit, you might again look ahead and identify if there were any things that I would expect students to know and be able to do to really succeed in this one. And I could take a little bit of time.

And what we find is that if teachers can do that—to look ahead rather than looking back and take a little bit of time at the beginning of every instructional unit to really emphasize those prerequisites—not only does it mean their instruction can increase in its pace later, right? But it gives kids the necessary preconditions for success. So it's the idea that instead of remedying problems from the past, we look ahead to what kids need and then provide that in a really structured environment and then be able to build on that as we go along.

Alright, I rarely—I don't like to use these to plug Khan Academy things, but I just wanted to run this by you. So we have what we call "Get Ready for Grade Level" courses, which essentially your sixth-grade course—say you have a unit on decimals. The "Get Ready for" course has the "Get Ready for Decimals" unit, which has those two or three prerequisites that you need to do decimals and that we want teachers to use in that way. And we're leaving them in so we have read what you're suggesting and taken that in and built on that.

So, Kristin, one of the reasons I'm such an admirer of the Khan Academy work is because it's so thoughtfully structured, and you do such a very good job of building on exactly those ideas. Those "Get Ready" things are just absolutely on target and just precisely what we would need to do to ensure that higher level of success for our students.

Yeah, always excited to hear folks saying and talking about the things, and I can say, "Yes, we're trying to do that!" You were so thoughtful in design. Absolutely!

Let's move into talking a little bit about grading and where you've been writing and thinking about grades and grading. What would you say are the big factors in effective and accurate grading?

Well, we actually did a very large project in 2016, where the American Educational Research Association was celebrating its 100th year anniversary. And as a part of that celebration, they put out a call to the field to researchers and scholars to ask if they would take on the challenge of really reviewing all the research being conducted in these different areas of education over that 100-year period. So together with my friend Susan Brookhart, we put together an amazing team of really important scholars that had looked into the grading area and asked them to take on that challenge.

We developed our paper, which was published in the Review of Educational Research, and then actually extended that into the practical implications of that work in a book we did for ASCD called "What We Know About Grading." But what surprised us most in getting into this area was to discover how much we know, how long we've known it, and how little of that has found its way into practice today. I mean, I've grown absolutely convinced there's just not another area in all of education where the gap between our knowledge base and our practice is greater than in the area of grading.

And so we've got an amazing knowledge base that gives us guidance on how to do this well; it's just that we are so bombed by tradition that it's very difficult to challenge those traditions and implement change in this area that is so confirmed in our practices in the past that it's hard to break.

So what are some of those things that you would do differently or that we need to change in how we're thinking about grading?

Well, there are three things in particular that we need to do. Number one, we really need to have educators sit down and establish the purpose. You know, why are we doing this in the first place? Researchers have asked that question in studies that they've conducted and found that we are all over the place in why we're doing it in the first place. And that means that there's great diversity then in policies and practices because teachers believe they're doing it for different reasons. And then when we study parents or students and ask them their greatest concerns about fairness and grading, the number one problem that they always identify is the inconsistency among teachers in the same school. The only way to establish that consistency is to start by defining your purpose.

And so it means that school leaders need to sit down with their faculty and really hash out these differences and come up with a consistent purpose statement of purpose, which we can all have consensus. And then we look at our policies and practices and see if it lies with that purpose.

Then second, we need to recognize that we combine all these diverse elements when we grade students. We’ve found that there are three different types of criteria that teachers use in grading students. So we've labeled those product, process, and progress. Product criteria are sort of common administrations of learning. You don't worry about how they get there; you worry about what they've learned, what they have to do. Process criteria are actually behaviors that enable learning. If you count homework, you're grading in terms of process. If you count class participation, if you count formative assessments—those are process criteria.

And finally, we have progress. With progress, you don't worry so much about where they are but how far they've come, how much improvement they've made. So it would be possible for a student to make excellent progress but still be achieving below grade level.

Now, we know that all three of those are important. What gets us into trouble is when teachers combine all three into a single grade. Because then the grade is impossible to interpret! I mean, she got a high grade—what does that mean? It doesn't mean she learned anything. Well, maybe! Maybe it just means she tried really hard. Or if you know where she started and she's come so far during this time.

So the second thing is we need to move away from a single grade to multiple grades. We need to give students multiple grades to represent these different types of criteria. So they'll get an achievement grade, and we can still use that for GPA and all the kinds of things we are important, but maybe give a separate grade for homework or a separate grade for class participation. Now, although that sounds like a lot of extra work, what's amazing is if you go to other places around the world—if you go to Northern Europe and the Scandinavian countries, if you go to places in Asia, if you go to schools in Canada—they've done it for decades.

We are one of the few developed nations in the world that persist in this idea of combining everything into a single grade. And when I asked the Canadians with whom I worked about the extra work, they said, "Actually, it's easier than what you silly people do in the States!" I mean, we collect the same information as you; we just don't worry about combining it at the end. So all those arguments you have about weighting things, we don't deal with—we keep it separate!

The teachers love it because they find it's so much easier to explain students' performance in school when these things are separate. They find the kids pay a lot more attention to things like homework and class participation when it's not disguised in an overall sort of conglomerate grade. The parents like it because it provides a profile, and the kids’ college universities love it because all grades are carried over to the transcript.

And so you get a much more interesting, much more complete picture of a student's performance in school.

And the third we want to do is we need to get down to fewer grade categories. We need, in particular, to get rid of the percentage grade system. It's just impossible for us to get any consistency in grades with 101 discrete categories of student performance. It just doesn't make sense. And I know a lot of people think, "Well, but it's giving you a fine discrimination. It allows us to finely discriminate among students' performance." But what we have to understand is that's only true when you have direct measures.

Direct measures are things like height or weight, where if I have a finely tuned scale, then I can measure more precisely. But most of what we measure in school is indirect. And that means we give kids a series of tasks and a series of questions to answer, and then we infer from their performance what level of achievement that represents.

So we move from indirect, and when you move to indirect measures, then 101 levels just becomes impossible to do in any meaningful way. The average amount of variation among teachers with a percentage grade system looking at exactly the same body of evidence is 10 to 15 percentage points—huge!

Yeah, yeah!

And so huge! To get consistency, we need to get down to a much more reasonable number of categories of student performance. So those three things: be clear about purpose, multiple grades, fewer categories of performance.

That makes sense! We have a question from Jorge Lille on Facebook that says, "What are the assessment implications of this kind of grading? What does that reveal what we should be doing in terms of assessment practices?"

Well, it does imply that if you're going to actually evaluate or judge students' performance in these non-academic areas, then we have to have a clear rubric of what that means. If you choose collaboration as one thing you want students to be able to develop, then, number one, you're going to have to deal with the parent who comes to you and says, "You gave my kid a 2 in collaboration! How can he get a 4?"

And we need to be able to say, "This is what level 4 collaboration looks like. This is what an example of level 4 collaboration would be. Here is how I would distinguish between those." And what many schools are doing is after the teachers determine among these non-academic factors they're going to consider, they turn the job of developing rubrics over to the students.

And the students actually develop these rubrics, where they say, "Okay, how can I show the teacher that I can be collaborative? How can I show the teacher that I am responsible? How can I show the teacher that I accept responsibility and can be acting in a responsible way?" And what we find is that they are tremendously talented in being able to define that, especially when the teachers give them some guidance and direction there.

And so you make a really good point. We need to be able to specify that clearly. It's not something we can just make some judgment of and have it be out there. We need to be able to do that.

Now, when it comes to these process elements, there are three different times. There are, first of all, what we call these learning enablers—that's what I described earlier—and that would include things like homework, class participation, effort, formative assessments, etc.

The second area is what we call the social and emotional learning skills. That would include things like grit, perseverance, growth mindset— all these kinds of things we associate with social-emotional learning. That's the third area. The third area is just compliance. Compliance is: did you do what I told you had to do?

Now again, we don't know at this time among those what's most important for students. We're just starting to study that now. A new center has been established at the University of Chicago to look into this. Harvard University has established the center to look into social emotional learning skills. We do know this: you can't do it all.

And so the means of faculty again has to sit down and decide their priorities and among those things what they consider most important. Then you have to develop meaningful rubrics for it, you have to be able to communicate that rubric to all the stakeholders—parents and students in particular—and then make sure that you include that on your report card in a meaningful way and in your purpose statement that you're going to report those things separately.

That absolutely makes sense in terms of thinking about all of the pieces that need to fall into place. We are already almost at time, but I want to come—make the circle back to mastery learning. I know you also have a book, "Implementing Mastery Learning," and what that looks like. How do these grading practices and mastery learning all come together to make that implementation of mastery learning happen through these grading practices?

Well, it's certainly true this new version of our book on implementing mastery learning is actually a third edition. It was my very first book, and Benjamin Bloom actually wrote the issue forward to the book. So we're taking those ideas and adapting them now to a 21st century format.

One of the things we talk about in the book is the use of technology within mastery learning, especially as a sort of instructional advice when you have a flipped classroom or especially in corrective activities, where teachers have tried their best to teach particular concepts and skills in one way. They need other options, and the availability of instructional alternatives through technology now is so wonderful and so broad that it gives teachers much more latitude and much more flexibility in what they can provide for students.

So it's trying to interpret that in the modern context in which teachers are now working and be able to use these new tools that we have in good and productive ways. So we're hoping that this book will allow teachers to take those ideas of having all students funding excellently and the practical ways of accomplishing that, as Benjamin Bloom described, interpreted in a modern context, and giving them lots of ideas and examples from what successful teachers who are using mastery learning have found to work pretty well for them.

That sounds great! We definitely, not surprisingly, feel like technology can help teachers scale the kinds of practices around mastery learning that was difficult even for one teacher in a classroom of 30 to do, and that technology can be a tool to help them scale some of these ideas to their classroom sizes.

So I look forward to reading more!

Yeah, you're absolutely true, Kristin! In fact, that's I think one of the things that all educators have learned through the pandemic is that, you know, if you were unwilling to consider the viability of technology before, now it's a must!

You know, where—and what that did is it opened the door to all kinds of different possibilities. It certainly increased the demand for those kinds of services, but I think it made educators more sophisticated in determining quality of those as well. There's a lot out there on the internet that is not very good, and being able to distinguish those in terms of the quality programs.

I think that's the other thing. I mean, I don't mean to be an advertisement for the Khan Academy, but I think we have done a really good job of looking at quality control for these things and really trying to ensure through various devices that the kinds of things that you put out there are things that are going to help students in ways that are meaningful to them. I think that's a tremendous asset.

Yeah, that is fundamentally what we are about! It is already past time for us to close up. Thank you so much for joining us today and for sharing your views about mastery learning and grading with the audience today!

Kristin, it's been my pleasure to be with you. Thank you for this wonderful honor. I do appreciate it, and I hope that we'll have other conversations again.

Absolutely! And thanks to all of you listening for joining us. Do stay tuned—tomorrow we have Chase Nortongren, who will be joining us from NWEA to talk about goal-setting, motivation, and how to start and reset learning routines, which are good things to think about as we head into summer. Thank you all, and we will see you next time!