My colleague and I have been using standards-based assessment and reporting (SBAR) (a.k.a. standards based grading (SBG)) and a mastery learning methodology for the past three years. We have been very pleased with the results and have continued to evaluate and improve our methodology each year. However, this semester, it appears that the students aren’t drinking the kool-aid. I need some advice.
We are planning on administering an anonymous survey as well as having a class discussion on the topic of SBAR, learning, and assessment. I also plan to talk privately with various students. However, I wanted to solicit the advice from the larger SBAR community first since I expect that will shape the survey and direction of the conversation.
The problems this semester are that students are doing worse on initial assessments and the quality of work has deteriorated. To be fair, this doesn’t apply to every single student, but it does appear to be an overall trend. While I’m not sure of the reasons behind this change and I hope that the survey, class discussion, and individual discussions provide some clarity, I have a hypothesis. I believe that there has been a change in attitude this semester compared to the previous five.
In past semesters, many students would have the following attitude towards class: learn the material as well as possible throughout the unit, do some of the homework, and try to master every standard on the initial exam. If they didn’t master a few standards, they would complete extra practice and take advantage of a reassessment outside of class. These reassessments were best to be avoided, however, since they required extra work and time outside of class.
This semester, I believe many students have the following attitude towards class: try to learn the material only based on in-class activities, do nothing outside of class, and attempt the initial exam. They expect to have to reassess every standard; so, any that they happen to master on the initial exam is considered a fortunate bonus. They then prepare the required extra practice and take advantage of the reassessment outside of class. In addition, since they only have to master a standard, do the bare minimum amount of work or quality of work to meet that expectation.
I still believe our methodology is philosophically sound. However, I fear that this deferred effort approach will result in less understanding and less retention. In addition, this bare minimum approach leads to sloppy and careless work and poor habits. This is not okay regardless of how sound the philosophy is.
I’m considering a couple of changes. One, abandon the mastery system for a 1-5 scale like what I use with my AP-level class. While this allows for more differentiation in terms of quality of student work and depth of understanding, it feels like such a huge step backward in terms of trying to de-emphasize grades. Two, adopt a cap for reassessments. My school is pushing an 80% cap for reassessments. Only students who score less than 80% are eligible to reassess. In addition, the maximum score on a reassessment will be an 80%. While this may motivate students to develop their understanding and practice before the initial assessment, it seems to contradict the very foundation of SBAR.
Perhaps even more importantly that understanding the change in attitude, I’m not sure what has precipitated this change. What is different this semester compared to the previous five? How am I, my student, or my school different?
It is all somewhat depressing since I felt that we really had created something special that was meeting the [goals I set for my students](https://pedagoguepadawan.net/37/whysbg/). Instead, this semester, I feel that our fragile ecosystem has been shattered and I’m not sure we can recover.
Don’t backtrack – you’ll compromise your message and lose their trust. I’ve read of a similar experience on someone’s blog (can’t remember which!), and his conclusion was that he’d lost their trust by panicking and backtracking. What are the admission tickets to reassessment? Mine are relatively stringent – all corrections done, more evidence in addition to show that you’ve improved. Make that part onerous and effort will shift forward to avoid that. It may be too late to do that this term, but it would have a neutral to positive effect on your ‘usual’ group of attitudes and a positive effect on a class like this, I think.
We’re not planning on making any major changes this semester. Rather, we are thinking about what adjustments to make for next year.
Our admission ticket consists of all of the assigned homework problems, for that standard, completed in detail (sketches, diagrams, givens, equations, solutions, etc.).
I am interested to see what the students say on their feedback forms, and in the content of the conversations you have with students. Please share when you’ve reviewed the results.
Is it possible this is somewhat the result of “end-of-the-year-haze” on everyone’s part? Around this time of the year I am also thinking about how terrible of a job I have done and how much my students probably didn’t like any of it. What I’m saying is it’s easy to be down on yourself and your policies when everyone’s tired.
But like I said, I’m interested in what they have to say.
And how different is the 1-5 AP scale than the 0-4 mastery scale?
It may be end-of-year malaise, but we do have some quantitative data to compare with previous years….
I’ll share the results of the survey and conversations. Those from a previous year are posted here: https://pedagoguepadawan.net/33/studentfeedbackonsbar/
While there seems to be some differences among the different scales, I think the 1-5 AP scale is probably quite similar to the 0-4 scale. I use the 1-5 simply to reinforce the AP exam scoring…
We’ve run in to problems here with students not doing any prep for the first assessment, then using that assessment to study for reassessment. One thing I’m trying (with mixed success) is giving them access to practice assessments beforehand. Next term I’m also going to up the bar for reassessment by requiring them to show they have done practice problems as well as correct the original assessment to earn the right to reassess. Hopefully these two things will help us motivate students to study BEFORE the assessment more often.
I’m now definitely going to add a question related to this to my survey. I’ll try to ascertain if requiring more work to be permitted to take advantage of the reassessment would motivate students to better prepare for the initial assessment.
This comment is from Kelly O’Shea (http://kellyoshea.wordpress.com/) but due to issues with my site, she tweeted it instead:
By the way, the 80% thing is one of the most hideous things I’ve heard recently (heard it on the AP listserv first, not here—told my students about it and they were equally dismayed that anyone would do that to students).
Some other (less horrifying) things that will help them reconsider relying mainly on reassessment could be… offering the opportunities less frequently (I have kids coming in earlier and more frequently this year when I only let them do it once per week than last year when I let them do it at almost any time of any day). Something about reducing the supply increases the demand (or something). And/or limiting the number of objectives per reassessment to something like 3. If they know they can only test on a few at once (not an entire in-class test again), the overly-strategic student will figure out that they would rather get more of it in class than have to come in so many times.
Or, my favorite—don’t let them take assessments in class until they (or you?) think they are ready to show mastery (on an individual basis). This could work especially well with more frequent formative assessments before the end of the unit (quizzes). You could even have the quizzes as “homeworks” if they would eat up too much class time. One option could be to have the quizzes be “graded” with scores that will be ultimately replaced by the in-class test score. You (and they) can use the quizzes to determine whether they are ready for the in-class test or not. Cheating would be disincentivized (a word?) by (a) the grades don’t count and (b) it would be immediately and glaringly obvious if a kid got all the quizzes correct and was obviously not near mastery on the test. If they aren’t ready for the test in class, they could spend that in-class time doing the corrections that they haven’t yet done on the quizzes, then come in outside of class to take the test instead of to take a “retest” like they would have had to do anyway. Now they are (a) getting tons more feedback ahead of the for-real tests and (b) if they are not staying on top of things, they now have twice as much out-of-class work to do for reassessments since they would have to take both the in-class and out-of-class tests out of class.
I know you have more students than I have, but I think some of those ideas can be massaged into ones that scale up well.
P.S. I don’t mean for these suggestions to come across as pushy (though they might well come across that way… sorry!). I’m just letting my brain run and typing out the ideas I’m having so far.
P.P.S. Good luck! And let us know what the kids say. Their feedback will of course be way better than anything I can think up from a few states away. 🙂
You may be right about more formative assessments, I’ll definitely probe that on the survey and in the discussion. The current formative assessments that we do before the summative assessment (yes, the unit exam) are whiteboarding of problems and lab activities. Both of these are group activities as opposed to individual ones. Quizzes may be very helpful. One challenge we have is that this course moves at a very fast pace. However, perhaps quizzes could be done outside of class.
We currently offer reassessments for a two-week period after the unit exam on Thursdays before and after school. Students only get one additional opportunity per standard to demonstrate mastery. However, they can reassess as many standards as they want in a single session. Maybe limiting the number of standards would help. Or, as others have recommended, a more formal request process where students, in advance, request a specific standard and document the additional practice that they have completed.
I very much like your favorite idea. It has similar elements to what Josh Gates (@DeltaGPhys) described on Twitter recently: http://www.twitlonger.com/show/h1fa8l I would have to consider how to manage those students who purposefully miss the scheduled in-class summative assessment in hopes of learning from those students who take it details of the exam.
I agree that your ideas could scale to our 130+ honors physics students. We currently leverage WebAssign as a reassessment tool which help significantly. It may also help as a formative assessment tool if done correctly. I don’t want to repeat my previous mistake with WebAssign and homework: https://pedagoguepadawan.net/166/no-more-credit-for-homework/
Thanks so much for you thoughtful comments and help!
I have a SBG question that I haven’t seen addressed anywhere in the various blogs – maybe someone has a suggestion? Since you’re discussing problems with SBG … I ran into this one this year: I give a unit test, say on Thursday. A student is absent. The expectation is that they make it up on Friday if they are back in school. They then tell me that they can’t do it Friday because of some reason. My classes meet later in the day, so that’s even worse — I can’t say to come at lunch because it’s past already. So, the student says, Oh, I’ll make it up on Tuesday, or Wednesday. Whoa, pardner! We took the test on Thursday. Why should you get an extra 5-6 days? But in SBG, it’s not WHEN a student learns….but…. I really have an issue in regard to tests. So, do I return tests for students to look over? I have to. But then the student who delays taking the test has an advantage. Do I write an entirely new test? I feel like I’m being punished for their absence.
One thing I have done is put “timeliness” as one of my “skill standards”, so that’s something. But it still doesn’t really solve the problem of making up a test in a timely manner, and the fairness issue about having extra time to find out what’s on the test. Because, what am I testing? What they know, or what they asked others who took the test what they know?
I’m simmering this on the back burner, hoping for a creative solution. I’d appreciate any/all feedback and ideas on this.
Many thanks for your blog — it’s been great seeing your work and your ponderings! (not sure if that’s a real word, but I guess it is now!).
We have a couple of versions of each exam with the same questions but different order. This somewhat thwarts students coming in with the multiple choice answers in order.
For exceptional circumstances, I’ve taken a multiple choice exam and removed all the choices and turned it into short answer.
However, once some students see the exam, those questions can be shared with other students even if you haven’t passed back the test.
To completely address this issue, you have to have a culture where sharing test questions with those who haven’t taken the test is unacceptable. Unfortunately, I don’t have this type of a culture.
I get frustrated when students cheat on exams. It drives me crazy since I find it so unfair to those students who don’t. The stories I could share about the lengths I would go to to thwart and catch students cheating. However, I take a somewhat different approach now. I have a limited amount of time and resources. I have to balance between investing my time and resources to help those students who want to learn against thwarting those students who don’t. Neither extreme is best, but I find it much more satisfying to invest in those students who want to learn.
Geoff,
You said, “I still believe our methodology is philosophically sound. However, I fear that this deferred effort approach will result in less understanding and less retention.” You also alluded to culture in your classroom. From my experience, the question raised about students taking ownership of the first assessment (and any future assessment, for that matter!) comes down to the classroom culture and the relationship between teacher and student – the “art” of teaching, if you will.
What types of conversations do you regularly have with your students related to assessments & grading? Here’s an example of how I attempted to change the conversation in my classroom a few years ago: http://mctownsley.blogspot.com/2010/11/not-for-grade-homework-whats-point.html
Maybe we could start some sort of SBG-student-conversation-carnival or something like that, I think we could all learn from each other’s culture-building conversations (even though our contexts may be different).
Matt, thanks for reminding me of your post; it is great. While we do have conversations in class about assessments and grading, they are probably too infrequent. Near the beginning of each semester, I always share my high level goals: which provide the rationale for the SBG methodology:
https://pedagoguepadawan.net/37/whysbg/
I should also share, that the situation was not a dire as I thought when I wrote this post. Some of the feedback I received later was more positive and enlightening:
https://pedagoguepadawan.net/188/sbar-and-mastery-student-survey/
While the challenges with deferred effort still exist, I am now aware of other factors (e..g., need for additional formative assessments that also need to be addressed). An additional challenge, especially when having the “homework without points” conversation is that the conversation is significantly harder when you are the lone voice in the wilderness. As more teachers adopt SBG and stop motivating students to do homework with points, it becomes easier for students to prioritize how best to spend their time. Several of my students simply cannot do all of the homework that is assigned with the time that they have. (Another problem unto itself.) Unfortunately, homework for points, pollutes the prioritization by these students.
I like your idea of another SBG carnival focused on student conversation (there hasn’t been one in quite a while). John Burk shared a video of his conversation last year which was fantastic:
http://quantumprogress.wordpress.com/2011/08/25/my-grading-sales-pitch/