Saturday, June 28, 2014

My ISTE 2014 "Think topics"

One of the things I do at conferences is take time to step back and think about big picture items that are difficult to get my head around in the flow of everyday life. So at ISTE this year, in addition to the sessions, and conversations with member of my PLN, and workshops, and keynotes, and wandering the massive exhibit hall, I'm setting aside some time to try to do some planning for the next school year.

Among my topics are the following:

  • Instructional Coaching: How can I and the other ToSAs be effective at working with teachers in classrooms this year, instead of in pull-out workshops?
  • Google Drive workflow: We have several new tools, and Google has added and changed features. What are some best practices for teachers to use Google Drive with their students?
  • Getting tablets into teachers' hands: This year we had nearly a dozen Android tablets out for teachers to work with. Next year I'm hoping we can purchase several dozen more. What are some ways that teachers can use these effectively in their classrooms?
  • Next steps for Instructional Technology Vision: Our Tech Vision Team did a great job this year setting out our district's overall vision for instructional technology. Now, what are our next steps in turning that vision into practice?
  • Inspect Itembank rollout: What is the best way for us to introduce and support our new Common-Core-aligned itembank, so that teachers can make use of it?

Rubric Scoring and Grading Scales

If you use a rubric to evaluate student work, how do you convert that rubric score to a percentage grade? A recent conversation on Twitter turned to these considerations, which at first may seem trivial, but are actually quite significant when communicating performance levels to students. 

In the conversation, one person referenced roobrix.com, which is a website that claims to calculate “correct” grading based on rubrics, as opposed to most teachers’ practice, which results in “wrong” grades. The site itself is useful, if limited. As I read the rationale behind the “correct” rubric grading, something seemed off to me, but I couldn't put my finger on it for a while. I thought about my own grading of student work using rubrics until I realized my discomfort.

First, you should read the explanation given at roobrix.com. Then come back; I’ll wait.
.
.
.

I’d like to highlight what I think are the two problematic quotes from that page:

“Let's take a rubric with four levels and four criteria as an example. If a student's performance is at the bottom level (Level 1), that means the student has met basic expectations in a limited or unsatisfactory way. This effort should be considered a minimum passing grade.” 

“Forgetting that the lowest performance level represents a minimum pass, [teachers] are wrongly equating this bottom level with complete failure.”

If your rubric is such that the lowest performance level represents a minimum pass, then the reasoning and calculation at roobrix.com is rock-solid. I have no quibble with that.

However, if the lowest performance level in your rubric represents a minimum pass, how do you communicate to students when they have performed unsatisfactorily? Is there truly no room in your grading scale for “not good enough?” We all know that not all students master tasks or skills sufficiently on the first go-around. If a rubric is to communicate to students their level of mastery, a rubric that only goes down to “minimum pass”, does not communicate to students who have not performed up to mastery.

Here’s what I did in my class. I taught high school science, and used the following 3-level rubric for lab reports.


1
2
3
Data
  • Data was collected and recorded poorly or incorrectly.
  • Too few data points were recorded to effectively analyze the data and graph.
  • A data table, if present in the lab report, is unreadable or unlabeled.
  • No justification is given for data points that were discarded.

  • Data was collected and recorded correctly.
  • Enough data points were recorded to effectively analyze the data and graph.
  • A data table, including units, is displayed in the lab report.
  • A justification is given for any data points that were discarded.

  • Data was collected and recorded correctly.
  • Enough data points were recorded to effectively analyze the data and graph.
  • A data table, including units, is displayed neatly and legibly in the lab report.
  • A justification based on physical circumstances is given for any data points that were discarded.

Graphs
  • Graphs have incorrect or unrelated quantities.
  • Graphs are sketched quickly or are unreadable.
  • Graphs are drawn with inconsistent scales.
  • Graphs do not include a best-fit line, or data points are connected dot-to-dot.

  • Graphs are drawn readably.
  • Graphs have correct quantities, perhaps on opposite axes.
  • The scale of each axis is not explicitly labeled but is consistent and evident.
  • All straight-line graphs include a best-fit line.

  • Graphs are drawn readably.
  • Graphs have correct quantities on correct axes.
  • The scale of each axis is clearly labeled.
  • All straight-line graphs include a best-fit line. Curved graphs do not.

Analysis
  • The best-fit curve is missing or does not fit the recorded data.
  • The best-fit equation is missing or left as a mathematical function (y=mx+b) with no physical substitutions.
  • Unit analysis is missing or incorrect.
  • Slope and y-intercept are left as mathematical numbers with no physical meaning attached.

  • The best-fit line does not fit the data appropriately.
  • Physical quantities are substituted for x and y variables but not the slope or y-intercept in the best-fit equation.
  • Unit analysis is missing or incorrect.
  • Slope and y-intercept are left as mathematical numbers with no physical meaning attached.

  • The best-fit line is appropriate for the data, both mathematically and physically.
  • Physical quantities are substituted for x and y variables, slope, and y-
  • intercept in the best-fit equation.
  • Units are analyzed for the slope and y-intercept.
  • A physical meaning is assigned for the slope and y-intercept.

Conclusion
  • The outcome of the experiment is not explained.

  • The outcome of the experiment is expressed as an equation or formula.

  • The desired outcome of the experiment is expressed in terms of a relationship between physical quantities.


These can be summarized as follows:
  • 3 - Perfect or nearly so
  • 2 - Well done, but….
  • 1 - Unsatisfactory

Once I had evaluated those criteria, I combined them into a holistic, overall 1/2/3 score. (As I’ll write below, I would re-think the holistic scoring today. For now, let’s consider an additive score for all the rubric criteria.)

When I developed the 3-level rubric, I quickly realized that it was incompatible with a traditional 90/80/70 grading scale. Students who consistently perform at the next-to-highest level (2) would score a 66% for the class, which on a traditional grading scale would be a D. That’s not reflective of their true ability. Fortunately, I had the freedom at my school to modify my grading scale. So I re-wrote my percentage scale to reflect the scores in my rubric:
A - 83% to 100%
B - 66% to 82.9%
C - 50% to 65.9%
F - below 50%

What that meant for students:
  • A student who was consistently scoring 2’s would get a 66%, just barely a B
  • A student who scored mostly 2’s but some 1’s would be below 66%, for a C
  • A student who scored mostly 2’s but some 3’s would be above 66%, for a solid B
  • A student who scored mostly 3’s would be above 83%, for an A
  • A student who scored mostly 1’s would be below 50%, for an F

A rubric score of 1 was intended to communicate to the student that their performance on that particular criteria was unsatisfactory and needed to improve. It was not intended to be a “minimum passing level”. If your rubric does not have a level for “unsatisfactory” work, then your only choice when a student submits something like that is to give it a zero (off the rubric), which will seriously imbalance grades.

Everything in my class was graded on the same 1/2/3 scale: homework, labs, quiz and test answers, projects, everything. This maintained the integrity of the scoring system and made it easy for me to communicate with students and parents.

As I mentioned above, when I go back into the classroom, I have a couple of changes I would make to this system. First, I would eliminate my holistic overall grade. I think it left too much room for my (sometimes inconsistent) interpretation and was not specific enough feedback for students. Instead, I would use an additive method, and score students out of 12 points for each lab report (4 criteria x 3 possible points for each criteria). Second, I would require students to re-do work for any 1’s they scored. Too often my students settled for 1’s simply because they didn't want to go back to old assignments, and frankly, I didn't push it because I didn't want more papers to grade. Today, I would find a solution for that, because I think I better understand the importance of students moving toward mastery, no matter how long it takes.

I think the take-away message here is that your rubric and grading scale have to be designed to work together. If you are in a situation in which you can modify your grading scale to fit your rubric, you're in good shape. If your school or district requires you to use one particular grading scale, design your rubric to fit the grading scale. In either case, the primary consideration should be communicating expected and actual performance levels to students, even if that performance level is "unsatisfactory".