In my role as intructional faculty, I aim to grade everything anonymously, which is a provision I enjoyed as an undergraduate. My current method is to ask students to write their names on the back of their papers and exams, which also helps me to return them. One of my students remarked that I must do this because I am particularly biased. She may be right. But there is reason to believe that we are all biased against minority groups in our grading practices. Take this publication on the perception of grammatical and spelling errors by partners at 22 law firms: "The exact same memo averaged a 3.2/5.0 rating under our hypothetical 'African American' Thomas Meyer and a 4.1/5.0 rating under hypothetical 'Caucasian' Thomas Meyer. The qualitative comments on memos, consistently, were also more positive for the 'Caucasian' Thomas Meyer than our 'African American' Thomas Meyer." It seems obvious to me that these effects could have an impact on the grading of philosophy papers and exams. (It may be worth noting that the gender/race/ethnicity of the partner did not affect these findings, although "female partners generally found more errors and wrote longer narratives"). And take this publication on faculty assessment of a student applicant, mentioned a couple of years ago here at NewAPPS: "Our results revealed that both male and female faculty judged a female student to be less competent and less worthy of being hired than an identical male student, and also offered her a smaller starting salary and less career mentoring." The difference in mean rated competence, hireability, and mentor-worthiness was of the order of 10%. Again, it seems obvious to me that these effects could have an impact on the grading of philosophy papers and exams, which could be a grade-letter difference (i.e. the difference between a B and a C). Since perceived differences in grading standards could have an impact on whether students choose to stay in philosophy, it seems to me that anonymous grading would both be more just and would encourage a more diverse range of participants in philosophy (see other suggestions on this over at Daily Nous). What does everyone else think? Do you grade anonymously? If not, why not?
23 responses to “Anonymous Grading”
-
It’s worth one’s while to check one’s own consistency by grading things twice with the sessions a few days separated and keeping comments on separate sheets. Scary.
LikeLike
-
Grading anonymously where it’s feasible and easy should be a no-brainer. The issues against it, I think, are that it sometimes comes at the cost of time and energy. Therefore, I try to dampen the potential effects of bias by whatever tools I have the time and energy to employ. That includes grading anonymously, maximizing the degree to which I feel accountable for my grades, randomizing my patterns of grading, and paying attention to things that affect mood and my cognitive resources.
Ever since I read about the study where judges’ hunger and blood-sugar levels affected their decisions in the courtroom (http://www.nature.com/news/2011/110411/full/news.2011.227.html), I try not to grade anything while I’m hungry, tired, or in pain. I also refuse to grade students’ work if I’m irked by them. Once a student emailed me a paper where they were really supposed to hand in a hard copy. I printed out their paper, but it fell off the side of the copier. I picked up the 6 pages and realized the student hadn’t numbered any of the pages. I was already having a bad day, and was behind in grading. But I forced myself to wait until later that night, because I was so irked at the student at that moment. I wouldn’t have given them a worse grade on purpose, but I knew my annoyance could potentially color my judgement. Granted, we’re always in various emotional states, so it’s not like I can wait until I’m feeling completing serene and ‘unemotional.’ But I try to do damage control by not grading when I have a reason to think that I’ll have to expend a significantly greater amount of energy than normal to be focused or charitable. That also just strikes me as a cognitively efficient way to grade.
I’m certain I’m more charitable to students whom I like and whom I think are smart, when it comes to interpreting the strength of their analysis in papers. I have caught myself a few times thinking, “so and so must mean something more nuanced here, because I know they understood the material.” This one is hard, because I might be right in this instance to be charitable to this student. But, I know I might not spend the extra five minutes looking for a more nuanced claim from all the other students. So, to compromise, I allow myself to grade on these interpretative hunches, but I first force myself to articulate (in writing or just as a mental note to myself) why they are receiving the grade I am giving them. As a check, I imagine, what if another student who got a lower grade saw this paper and its grade. Would I be able to justify to them why they got a lower grade? If so I slap on the grade and move on.
I also have a hunch that I’m more charitable towards men in a similar regard. This is one reason why I think everyone by default should consider anonymous grading. I’m a freaking feminist who studies bias and spends a bunch of time reading articles about bias in educational settings…and I still have the nagging sense that I more easily interpret men’s arguments as competent or insightful. But again, due to diction and whatnot, a lot of us can easily discern an author’s gender without ever seeing their name. So again, I try to combat this additionally by forcing myself to be as explicit as possible about my grading criteria, since bias seems to spike when we know we won’t be held accountable for our decisions.
I also have seen evidence that I’m more charitable to papers I grade in the beginning of a stack, as opposed to those at the end. Ideally, it would be great to go back through the stack and re-assess grades a second time. But with 40-60 papers or finals, that’s just not going to happen. So to do damage control, in a big class I grade either in a random order, or, if I have less time that semester, alphabetically but I reverse the order every other assignment. Not ideal at all, but better than throwing up my hands and not even trying.LikeLike
-
I don’t grade anonymously. I am not sure that I’d be able to see how the students improve or worsen in crafting their papers if I were to grade them anonymously, and it’s important to me to see their work over the course of the semester to better understand where they are during the semester, in case I need to talk with them about persistent problems they’re having. Getting a sense of who the students are in the way they write also allows me the opportunity to notice when there are sudden changes in the quality of their papers, whether due to the material finally coming together or due to outside and inappropriate assistance. Tracking their thinking humanizes them as people struggling through the material. Some struggle more than others.
Maybe these are things anonymous graders learn after the fact once the names are revealed and the grades assigned, though.LikeLike
-
I grade anonymously, and it’s hardly any hassle at all. I do all my grading electronically (I prefer to type comments than to write them by hand), so I just have students submit papers without their names on them, and with random filenames (I sent them to a random number generator site). I have the students include their 9 digit student IDs at the end of the paper, which I use to identify the papers once I’m done grading.
It doesn’t make for complete anonymity; sometimes students will have discussed a paper with me in office hours in sufficient detail that I can recognize it when I’m grading. But that’s only ever a minority of papers, and it seems to me that largely anonymous grading is preferable to grading that is not at all anonymous. (This just goes for undergrad papers–for grad papers, effectively anonymizing is much harder.)
I really don’t see two sides to this issue. The only arguments I’ve ever heard in favor of non-anonymous grading are really (it seems to me) arguments in favor of non-anonymous commenting–e.g., it’s sometimes helpful to target comments to particular students. But it’s certainly possible (and I often do it) to go back and add comments to a paper after de-anonymizing it, without changing the grade.LikeLike
-
I grade anonymously, with students putting their student ID#s on papers or exams. As Daniel Greco says, this doesn’t make for complete anonymity, since there are cases where I’ve discussed a paper or seen a draft previously and recall the student. But it’s far superior to complete non-anonymity.
I’d like to add that anonymous grading also helps with discussing exam and paper grades with students. Many students who receive low grades suspect that their grade are largely due to the instructor disliking them or otherwise having it in for them. It’s useful to be able to say (honestly) that I had no idea that it was their work when I gave it a C-.LikeLike
-
I’ve used anonymous grading for the first time in a large lecture (~150 students) this spring. I have three TAs for the course. This is my fifth time teaching the course as a large lecture.
We have reason to believe that the benefits of anonymous grading, in terms of equity, are quite significant (e.g., the studies you mention), but I think it is important to be honest about the potential costs. One type of cost is administrative: educating students about the procedure, producing compliance with the procedure (try getting 100% compliance among 150 people on any policy), dealing with non-compliance, actual anonymization of papers/exams for the TAs, and then de-anonymizing them to record grades and return them. For a large course, these administrative costs are not nothing, but I regard them as relatively insignificant given the equity benefits. Of greater concern is the overall effect on grading. The average grades given were between 1/3 to 2/3 letter grades lower, FOR EACH ASSIGNMENT, compared to the previous four years among all TAs. Although I have mostly different TAs every year, this is quite striking given the relative consistency among all similar assignments of other years. This was not disastrous for the students as a whole – I applied a curve to bring the grades within the normal range (and the range our department targets). Yet, on the face of the limited evidence I’ve given, grades are suppressed when the grader is unaware of the person. I don’t quite have a full theory yet, but my initial guess is that the explanation lies in the ease of being harsh and less sympathetic to the faceless paper, and I’m uncertain that this an attitude properly encouraged. It might lead, for instance, to more mechanical grading on the basis of a rubric.LikeLike
-
I almost always grade anonymously. It really doesn’t make trouble for my workload at all, and I think it’s probably the right thing to do. I type drafts of my comments and enter grades, and then I go back and read through my drafts again after entering grades, and, finally, I compose a brief personalized introduction to each set of comments before printing them out.
I must add, though, that the response from students and colleagues has not been very positive. The students report that they’ve neither met nor heard of any other course instructor who requires papers prepared for blind grading, and some students spread a rumor that something dreadful must be in my past–that I must have been formally charged with grading unfairly so that I adopted this approach as a way to block future charges.
My colleagues, on the other hand, seem worried that I’m making them look like graders who don’t care about fairness or who assume themselves to be immune to unconscious biases.
My advice is to be careful about what you say to students and colleagues about blind grading if the practice is unfamiliar to them. And if there’s a convenient way to do it without telling anyone that you do it, that might be the best way to go. (I talk about it because I don’t want students to find jarring the failure of fit between the personalized introductions to comments and the more impersonal comments themselves.)LikeLike
-
Lots of good points here. I actually had a snack after reading this since I am in the middle of grading finals. I do think that sometimes there are practical constraints, but hopefully we can find solutions for those on an individual basis without adding to instructional workload in any significant way.
LikeLike
-
I agree that there are benefits to keeping track of student progress in the way you describe, but I think I have been able to do this relatively well by checking the name of the student post-grading. But my class this semester had 50 students, and I don’t think I was as on top of their work as you seem to be with your classes.
LikeLike
-
The last point you make is especially interesting to me. I am not sure how to evaluate whether the grade difference is a good or bad thing, but it is at least news to me. Separately, it would be nice if the anonymization procedure could be automatized. Does anyone know the technical capacities of Blackboard or Turnitin on these points? I actually use a system (CROPS) that would make anonymizing electronic submissions a difficult procedure. It would be great to find a solution to the practical issues if anyone has suggestions.
LikeLike
-
I had a post on this topic last year (grading season…):
http://www.newappsblog.com/2013/05/on-anonymous-grading.htmlLikeLike
-
Seriously, this isn’t the norm? I’ve taught at four institutions in the UK in the past 10 years. All institutions have had mechanisms in place to ensure papers are marked anonymously. These have always seemed trivial and not at all administratively burdensome for the marker/lecturer. If you want to know if a student is improving, which is of course something all markers should be interested in, you de-anonymise the papers once satisfied with the marks. Complete no-brainer.
LikeLike
-
I’m not sure how on top I am, but my classes are very discursive to the point where I am not sure I could completely remove the voice of the student from who the student is. We have both in-class and online discussions, so I read and hear them often while the class contiues. I’m probably just ‘reading into things too much’ on this, since if I were to anonymize the papers, perhaps I’d have the kind of emotional or cultural distance to evaluate the papers from that translated distance in an equitable fashion. But I’m also strongly influenced by womanist thinking, so I’m constantly wondering to what extent removing voices from persons makes me less inclined to see these words and acts as the life of particular people with particular problems from particular histories. TR’s theory about the ease with which we can be harsh with the anonymous (or, I can see someone say objectivity just seems harsh because we’re habituated to a life having our little subjectivities soothed and coddled, and abstraction is a necessary corrective to this larger cultural problem) makes a lot of sense to me given all the time I’ve spent in online communities where anonymity or pseudonymity were the norm. Embodying the papers as the work of a distinct student whom I have to see, smell, talk with, share time with counteracts the habit in me to be cruel towards the “faceless paper,” as Tim put it. I can’t read that without thinking of Levinas.
At the same time, I know others don’t see justice this way and look for more formally universal standards and practices structured to mitigate the effects of conscious and unconscious biases. People easily approach each anonymous paper with a spirit of charity and thoughtfully evaluate arguments and evidence; I don’t think anonymous grading leads ineluctably to mechanized or dehumanized grading, but it can, just as person-tracking grading can lead to prejudiced (for better or worse) grading, but might not. I’m not saying anonymous graders cannot see the person through or within the paper; I’m saying I understand who I am and what tempts me. It is good for me to see that others strongly emphasize the superiority of anonymous grading, for the challenge and corrections to how I think about pedagogy. I’m hardly settled in how I teach and evaluate, and I fully anticipate anonymous grading in the future.
I realize I’m in the minority in this, but it’s not the first time being a minority. The discipline does well with a variety of voices, learning from one another.LikeLike
-
I find it interesting that many people use student ID number in lieu of name to keep track of grades. When I was a teaching assistant in grad school, the official registration sheets that I got for each section were essentially a list of student IDs, names, and the number of credits the student was registered for (if the possible number varied). Given that I did my best to learn every student’s name within the first few weeks, this meant ID numbers were not particularly anonymizing.
(There are, of course, other things which will make anonymous grading impractical. I still laugh over one morning in 2nd semester Greek when we had a short in-class quiz. One student was missing, so all the rest of us put his name on our answer sheets. The next morning, that student was rather shocked to find the prof handing back each assignment to the right person, calling each of us by his [the student’s] name. A few weeks into the semester with regular in-class quizzes, our prof could ID all our handwriting.)LikeLike
-
Carolyn@10:
Here is what I do: let’s say that I’m teaching Phil 4010 (Plato), and I require 2 papers. I tell students not to put their names in their papers and to use the following sort of filename naming procedure for their first paper:
-Phil4010paper1.doc
Then the students e-mail me their papers as attachments, so I have a bunch of papers with names like 3583-phil4010paper1.doc. Then I right-click on each attachment (or control-click if I’m using my Mac laptop) and do a “save as,” saving the papers to a folder I’ve created (“Phil4010 paper 1”) for the papers. (That way I don’t have even to open up the papers when the person’s name is on front of me, and I trust in the rottenness of my memory to have great confidence that I’m not going to recall that “3583”=Jane Doe when I’m grading the paper.)
I do it this way just because I’ve been having students e-mail me their papers for a long time, and I didn’t want to bother to learn how my course management software does it. (Especially because it seems like we’re changing or updating said software every couple of years, making learning the new interface / procedures a massive PITA.) So when I went to anonymous grading, I just tweaked my former practices with regard to the filenaming convention and leaving one’s name out of the document.LikeLike
-
Anonymous grading is possible with Blackboard; one of the rather few nice things about it (an option in the drop-down menu for the assignment’s column in Grade Center). It does require getting students to leave their names off papers, filenames, etc., which in my experience requires repeated reminders. On another note, once I’ve explained my reasons, my students seem to appreciate that anonymous grading is a fairer policy in which everyone can have greater confidence.
LikeLike
-
I see all the reasons to grade anonymously (to check implicit biases, etc.), but my pedagogy would make this difficult to do. Since I teach at a small liberal arts college with relatively small class sizes, I regularly meet with students individually to discuss their works in progress. And all of my exams are in-person oral exams. I would be more interested in suggestions for correcting for the factors that might skew a grade (in either direction) when the identity of the person is known.
I do several of the things already suggested (randomizing the order of grading, using a rubric, etc.), but if there are particular suggestions that come out of the literature you are citing, I would like to know.LikeLike
-
Thank you! I took a look at it and some of the themes are similar. Do you have an updated view on the topic?
LikeLike
-
I admire the approach you are taking here on justice, but I suspect that we have different stances on the purpose of grading. I take grading to be an assessment of what the student has learned out of what I take to be an objectively available body of knowledge, presented through lectures, discussions, and readings. If I want to know whether the student understands Lewis’ objection to the Consequence Argument, I don’t think I need to be sensitive to their particular background. In fact, I try not to be sensitive to their background when making these assessments. Sometimes my students will express being confused about what is true with respect to a particular issue and this semester I have encouraged them to set those worries aside for the purposes of their papers. This is because I think they need to work on learning the material first, to be critical second, to be generous third, etc. etc. I can imagine another stance on the education project, where this sort of learning is seen as less important and maybe even less real than a more personal form (one of my sisters attended Hampshire College for a time). I am guessing that you have this sort of stance. Is that right?
LikeLike
-
For me, likewise, I think the numbers would be identifying over time. Short quizzes might not be as much of an issue as papers and exams. I am grading hand-written finals now, but I don’t recognize the hand-writing since the students have turned in typed papers all semester.
LikeLike
-
This works. And it would work on our system (CROPS). Thanks!
LikeLike
-
Well, I don’t think other pedagogical strategies for grading or evaluating are less real. Graders are going to suffer intimately no matter how we do it, if we’re doing it sincerely, maybe even worse if we’re doing it grumpily. It is true that I’m asking my students to prepare analyses of our readings, so they’re not just being evaluated on whether or not they get certain movements or positions correct but how they show their interpretations. If it is strictly a matter of getting a particular fact right, then I’ll incorporate those questions into quizzes the class machine (Desire2Learn in our case) will grade for me (maybe that counts as anonymous grading?).
To be honest, I find I can’t really express in one way what all is going into this internal debate I have regarding anonymous grading, since I’m inclined by the various arguments and anecdotes given to consider changing my practice. And rehashing my dialectic on this will go immediately to the deep questions I have about philosophical praxis to begin with, and I’ll spare the gory details of how quickly I see the values in the various sides.
So, at this point, I’m going to say that I just might change things next semester and see how it affects my judgments about my judgments.LikeLike
-
I have views, and this might be the time for an update indeed. Will try to find the time 🙂 (Time for blogging has been much too little…)
LikeLike
New APPS 
Leave a reply to Charles R Cancel reply