While I'm not as huge a propononent of overcoming bias as these folks, I think this case deserves wider recognition and some action....
There have been a ton of studies going way back (to at least 1927) that show student handwriting impacts teacher's evaluation of their work. To me, the most obvious explanation for this phenomenon is that good handwriting is a signal of intelligence (or high achievement) which teachers use as a shortcut to evaluate the work.
It is potentially very easy to rule out this explanation. If handwriting is not related to intelligence or school performance as measured by a computerized test, then we can rule out the above explanation. Of course, if it is related to these measures, it doesn't mean we can rule out competing hypotheses that may play a role in the story.
As far as competing hypotheses are concerned, it's possible that good handwriting signals something else (or something more) than intelligence or high achievement. Based on my experience grading plenty of university-level homework and exams, the most likely candidate is gender and, to a lesser degree, race. That is, female and Asian students tend to have the nicest handwriting. So one possible explanation is that teacher's behavior is biased by gender and/or racial attitudes.
Regardless of what exactly is driving the bias, the costs are potentially severe. In addition to the possibility that this bias can affect a child every time he/she turns in a piece of handwritten work is the fact that the single most important test that most students ever take is also subject to the bias. The SAT is supposed to be a great measure of ability because it's standardized and because the scores are objective. But how can these tests (in which cursive essays score higher than printed essays) be truly objective while maintaining a handwritten writing section?
What does all this mean for you? You should put effort into improving your child's handwriting. This is unfortunate because it really seems like a waste of time. (That is, it's unlikely to improve their real productivity as adults.)
What does all this mean for us? Given that this problem has been well known for so long, it's a wonder that no direct efforts have been made to deal with it (that I know of.) Perhaps remedies weren't so easy when a lot of this research was done in the 70s and 80s. In any case, today there are a number of low cost measures that can be taken immediately to mitigate the problem. First, we can spread the word about the bias and commit to not succumbing to it ourselves. Second, student evaluation can be more focused on multiple choice type questions. (Cheer on fellow TAs!) Of course, the extent to which multiple choice questions can test understanding is limited. For this reason, term papers (in which the identity of the students are masked prior to evaluation) are a good option. Fully computerized work by students seems to be the ultimate solution--perhaps this is coming anyways.
As far as research goes, as I said before, much work has been done on this topic. At the same time, based on the limited amount of reading I've done, the quality leaves much to be desired. Here's the experiment I'd run:
1. Have a collection of students write a page of random words.
2. Give students an answer to a question and have them write it in their own handwriting.
3. Randomly assign samples from #2 to teachers to evaluate.
4. Randomly assign samples from #1 to teachers. Have them evaluate the quality of handwriting on a 1-10 scale. Also have them evaluate how confident they are that the sample is from a boy/girl on a 1-10 scale.
5. Make sure to collect demographic data on students and teachers.
There are two main innovations of this experimental design. First, the evaluation of the handwriting quality is separated from the evaluation of the "work." Second, and more importantly, is obtaining data on the teachers impression of the child's sex. With this data we could to separate the sex signal from the handwriting signal. And we could also test the initial hypothesis I posed above (by collecting data on the kids prior academic achievement.)
Too bad this would probably be a lot of work. Anyone want want to do the heavy lifting?
2 comments:
interesting, but no I don't want to do the heavy lifting. But I was thinking that if I am guilty of this, it would be because I tend to reward clarity of thought in my grading (especially as a Poli Sci TA) and if I can't sort out what is being said, then my grading will be biased downward. But perhaps this research controls for the extent to which the content of the writing can be discerned?
Then again, I may very well associate bad handwriting with laziness at some level, though I don't think I am biased with respect to gender/race. Unless she's cute of course (strike that from the record!)
I agree with you. The comparison has to be ugly but legible handwriting versus nice and legible.
I wouldn't be too surprised if it *were* gender based. One reason I think this is that boys tend to be very disruptive to classrooms--I could see this fact gradually influencing teachers' behavior.
Of course, even if this is the root cause it doesn't mean it's gender based. It could be as simple as bad handwriting being associated with bad behavior. In this case, it would just happen that boys would be disproportionately affected. Causality is always tricky to discern....
Post a Comment