Student Evaluations, Equity, and Advantage

I posted the following tweet this morning: 

This is the time of year when I ignore student evaluations because the research shows that they disadvantage my colleagues who aren't like me (i.e. straight, white, cisgender, man, etc.) and they harm my mental health. It's also a good time to remind administrators of it, too.

— Brad Koch 🎶🚲📚🍻 (@BradleyAKoch) December 10, 2020

There is an unfortunate irony to my tweet in that I am, to some extent, able to ignore my evaluations because of the advantages conveyed by my identities and tenure. Literally everything about my identity grants me structural advantage (a/k/a privilege*) relative to those with identities different from mine, and there is no way for me to opt-out of it or to compartmentalize it. On the one hand, this is unfair to me in that I will never know to what extent my successes were earned or were a product of those advantages. (Cue world's smallest violin.) More troublingly, it is unjust to those who are not like me who are less likely to have successes in the first place.

I followed up with a comment linking to the press release (9/9/19) from the American Sociological Association (ASA) on Reconsidering Student Evaluations of Teaching. I, however, don't fully agree with the ASA recommendations. Confusingly, they argue that "SETs [student evaluations of teaching] are weakly related to other measures of teaching effectiveness and student learning" and that:

A scholarly consensus has emerged that using SETs as the primary measure of teaching effectiveness in faculty review processes can systematically disadvantage faculty from marginalized groups. This can be especially consequential for contingent faculty for whom a small difference in average scores can mean the difference between contract renewal and

dismissal.

but then continue nonetheless with recommendations on how to include SETs as "part of a holistic assessment." If the evidence suggests that SETs are inherently inequitable, they should have no role in the faculty evaluation process. 

Imagine forcing students to wear glasses in certain professors' classes that make everything look blurry and unfocused and then asking them at the end of the semester to accurately describe the professor's physical attributes. We should not be surprised when the students are unable to give anything approaching an accurate description. Now imagine arguing that if only we asked the right questions, we would be able to suss out a fair characterization. This would be ridiculous! One cannot reconstruct data from a concept that was obstructed from the data in the first place. No, the only way to accurately measure the professor's physical attributes would be to remove the glasses from the students' eyes before taking the classes.

Likewise, I don't see a way that we can reconstruct an accurate measure of teaching effectiveness that is hopelessly clouded by culturally imposed implicit biases short of eliminating the source of these biases before students enter the classroom, which, while a worthy project, is one beyond the scope of colleges and universities working alone.

The non-expert, naive opinions of students are of little utility, regardless, but that's best saved for another blog post.

--

I prefer not to use the term "privilege." For a good explanation of why, see Kaufman and Schoepflin's discussion on the topic