By now, anyone who knows me knows that I am a *huge* fan of metrics. Particularly when they are used uncritically. So perhaps it was inevitable that I would end up in an environment where metrics play an increasingly ubiquitous role: academia.
I want to introduce three metrics.
Student credit hours: a number that measures by class/faculty the number of students a person has taught. You will have a larger number if you teach larger classes. It’s also the number that is at the beginning of a formula that computes the portion of the Institute’s state budget (and presumably how that is divided, although that part of the budgetting process is a complete mystery to me). Higher is better, and in fairness I can imagine that larger classes can create their own organizational structures that need managing and more potential problem cases.
What’s missing in this metric are some other fundamentals about class.
- Smaller might be better for the student experience including but not limited to mentoring, one-on-one time with individuals, managing different learning styles… and that this might be exactly what distinguishes a University education at a bricks and mortar institution from an online experience.
- Class preparation time, do classes with more students involve more course preparation time. I taught a class recently that was about 1000 pages of reading for 12 people, but it would have still been 1000 pages if it had been 120 people.
- The lack of institutional support for say, grading, that larger classes receive.
Research expenditure. This metric measures the amount of money that the Institute receives when a faculty member spends their grant. Again, bigger is better. But this metric assumes that all research costs the same. Not all research costs the same amount to achieve, and funding agencies know that. It does not account for how much it costs to do research.
H-index. I’ve already written about this.
Imagine my joy when someone suggested that we plot all three against each other for an individual. What would that mean? Someone with a larger class, in an area of research that was more expensive to do, and with a high index does well. So, should we optimize (which is the purpose of metrics, to drive behaviour) for large classes at the sake of not giving students the opportunities that come from small ones? Should we optimize for expensive and popular research, and ignore the intellectual, social and political good that might come from less expensive research areas? Should we give even more legitimacy to the papers of an h-index and not ask about the papers that were potentially unpopular but changed a person’s thinking, deepen their intellect…?
Needless to say this epitomizes all that worries me about metrics. The desire to rank and compare, and use numbers to support that is to think uncritically. Sadly, it’s all too common in academia.
academic metrics, comments, evaluation, metrics, qualitative data, scores, teaching evaluation
Why I Wish to Keep my Teaching Comments Out of My Evaluation
In academia, academic management, empirical on September 21, 2011 at 8:30 amI’ve written a lot about metrics in the past, today my focus is on how qualitative data is generated and the implications for evaluation. I am aware that my management (and I use that term deliberately since this is an evaluation situation) want to see the comments that students write. They currently only see the numeric scores. Their argument is that the comments would enrich their ability to evaluate my teaching.
But, I find myself very resistant to the idea.
First, how do comments shed light on teaching? How do the comments, often typed out hastily in the throws of week 15 of a 16 week semester explain the ebb and flow of the class, the work I did to bring the class together, to draw the timid into discussion, to manage the differences in perspectives among class participants, to listen and counsel the students who brought them problems not related to the class but to their lives and their struggles and joys? These are subtleties of the experience that I’ve never seen in students’ comments. Not surprising, they’re not teachers! Teaching is an intimate and deep experience, one that can only be truly understood through experiencing the classroom. I realize the desire to measure it, but teaching evaluations are only partial instruments hence the ability to improve the scores without improving the actual teaching. Adding comments won’t change that.
Second, I have a particular concern as a woman. I am sure I am not alone in having comments about my body as part of the feedback. It’s tough enough knowing that as a woman my body and its “problems” is a part of the students’ discourse. But I accept that to be young is not always to be thoughtful or kind, and I teach despite that, knowing that I get to keep those indiscretions out of the professional discourse about me. While I respect my all male management, I find the idea that they can read remarks about my body embarrassing. It transforms an annoying inequity confronted by female scientists into a public humiliation.
And that’s why I don’t want my teaching comments made public.