Politics of University Metrics

This piece in the Chronicle needs very little editorializing, but it does need to be read. What we measure, how we measure it, and who gets measured increasingly influences funding. Metrics have teeth, but they are also very political.

Diversity and Service

As I mentioned in a previous post recently I read this article about the advantages of being married for male academics versus the disadvantages of being married for women academics. It’s left me with a lot of questions. And being inspired by  Female Science Professor‘s question “why don’t more senior women in STEM blog?” I want to continue

In addition to teaching, research, and publishing responsibilities, service constitutes a major part of a professor’s career. … The gender breakdown within a department plays a significant role. Typically, there are more men than women within a discipline, and yet committees seek as much diversity as possible. Women, then, are often asked to do double the amount of service as men, a number that increases for women of color. While service is certainly considered when promoting, publications play a much larger role.

I understand the logic, to have a diversity of representation/voices at the table and so forth. But this is clearly the flip side of it, that women and minorities can get over-serviced. And since time is limited, service will eat into other important activities like research and teaching. This is a serious problem. But I don’t know what to do to change it. In the long-term we do need to recruit and retain women and minorites in STEM, but what do we do in the short-term? There seems to be a conflict here: we want to hear from diverse voices but in so doing we ask them to participate in things that compete for their precious research time.

One short-term piece of advice I would offer to anyone who fits this potential category, is to be very aggressive about saying no. Benchmark your service against a non-minority in your department at your rank. Do no more. (Read studies such as Link et al. “A time allocation study of university faculty” to see broad trends and uneven distributions as a reminder to do no more.)

The Marriage Advantage for some Faculty

I was just catching up on Female Science Professor’s blog (fabulous). Last year she asked “why don’t more senior women in STEM blog?“

I’ve been quiet on my blog for a while, I had lost touch with it. It was out of my routine. So it sat quietly.

Recently I read this article about the advantages of being married for male academics versus the disadvantages of being married for women academics. It’s left me with a lot of questions.

Female professors were more likely to have a spouse or partner with a doctoral degree, 54.7 percent to men’s 30.9 percent. Their partners were also more likely to work in academe, 49.6 percent to 36.3 percent.

I wonder whether the same is true in Computing? I was thinking of my department, counting up the numbers of men and women married to other academics. There’s a difference.

A woman is quoted with her theory about why the balance is the way it is, she says

“I have a theory about this,” said Tara Nummedal, an associate professor of history at Brown University. “It seems pretty clear that smart women are going to find men who are engaged, but I just don’t see that it works the other way.”

I have another theory, based on my experience of dating, which is that some men find dating women with doctorates (when they don’t have one) difficult. I recall with some pain a date in which I was subjected to something that felt a bit like being on a quiz show. Yes, I happen to know what the second longest river in the U.S. is the Mississippi since the first longest is the Missouri, but I didn’t need to spend an evening playing this game. And, more crucially, a Ph.D. is not actually about being good at quiz questions. You can guess that the relationship didn’t last long, but this experience was emblematic of the problems I had dating non-Ph.D’s.

She added that a female professor with a stay-at-home spouse is quite rare, but often sees men with stay-at-home wives, allowing them to fully commit themselves to their professions.

I’ve wondered this before also. In one job I had, where I was one of a very small number of women, two of us were single and the other married to an academic. There were some single men in the department, but it was a small fraction of the entire department and a healthy number of my male colleagues, including all the managers, had stay-at-home wives. At that time being married to someone who could take care of all the things that arise in life that require being dealt with during office hours seemed like a huge advantage to me. Some of it was probably that I was often lonely (I had very much made my employment decision because I knew it would advance my career and not my personal life, that was hard, but I think it was crucial for getting to the next steps where I was able to balance both). Years later, I’m not sure whether it’s an advantage or not, because I’ve not ever experienced it. I have no comparison points, nor am I sure that the division of labor that I’ve described is ideal (accurate, enthusiastically embraced)… and I am more aware that my salary is a luxury that these families do not have. But, returning to the point of the article, I think it’s important to pay attention to the last part of the sentence, if there is the possibility for someone to fully commit themselves because that’s what the relationship supports, then yes, I still think that is a type of advantage.

I’ll cover another piece of this article later. That’s enough for now.

Program Chair: Reasons to Say Yes…

I’ve chaired the papers track of a couple conferences now. I could write about the process itself, but instead I want to write about the learning experience of doing this. The first conference I ever co-Papers Chaired was CHI 2006 (with Tom Rodden). I owe Tom a huge thank you because he taught me several useful management strategies that I used during both processes, but also have found useful in my day-to-day activities.

And that is a good reason to volunteer to chair a conference. One of the reasons you’ll hear most often for agreeing to do this kind of service is that it’s good for the community. And you do give your time, as you do as a reviewer, member of the program committee and so forth. Another is because it looks good on the vita. I was told, for example, that serving for CHI meant that the community trusted me with the products of their academic research. I’ll add another one into the mix. For anyone who has ever complained about the way a conference is run, or what happened to their paper, nothing beats seeing what the processes are by which the conference is put together. Actually, I think it should be mandatory that anyone who complains, especially more than once, have to get involved with the organization of the conference.

And today I want to offer another reason, what you learn in doing this. Papers Chairing throws up a myriad of management situations. Each one requires a thoughtful response, many require subtle negotiation to balance needs of the various parties. As a program chair, you are responsible for ensuring that everyone who is giving their time to review etc. gets a fair shake and feels that you support them in their service. I like doing that. I feel its a great way to say thank you. Sometimes it’s harder though, as you have to work something out as best you can. Sometimes there are difficult messages to write, and the practice in getting tone as well as content right is invaluable.

Using those Six Years

One of the most common things I hear from new PhD students is that they do not want to be in graduate school for six years. There are a variety of reasons for that including explaining it to parents, wanting to earn a decent salary, and just not being able to imagine what one might spend six years doing.

It’s this latter point I want to take up today. What might one spend six years in graduate school doing. Recruiting for the next job is one way to think about it. Many students come into graduate school not really knowing what they want to do. I respect that, many people who embark on a PhD are fairly young, life is going to involve many changes (as an advisor one of the loveliest things I’ve experienced in graduate school is weddings, seeing my students and others find their partner). But that doesn’t mean that one shouldn’t give some thought to what you want to do post-graduate school.

Perhaps a different question is what are the options available that I would like to explore when I leave. And this begs a second and far more important question, what will I have to achieve while I am in school if I want to explore those options? Since you are probably only going to do one Ph.D. you might not want to leave these things to chance. And the later in graduate school the more your options have been made, either by what you chose, or by what you didn’t. This applies not just to how you are mentored while in graduate school but what you are being mentored for post-graduate school. Understanding that you are responsible as the student for finding someone who can do both is part of (as the book says) “getting what you came for.” I want to reiterate this point… in addition to finding someone who can mentor you effectively during graduate school, you want to find someone who can effectively mentor you for what you want to do post-graduate school. You can not leave this to chance. Or you can, but then you’ll have effectively made some choices that you may never have realized.

So, how do you know what you want to do? Look at what students who’ve graduated have gone on to do. Can’t decide among those choices, then plan for the one that looks the hardest for you to accomplish. You can course correct later on, but only if you’ve started down a path that allows for it. This is what you should use those six years for, and this is what you, as a graduate student, are responsible for doing. Irrespective, I would say, of what you might get told, you need to develop that internal sense of what you want and decide whether there’s appropriate alignment. You may have to ask to get what you want, or whether that’s even possible. Its crucial to understand that your goals may not align completely with another person’s, but it’s still your responsibility to get what you want (or risk becoming what someone else wants you to be).

Aside: to me this is just a variant of understanding evaluation. Every year in corporations, and regularly in academia, people are evaluated routinely as well as for promotion. Understanding how you are going to be evaluated is essential to understanding what you need to do to be successful. There is not always a clear alignment between what others may want and what you need to succeed. But your odds of succeeding are so much better if you pay attention to the means by which you will be evaluated and you plan to achieve it.

Woman@GT: Reflecting on Identities

Recently I received an invitation to join the Georgia Tech Faculty Women’s Club. Ultimately I have decided not to, but the process of thinking the decision though caused me to reflect a lot on the multiple identities that I manage, and on how perhaps Georgia Tech (and other places) might consider women, and in particular, their multiple identities.

When I first got interested in Computer Science, I was the sole woman (girl) in the classes I took. At that time, recruiting any other woman to join me through female focused outreach mattered to me. But in deciding what to do about the faculty wives club, I was forced into a very valuable reflection on what matters to me now.

The history of the Faculty Women’s club is that it started as an association for the wives of GT faculty, although more recently it has replace the wives with women as part of a shift in their recruiting. I think there is an important role for the group in supporting women who do move with their husband’s career (whether they work or not, it is an adjustment to a new place, and the club seems to provide an important outreach, a place to make friendships). Further they do a variety of really important philanthropic work for the campus, raising money to support students and so forth.

But, while I qualify for the GT Faculty Women’s Club, I find that it conflicts with an identity I am trying very hard to manage in an entirely different way. And it’s the part of my identity which is that I am also a Faculty wife, as part of a dual body hire. At  work (i.e. Georgia Tech), I believe that it is my responsibility as a dual body to ensure (to the best of my ability) that my colleagues feel that they have hired two individuals.I also want to be treated as an individual professional actor (and I am). There is one time when I want the duality to be considered, and that’s when it could be a conflict of interest (I haven’t encountered any others yet). My challenge with the GTFWC is that its history (and I think to some degree its current membership composition) collides with how I want to manage my workplace identity. I realized that I was not willing to join the club because it would be a workplace-based identification with a piece of my biography that I mostly want to keep out of the workplace!

And its not the only identity conflict I’ve seen. For example, when faculty women’s groups take up issues related to childcare they equate woman with mother. And there are important discussions for parents to have, like about access to resources for child raising as part of employment. But Fathers are parents too, and there are many more of them on campus than their are women.

In other words what I am saying is that “woman” is very broad category, too broad. And, OK, this is not terribly surprising, but why don’t I see an explosion of other sorts of groups promoting categories, like say parenthood? Why don’t we continue to focus more particularly on groups within a single category, like wives?

Concerns about the Omission of HCI in Impacts of CS “Tire Tracks” Diagram

Recently, the “tire tracks” diagram of how Federally funded research has led to impact on the Computing industry and American experience was updated. Appreciative of the work that it does to continue to make the case for basic research in Computer Science, I was keen to see it. Imagine my disappointment when I realized that HCI was not present.

Maybe HCI is omitted because the “tire tracks” diagram focuses on product and not business practice outcomes. One major impact of HCI on industry is User Experience design. Don Norman and his team at Apple first popularized that term, in a paper they wrote about Apple’s User Experience (UX) practice, the research that continued to inform those corporate practices, writing it up for an HCI conference, the ACM’s Conference on Human Factors in Computing Systems (CHI). Now consider, Google’s focus on keeping it simple. I’ve asked UX professionals and global estimates (based in part on membership in professional societies) are as high as 44,000 people who work in the field (another estimate was that about 10K of those people work in the United States). That’s a pretty significant type of employment impact!

Returning to the product-oriented outcomes, Brad Myers has done an eloquent job of explaining the role of HCI in the software and systems we take for granted today. He has even produced tire tracks. His diagrams suggest to me that HCI may have gotten folded into Computer Graphics in the CRA’s version of “tire tracks”. But, to omit explicit recognition of HCI is to preclude innovations that came from explicit consideration of humans, whether it be as information processors or soci0-technical beings. Yes, sure, the product innovations happened in the graphical user interface and how it is manipulated and interacted with, but reading the history of innovations at that interface it’s clear that the inspiration and the basic science that was drawn on included science about people, and developing a basic science of the interaction between people and machines. One particularly fruitful area for development that is of interest within HCI and appears to be missing from the tire tracks is input devices and techniques…

PARC features in some of the industrial work, and it is worth noting that the culture of PARC was to have a focus on the relationship between people and machines. J.C.R. Licklider’s Introductory Note to an anthology of papers about Computers and Systems in the book about the first decade of PARCs research: A Decade of Research: Xerox Palo Alto Research Center 1970-1980 captured this as follows:

In my opinion, the interaction between people and computers is the most exciting new frontier of our time; here, in these pages, are many superb contributions by colleagues who not only share the excitement of that frontier but are developing the new land and building a new way of life in it.

These papers are an eloquent testimoney to the fact that, in the ten years since it’s conception, the Palo Alto Research Center has flourished. In that short time, many computer scientists, I among them, have come to consider it as the leading center of research in interactions between (or, in 1960’s terms, the symbiosis of) human beings and digital computers. …

The science and technology of the human use of digital computers are being created right now. p3

I’m disappointed that as we continue to chart this frontier of novel people machine interactions that the role of HCI has been omitted by the people making the case for the future of our science. I think Computer Science will come up short without taking people seriously. After all, in the marketplace we can ask whether the truly successful technologies are just a matter of their hardware and software, or is it that they deliver something that people want, desire, find useful and useful, and connect them to others.

Metrics: Numbers and Processes

Surprise, surprise, another post about metrics. Its not just the numbers themselves that can be problematic, but here’s a recent encounter with the processes used to compute the numbers.

Some time ago, Paul Krugman wrote this:

As I see it, the economics profession went astray because economists, as a group, mistook beauty, clad in impressive-looking mathematics, for truth. (Full article).

I love this phrase, and I tend to concur with him. Perhaps its because he has a Nobel Prize in Economics (2008).

But, I don’t think you even need to have very impressive-looking mathematics. Perhaps in Economics, perhaps to justify research outcomes the nature of the mathematics matters. Not, however, for a variety of metrics.

I was recently asked how I computed my h-index. While many people use tools, I compute mine manually. The mathematics of this is not complicated, although the truth is.

R. Grinter and R. Grinter are two different people. Some of my citations omit the E. that I use (Rebecca E. Grinter). This can, when not caught, lead to grade inflation. One way to catch it is by looking carefully for any focused on spectrosopy. But, even taking those rogues out you can see the consequences of interdisciplinarity on the h-index. Many of the tools let you refine by discipline, but while that removes most of the Chemistry, it doesn’t catch the interdisciplinary pieces that the other R. Grinter worked on. More over, if you get too restrictive, R. E. Grinter and of course my nemesis mis-cite, R. Grinter get lost categorised into other disciplines. Finally, there’s my nick-name. Beki. Happy to be Beki, but people will pick me up as B. Grinter. So, I usually need to pay attention to that. In retrospect I wish I’d just started with B actually, because as far as I can tell there are no other B Grinter’s currently research active.

So that’s why I compute my h-index manually, I do to provide a truth, one that reflects my awareness of the flaws associated with the tools. But, by computing manually I’ve separated my h-index from the one computed by tools, and thus to compare mine to others requires computing all the h-indexs manually. And comparing is frequently at the heart of metrics. And this to me seems like another fine example of the problem of numbers as truth. In the process of trying to supply truth (an accuracy through manual computation) I’ve simultaneously taken away another type of truth, one that is comparative based on using the same process to compute that “truth.”

MOOCS: Another To-Do in a Culture of Busyness

My institution, Georgia Tech, recently signed up as a participant institution in the Coursera MOOC. This, and other events, have caused a lot of discussion about MOOCS. I won’t rehash them all here, but I will point to Mark and Ian’s blog posts about learning and the assertions made both explicitly and implicitly made.

I was doing my readings about MOOCs while reading another book, an ethnography of the contemporary LA middle class family. It’s a rich portrait of the lives of 32 families specifically, and a reflection on our lives at home. And this led me to another question I have about MOOCS. When, exactly, are we supposed to find the time for the courses?

It’s not just the faculty instructors who have to take on workload to prepare the course (at Georgia Tech this is extra workload not associated with or balanced against our other responsibilities). But, what about those who take the classes. This imagined modern American learner likely has many things to juggle, a job (or multiple jobs), a family, and so forth. As the book makes painfully clear, the average American has very little time left to spend on anything, let alone education. So, how are they supposed to cram in a 6 week course and assignments on top of all the other responsibilities?

I wonder whether the high rates of dropping out can be partially explained by people wanting to learn, feeling that education and self-improvement (which is a big industry marketing its own life-long worth) matter, but then it turns out to be impossible to juggle the programming assignment with their children’s homework and extra-curricula activities. If, as this book claims, Americans are down to meals that last less than 30 minutes and don’t feature all the family, because they are so pressed for time, where does a MOOC come into that equation?

On Not Leaving Academia

An article written by a faculty member who is leaving academia for Google has been circulating on Facebook. I agree in parts with what he is saying, but I am not going to leave academia (I should add that I am a tenured professor, like—I believe—the author of the original piece). Here, for what its worth, are my thoughts. I’ve organized them around some of the same sub-headings.

Opportunity To Make A Difference

He wants to make a difference and I can really understand that. Despite the enthusiasm that many Universities have to make things out of ideas, it is hard to start from here and get to there. But, my experience of working in two corporations (I was in the research division, but tasked, particularly at Lucent, to change things in the commercial side of the organization) is that its really difficult to affect change in organizations. Perhaps I picked the wrong things, but a colleague and I spent multiple years implementing an approach to collecting metrics that we thought was sane (as opposed to what they had, which wasn’t). We had the blessing of and support from the CTO which is a huge starting advantage (when you show up for meetings you can’t so easily be ignored—I have a whole other story about that).

There is one really visceral opportunity to make a difference in the University. Students. I’m looking at a stack of cards that students have written to me over the years to thank me for classes and advisement. To be there in someone’s life, to make a difference, to have that person write to you (by hand, on notepaper, mailed) humbles me to this day. I love the emails too, but truthfully its the cards or the students who nominate me for “favorite professor” or some such that really move me because of the time and effort they took to tell me something positive.

Work And Family/Life Balance

He wants a better work/life balance. I wonder if that’s true for everyone who is working.

I also wonder a challenge with a work-life balance is the constant discussion about it. I feel that the way arguments about work-life balance are often constructed are part of the problem. For example, the work-life balance discussion often seems to imply that the solution is with the individual, you must change it. But as the author notes, thats not at all true. Our work-life balance is also made all around us, through legislation (and the lack of it) about holidays and leave, and through the culture of the importance of work in this country.

One of the more emancipating moments I had in my own work-life balance was realizing that some of it is just out of my hands. Not just because its been set for me by other institutions, but also because my work depends on others, and sometimes the coordination is just really bad (who knew that these two commitments were going to collide in this way at that time, but others are depending on me). Accepting that its not something we can “solve” just for us, as if we are isolated from the employers, country, and colleagues, has been uplifting in my own thinking about work-life balance. I’ve gone from thinking that I’m a doomed work-life screw up to realizing that it’s an ongoing balancing project.

Centralization Of Authority And Decrease Of Autonomy

The rise of the number of administrators in the American University has been the subject of much discussion. I feel that that increase is happening at Georgia Tech although exact numbers are hard to get a hold of. I am not sure whether its led to a direct decrease in my autonomy though. In no small part because while it’s happened I’ve moved from untenured, to tenured, and from Associate to Full Professor. That is remarkably empowering.

I’m also lucky that I’ve interacted with a number of senior administrators, and while I don’t necessarily agree with all their decisions (excepting, of course, for the ones to promote me I am confident that they have Georgia Tech’s best interest at heart and they are trying very hard to do what they believe to be right. Perhaps if I had a message here it would be to encourage all administrators to give time to the faculty, to try to interact with them enough that these sorts of relationships can be forged.

Funding Climate

The funding climate makes me sad too, it’s one reason I wanted to vote, and naturalized in order to do so. I agree with him that there is pressure on faculty to raise more resources, from broader variety of sources. And at least here at Georgia Tech I wish we had more help from the Institute to develop these new streams.

But, while this does suck I was reminded of other utterly dreadful things I’ve done in corporations. Patent maps, marketing analyses (yes, me). Horrible. And when I think about funding, and particularly the chase, I try to remind myself that yeah, every job has something that’s no fun. The upside to pursuing funding is that grant writing is a very intense process of thinking which I find enjoyable. So, its not all bad.

Risk in Research

His argument here is that research has gotten more conservative, and that’s an argument I also hear. But that’s actually a very hard thing to know. Personally, while I feel I’ve not been risk-averse, I have written grants to look less risk-averse. How others see my risk-averseness I couldn’t say. So, how can we tell? We can assess each other, would we use the same definition of risk. Perhaps we point back to a time when it seemed like there was more risk and result, but there are still (at least in my field) many inspiring researchers who are taking risks, and who are wonderfully creative.

Poor Incentives

The need to publish more and more, and procure ever increased amounts of funding (during a time of decreasing support) are not good incentives for research. They don’t encourage activities that don’t lead to those outcomes quickly. I share this view. And I have more to add.

Its clear that Universities want to encourage a diversity of career paths, for example, emphasizing impact on industry or the creation of start ups. What’s not so clear is how that matches the processes by which we are evaluated. Tenure and Promotion processes turn on a committee seeking letters from academics at other institutions about the importance of the person’s scholarship and teaching. How are academics at other universities going to write about a start up? Do they even know? If they do, what are the criteria by which you describe the impact? (Metrics are one criteria, but they can be very problematic. I’ve written about the h-index before, others I’ve seen include number of users or downloads, but what you actually mean is number of consumers because how does anyone know who is using a piece of software?)

The point I want to return to here is that there is a serious incentive mis-match between what is desired and what is evaluated. You can not change one without changing the other effectively, at best the change won’t happen, at worst people will be cruelly disappointed. And when you consider that change, not only do you want to consider both the front (encourage) and back ends (evaluation) you also want to ask what you take off the table—what’s not going to happen if I work on this, expecting people to do both is also a problem.

Is this different in corporations, I’m not so sure, I think this is the academic version of we’re all doing more in the job today than we were a decade ago. Also, worth noting that we have jobs. In this economic climate that’s a privilege.

Mass Production Of Education

He’s worried about MOOCS and I agree. I view them as the latest technology being subjected to utopian rhetoric about the possibilities without examining them carefully. Also, its not new, the School of the Air in Australia or the Open University in the United Kingdom have been teaching in this fashion for decades. Perhaps the technology has changed, but the years of experience that has already accumulated suggest that there’s a lot of unaccounted for work in this current rhetoric. My colleague Ian has another fabulous critique.

And finally, the most important reason why I won’t leave academia is the colleagues I have. In the invisible college they encourage and support the research that I do. In the local college they do that and also mentor and advise, they inspire and engage, and they make me laugh at times (always necessary in the workplace). And then there’s the students, from whom I learn, and who trust me with their future. They laugh (sometimes with me, sometimes perhaps at me) and when they smile, it’s a good thing. All that and a salary, it’s still pretty good.