I’m attending the HCIC (Human Computer Interaction Consortium) conference. During a session today I learnt about a challenge that Systems HCI researchers face when publishing the results of their work. They seem to feel based on their experience that it’s hard to publish more than one paper about any system. Given that developing a system can take 1-2 years to build, this is a problem. Indeed, it is suggested that this is discouraging people from focusing on Systems contributions in HCI.
Since I do not work in this space, I was quite surprised and disappointed. I was surprised, because I similarly feel that I can find people who feel that it’s hard to publish the results of a study more than once (not the same ones obviously). And like systems, I can find people who would tell me that the contributions of a piece of empirical research can not be put into a single paper, but when it is written about more than once people are concerned that it is a repeat of the previous research. Further, I would add that in both instances people were also not focused on publishing the least publishable unit (LPU).
My disappointment is obvious. It seems like we’re focusing on the wrong things if this is going on. Rather than focusing on the contributions that the piece of work makes to the community we’re focused on the unit of research itself (a field study, a system). I know that’s a statement of the obvious, and yet, here we are having conversations about how we perceive the review process to be working, potentially quite pervasively (since it spans a number of areas within the field and multiple people within each).
A related concern that cropped up during the discussion was the question of how systems work is evaluated. I was reminded of Paul Dourish’s Implications for Design paper. In that paper, Paul argues for broadening the criteria by which Ethnography is evaluated, what contributions it might make, why they may not be design (or even make sense to be design recommendations). I wonder whether it’s time for a contribution to the discourse about how systems HCI research might be evaluated. I don’t review Systems HCI papers very often, but I want to hear more about how best we learn from a wide scope of contributions.