Beki Grinter

12 does not equal Theoretical Saturation

In discipline, empirical, HCI, research on September 1, 2010 at 9:18 am

Since I’ve got a Grounded Theory focus right now, there’s something else I want to clear up.

12 does not equal theoretical saturation. Full theoretical development leads to theoretical saturation. And that is, of course, the stopping point for Grounded Theory research.

In my own experience, it was approximately 6 months in one field site, where I conducted approximately 200 interviews (mostly without a guide) and then visits to a number of other field sites. At these sites, I added another nearly 100 interviews, and the hours of observation in total are still measurable in months. In the end I visited seven different companies, although in my thesis I wrote about just three. At the seventh and last company, I heard nothing new with respect to my theory (I heard other things that were new but they concerned issues not relevant to the explanation I was attempting to build).

Since I was studying the relationship between technical and human dependencies in software development, it seemed crucial that I sample among different types of development, so I looked at companies who contracted, those who worked in a monopsony market, others who sold their software in the commercial marketplace. I wwanted to understand whether the market conditions had influence on my theory. I also tried to sample across size of company, start up small and growing to large stable organizations. Did size matter in coordination? I sampled across companies that built exclusively software for commercially available platforms, those that built on non-commercially available platforms, and those who built hardware. Where there differences based on the relationship to hardware, and did building hardware itself have any affect? Finally, I tried to get different types of product. Systems built for real-time operation, those for high reliability, others to address perceived or real consumer needs. In other words, to see whether the type of ode base and the prevailing concerns about its nature influenced my theory.

Throughout the six months at the field site, and throughout the remainder of the scheduling, visiting and meeting people at the other six sites, I conducted analysis. Data collection iterated with analysis. How many rounds did I do, I can’t even tell you. At first, I felt lost and bewildered, what on earth was I doing. Analysis generated more questions. Over time, the questions got more focused, and so the rounds of analysis and collection begun to converge and over time I got fairly specific questions.

I had gone in with a question about how software tools, specifically configuration management, structure the coordination of work that has an intangible quality i.e. software. Grounded Theory seemed like a good fit. First, I’d read a number of pieces about Articulation Work and knew that it was derived from Grounded Theory. So, thanks to Strauss I would be able to leverage the products of that theorising to give me direction in the form of a plan for my research questions, my interview questions, and some ideas of what I might find in my analysis and even some extra concepts to work with during my analysis (I looked for things that were similar, which is not hard given the nature of Grounded Theory analyses anyway).

There are other reasons, non Grounded Theory reasons to conduct research that may involve less empirical data than I collected. You may be evaluating a deployment (perhaps baselining and then evaluating). My point here is that that’s different from Grounded Theory, and should be treated as such, explicitly. As a colleague of mine says, when reading Grounded Theories, they always want to know what the theory is. If you don’t have one, how does it qualify.

  1. the last couple of notes seem to be reacting to something but i can’t quite tell what it is. where on earth did “12” come from?

    i can’t imagine there are too many people reading your blog claiming to do full-on grounded theory research. perhaps for the same reason that your example here dates back to your own graduate career: that for a typical under-resourced HCI/CSCW project, it’s too heavyweight when you consider the long list of other “applied” things that need to be done as well. (the same is probably true for the full-on application of nearly any approach you’d use for an applied social science dissertation.) you certainly suggest that above.

    i think your blog reader might miss the pragmatic forest for the rigorous trees, though. i sometimes throw charmaz’ introductory book on grounded theory (i find the strauss/glaser books to be practically unreadable by comparison) at junior researchers because it offers two things: (1) a relatively generic and incrementally adoptable toolbox for looking at data (you alluded to this earlier) *with* some “worked examples,” and (2) a relatively agnostic view on incrementally pulling in, well, ungrounded theory. i am constantly surprised at how many “post-coursework” students are stymied by having never actually applied anything more systematic than contextual inquiry to analyzing data, and are intimidated by the idea of digging into deeper issues than what you get out of contextual inquiry because they haven’t been reading X-ian/X-ist/X-ological theory for years like some of their social science-y peers. both (1) and (2) lend themselves to learning while doing. i would guess you do something similar in your methods class and with your students, for essentially the same reasons.

    so here’s the constructive question: what would you like to call that bastardized process? sure, one should not say you’re developing grounded theory. my thought would be “applying methods influenced by / derived from / adapted from grounded theory research.” thoughts?

  2. It’s a long story 🙂

    I don’t really know to answer your question at the end, but you are right… the contexts in which we largely operate are not appropriate for Grounded Theory (although it is nice to have had at least one opportunity to do it 🙂 … Quick and Dirty Grounded Theory? Perhaps not. I think what’s more important at a number of levels is to realise that Grounded Theory is not walking into a setting with no idea what’s going on, or thinking that 12 interviews will be enough to claim it. I’d probably settle for calling it a qualitative interview study with the data being analysed following the methods of… but say that it’s not a full grounded theory. The problem with not being candid is that I think we’ve potentially created confusion about what Grounded THeory is, how theoretical saturation works (or not) and the only people who come off looking bad are those attempting to practice the method, because it now looks slipshod and easy.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: