Sharing Instruments: SMS Logging

This is a second post sharing instruments to help others with their empirical research.

One of my most cited papers is “y do tngrs luv 2 txt msg?” which was a study that I did with Marge Eldridge when we both worked at EuroPARC in Cambridge, UK. What interested us both was how rapidly text messaging had been adopted by teens. What were they using it for? Why?

In the spirit of making more of my materials available I wanted to share the diaries that we asked the teens to keep. There are short excerpts in the paper, but here they are in full. I should say that we were trying to balance portability and privacy against collecting the type of data that would allow us to gain insight into how the technology was being used. This is why the diaries look the way they do.

We asked them to log all the messages that they sent and received, and provided instructions for how to use both the sent and received forms.

12 does not equal Theoretical Saturation

Since I’ve got a Grounded Theory focus right now, there’s something else I want to clear up.

12 does not equal theoretical saturation. Full theoretical development leads to theoretical saturation. And that is, of course, the stopping point for Grounded Theory research.

In my own experience, it was approximately 6 months in one field site, where I conducted approximately 200 interviews (mostly without a guide) and then visits to a number of other field sites. At these sites, I added another nearly 100 interviews, and the hours of observation in total are still measurable in months. In the end I visited seven different companies, although in my thesis I wrote about just three. At the seventh and last company, I heard nothing new with respect to my theory (I heard other things that were new but they concerned issues not relevant to the explanation I was attempting to build).

Since I was studying the relationship between technical and human dependencies in software development, it seemed crucial that I sample among different types of development, so I looked at companies who contracted, those who worked in a monopsony market, others who sold their software in the commercial marketplace. I wwanted to understand whether the market conditions had influence on my theory. I also tried to sample across size of company, start up small and growing to large stable organizations. Did size matter in coordination? I sampled across companies that built exclusively software for commercially available platforms, those that built on non-commercially available platforms, and those who built hardware. Where there differences based on the relationship to hardware, and did building hardware itself have any affect? Finally, I tried to get different types of product. Systems built for real-time operation, those for high reliability, others to address perceived or real consumer needs. In other words, to see whether the type of ode base and the prevailing concerns about its nature influenced my theory.

Throughout the six months at the field site, and throughout the remainder of the scheduling, visiting and meeting people at the other six sites, I conducted analysis. Data collection iterated with analysis. How many rounds did I do, I can’t even tell you. At first, I felt lost and bewildered, what on earth was I doing. Analysis generated more questions. Over time, the questions got more focused, and so the rounds of analysis and collection begun to converge and over time I got fairly specific questions.

I had gone in with a question about how software tools, specifically configuration management, structure the coordination of work that has an intangible quality i.e. software. Grounded Theory seemed like a good fit. First, I’d read a number of pieces about Articulation Work and knew that it was derived from Grounded Theory. So, thanks to Strauss I would be able to leverage the products of that theorising to give me direction in the form of a plan for my research questions, my interview questions, and some ideas of what I might find in my analysis and even some extra concepts to work with during my analysis (I looked for things that were similar, which is not hard given the nature of Grounded Theory analyses anyway).

There are other reasons, non Grounded Theory reasons to conduct research that may involve less empirical data than I collected. You may be evaluating a deployment (perhaps baselining and then evaluating). My point here is that that’s different from Grounded Theory, and should be treated as such, explicitly. As a colleague of mine says, when reading Grounded Theories, they always want to know what the theory is. If you don’t have one, how does it qualify.

Grounded Theory

Right, this has been coming for a while.

Grounded Theory is not an excuse to go out and study something when you have no idea what’s going to happen. That’s just madness.

Stepping back. Sometimes I hear that Grounded Theory allows you to go into the field, collect data, and only develop questions during analysis. That’s the part that worries me. Research is very expensive, not just financially, but far more importantly, in terms of time hence the madness described above.

So lets clear some some misconceptions.

1) It’s impossible not to have research questions. Perhaps they are not very well formed one (this is something people could easily say of me, I tend to work by instinct as much as by questions), but it’s pretty important to have questions—a sense that something is of interest. I’d go further though, I think it’s impossible not to have a particular set of hopes and interests, and even desires for the outcomes. Grounded Theory suggests that you capture these prior to going into the field. These are valuable resource and an important check (to check to the extent possible that you’re not leading the analysis towards the assumptions that you had before going in.

2) If you interview someone you almost certainly have to have expressable questions. Just saying.

So, Grounded Theory is a balance between exploring the data, and being open to developing new lines of questioning based on previously ill-understood, not-understood phenomena captured in the data.

Now, I also think Grounded Theory is tempting because it comes with a series of steps. Open coding, axial coding, and selective coding suggest that if appropriately followed a Grounded Theory will result. Many other intepretivist approaches do not come with those steps. Instead the reader has to pay close attention to the theory that drives the empirical work. One has to understand why, say, accounts matter to the ethnomethodological agenda, and then understand that the study of phenomena is likely drive in part to further illuminate the concept with respect to the particular setting.

If you want an example of something that also has a “steps” like feel to it, but is not Grounded Theory, try the Thinking Topics approach by Lofland and Lofland.

And if you want to understand the theory of Grounded Theory, try reading The Discovery of Grounded Theory.

So, what I am about to say is open to discussion (as if the rest is not 🙂 but its open to debate about whether and how much Grounded Theory is driven by data. Google if you will.

I have reasons to believe that it’s not entirely driven by the data, but that other factors come into play. First, you can structure grounded theory using any other theory developed by the method. Strauss says so, although Glaser may disagree (most people are following Straussian Grounded Theory as opposed to Glaserian Grounded Theory and the two differ).

Second, it seems to me that the questions you ask of the data during Straussian coding suggest certain types of outcome. The analysis, and the things that it sought to explain tends to have a temporal quality, it promotes an understanding of an arc of time. Causes, consequences, who did what to whom. All very temporal indeed. Many of the Grounded Theories I’ve read explore trajectories of work, of people interacting and acting towards an outcome (whether predicted or not). If you read enough of them you begin to get a feel for some commonalities, what they may work well at explaining. And since you’re not reading any related work (oh yes you should) you start to get a feel for the occasions when Grounded Theory might be most useful.

Third, surely the process of data collection in dispersed with analysis is also the reason that Grounded Theory is not entirely data driven. Data collection that follows a period of analysis must surely be driven by analytic concerns as well as data concerns. Gaps in the analysis that need to be addressed fuel the generation of further questions. I don’t think you can do Grounded Theory, at least not completely on one round of data collection, there must be cycles of collection and analysis, collection and analysis. This is also my defense for knowing when the process ends, when the analysis is complete. When there is nothing else left to explain. Surely then, and only then, you have a Grounded Theory that you can say the following of: that it describes the the world, that it is rhetorically powerful by being clear and persuasive, that it has inferential power (if a similar phenomenon is encountered the theory helps understand what may result), and that it has application.

While I’m here, let me clear up something else. I don’t want to read a Grounded Theory that doesn’t present what the theory is a theory about. I have a theory in mind, it’s a theory about why the division of labor among software developers, despite the goals of modularity, leads to the creation of dependencies that must be coordinated in order for code to successfully compile and run. Further, this theory shows how organizational hierarchies create distance that exacerbates the types of dependencies that exist and their ability to be coordinated. It enumerates dependencies that exist among individuals, between groups and divisions of a corporation, and those that span multiple corporations, and offers strategies for their coordination.