I was reading a paper when I came across the following sentence…
We intentionally biased our data sample in terms of type and size
There’s so much going on here but lets start with the high order bit that saying this in a paper might as well be accompanied with the following sentence
Please reject this now.
Lets start out with the statement, what sounds good about intentionally biasing our data sample in the following ways? Well I did have one thought, it’s better than unintentionally biasing it (which just seems careless). No the authors knew what they were doing. And, also a plus, they admitted it in case the reviewers didn’t know what they were doing. Whoo hoo, I would have given up as a reviewer and just written up “The authors admit that they’ve conducted a flawed experiment.”
Moving past the idea that the authors are flaunting the rules of experimental design this phrase raised other questions.
The paper in which I found this sentence was a qualitative piece of work. For example, one question, what is bias in qualitative sampling? In fact, some forms of sampling are the pursuit of quite intentional people. People with a particular expertise for example. (Imagine your three cycles into your Grounded Theory and you have some very particular questions that only a few people in the corporation you’ve been studying can answer because it falls within their job responsibilities, well then you’re either going to select these people to interview or you are going to waste a lot of time trying not to be selective in who you select to talk too).
Questions about size can be complicated as well. Size often suggests a numerical size but as I’ve said before, 12 does not equal theoretical saturation (tip: having a fully worked out theory does).
Behind these questions lie a type of care with terminology. The authors talk about data samples, bias, type and size, often words applied to experimental design. These are not the right ways to talk about qualitative research. Sure you want to talk about who you interviewed or observed, your participants, but they are rarely a data sample, they are the people that led you to the collection of a particular set of data… the logic of who they are is not about sampling from a population to ensure coverage, but about selecting people who can help develop the theory or analysis, and the size is dependent on different ways of determining completion.
I know that this sentence was thoughtlessly written. It was honest, but it sets up the reviewers in a variety of ways as I hope I’ve pointed out. And if I have to put it crassly, don’t use experimental terms to write about non-experimental ways of conducting empirical research. It’s just ugly.