4.5 Mixed-methods research

As we discussed in the previous sections, while we contrast idiographic vs. nomothetic explanations or inductive vs. deductive reasoning, the truth is that researchers combine both of these approaches when they conduct research. While these processes can occur in any kind of study, mixed methods research is an excellent example of how researchers use both approaches to logic and reasoning to improve understanding of a given topic.

So far in this textbook, we have mostly focused on quantitative and qualitative methods as an either/or choice—you can choose quantitative methods or qualitative methods. However, researchers often use both quantitative methods inside of their research projects. This is called mixed methods research.

For example, I recently completed a study with administrators of state-level services for people with intellectual and developmental disabilities. They implemented a program called self-direction, which allows people with disabilities greater self-determination over their supports. In this study, my research partners and I used a mixed methods approach to describe the implementation of self-direction across the United States. We distributed a short, electronic questionnaire and conducted phone interviews with program administrators. While we could have just sent out a questionnaire that asked states to provide basic information on their program (size, qualifications, services offered, etc.), that would not provide us much information about some of the issues administrators faced during program implementation. Similarly, we could have interviewed program administrators without the questionnaire, but then we wouldn’t know enough about the programs to ask good questions. Instead, we chose to use both qualitative and quantitative methods to provide a more comprehensive picture of program implementation.

Sequence and emphasis

There are many different mixed methods designs, each with their own strengths and limitations (see Creswell & Clarke, 2017^[1] for a more thorough introduction). However, a more simplified synthesis of mixed methods approaches is provided by Engel and Schutt (2016)^[2] using two key terms. Sequence refers to the order that each method is used. Researchers can use both methods at the same time or concurrently. Or, they can use one and then the other, or sequentially.

For our study of self-direction, we used a sequential design by sending out a questionnaire first, conducting some analysis, and then conducting the interview. We used the quantitative questionnaire to gather basic information about the programs before we began the interviews, so our questions were specific to the features of each program. If we wanted to use a concurrent design for some reason, we could have asked quantitative questions during the interview. However, we felt this would waste the administrators’ time looking up information and would break up rhythm of the interviews.

The other key term in mixed methods research is emphasis. In our mixed methods study, the qualitative data was the most important data. The quantitative data was mainly used to provide background information for the qualitative interviews, and our study write up focused mostly on the qualitative information. Thus, qualitative methods were prioritized in our study. Of course, many other studies emphasize quantitative methods over qualitative methods. In these studies, qualitative data are used mainly to provide context for the quantitative findings.

For example, demonstrating quantitatively that a particular therapy works is important. By adding a qualitative component, researchers could find out how the participants experienced the intervention, how clients understood the therapy’s effects, and the meaning the therapy had on their lives. This data would add depth and context to the findings and allow researchers to improve the therapeutic technique in the future.

A similar practice is when researchers use qualitative methods to solicit feedback on a quantitative scale or measure. The experiences of individuals allow researchers to refine the measure before they conduct the quantitative component of their study. Finally, it is possible that researchers are equally interested in qualitative and quantitative information. In studies of equal emphasis, researchers consider both methods as the focus of the research project.

Why researchers use mixed methods

Mixed methods research is more than just sticking an open-ended question at the end of a quantitative survey. Mixed methods researchers use mixed methods for both pragmatic and synergistic reasons. That is, they use both methods because it makes sense with their research questions and because they will get the answers they want by combining the two approaches.

Mixed methods also allow you to use both inductive and deductive reasoning. As we’ve discussed, qualitative research follows inductive logic, moving from data to empirical generalizations or theory. In a mixed methods study, a researcher could use the results from a qualitative component to create a theory that could be tested in a subsequent quantitative component. The quantitative component would use deductive logic, using the theory derived from qualitative data to create and test a hypothesis. In this way, mixed methods use the strengths of both inductive and deductive reasoning. Quantitative allows the researcher to test existing ideas. Qualitative allows the researcher to create new ideas.

With these two concepts in mind, we can start to see why researchers use mixed methods in the real world. I mentioned previously that our research project used a sequential design because we wanted to use our quantitative data to shape what qualitative questions we asked our participants. Mixed methods are often used this way, to initiate ideas with one method to study with another. For example, researchers could begin a mixed methods project by using qualitative methods to interview or conduct a focus group with participants. Based on their responses, the researchers could then formulate a survey to give out to a larger group of people to see how common the themes from the focus groups were. This is the inverse of what we did in our project, which was to use a quantitative survey to inform a more detailed qualitative interview.

In addition to providing information for subsequent investigation, using both quantitative and qualitative information provides additional context for the data. For example, in our questionnaire for the study on self-direction, we asked participants to list what services people could purchase. The qualitative data followed up on that answer by asking whether the administrators had added or taken away any services, how they decided that these services would be covered and not others, and what problems that arose around providing these services. With that information, we could analyze what services were offered, why they were offered, and how administrators made those decisions. In this way, we learned the lived experience of program administrators, not just the basic information about their programs.

Finally, another purpose of mixed methods research is to corroborate data from both quantitative and qualitative sources. Ideally, your qualitative and quantitative results should support each other. For example, if interviews with participants showed an association between two concepts, that association should also be present in the qualitative data you collected. Differences between quantitative and qualitative data require an explanation. Perhaps there are outliers or extreme cases that pushed your data in one direction, for example.

In summary, these are a few of the many reasons researchers use mixed methods:

1. Triangulation, or convergence on the same phenomenon to improve validity
2. Complementarity, which aims to get at related but different facets of a phenomenon
3. Development, or the use of results from one phase or a study to develop another phase
4. Initiation, or the intentional analysis of inconsistent qualitative and quantitative findings to derive new insights
5. Expansion, or using multiple components to extend the scope of a study (Burnett, 2012, p. 77).^[3]

A word of caution

The use of mixed methods, while having many advantages, is labor intensive. You must conceptualize a component using quantitative methods, another using qualitative methods, and think about how they fit together. This may mean creating a questionnaire, then writing an interview guide, and thinking through how the data on each measure relate to one another—more work than using one quantitative or qualitative method alone. Similarly, in sequential studies, the researcher must collect and analyze data from one component and then conceptualize and conduct the second component. This may also impact how long a project may take. Before beginning a mixed methods project, you should have a clear vision for what the project will entail and how each methodology will contribute to that vision. Always remember that you should make your project feasible enough for you to conduct with the time, money, and other resources you have at your disposal right now.