1.4 The scientific method

Learning Objectives

Learners will be able to…

  • Define science and social science
  • Describe the differences between objective truth and subjective truths
  • Identify how qualitative and quantitative methods differ and how they can be used together
  • Delineate the features of science that distinguish it from pseudoscience

Pre-awareness check (Emotion)

How confident are you in your ability to apply qualitative and quantitative methods in your research?

 

Science

In social work, science is a way of ‘knowing’ that attempts to systematically collect and categorize facts or truths. A key word here is systematically—conducting science is a deliberate process. Scientists gather information about facts in a way that is organized and intentional, and usually follows a set of predetermined steps. Social work is not a science, but social work is informed by social sciencethe science of humanity, social interactions, and social structures. In other words, social work research uses organized and intentional procedures to uncover facts or truths about the social world. And social workers rely on social scientific research to promote change.

 

Science can be contrasted to  its impostor, pseudoscience. Pseudoscience refers to beliefs about the social world that are unsupported by scientific evidence. These claims are often presented as though they are based on science. But once researchers test them scientifically, they are demonstrated to be false. A scientifically uninformed social work practitioner using pseudoscience may recommend any number of ineffective, misguided, or harmful interventions. Pseudoscience often relies on information and scholarship that has not been reviewed by experts (i.e., peer review) or offers a selective and biased reading of reviewed literature.

An example of pseudoscience comes from anti-vaccination activists. Despite overwhelming scientific consensus that vaccines do not cause autism, a very vocal minority of people continue to believe that they do. Anti-vaccination advocates present their information as based in science, as seen here at Green Med Info. The author of this website shares real abstracts from scientific journal articles and studies but will only provide information on articles that show the potential dangers of vaccines, without showing any research that prevents the positive and safe side of vaccines. Green Med Info is an example of confirmation bias, as all data presented on the website supports what the pseudo-scientific researcher believes to be true. For more information on assessing causal relationships, consult Chapter 4, where we discuss causality in detail.

The values and practices associated with the scientific method work to overcome common errors in thinking (such as confirmation bias). First, the scientific method uses established techniques from the literature to determine the likelihood of something being true or false. The research process often cites these techniques, reasons for their use, and how researchers came to the decision to use said techniques. However, each technique comes with its own strengths and limitations. Rigorous science is about making the best choice, being open about your process, and allowing others to check your work. It is important to remember that there is no “perfect” study—all research has limitations because all scientific methods come with limitations.

Skepticism and debate

Unfortunately, the “perfect” researcher does not exist. Scientists are human, so they are subject to error and bias, such as gravitating toward fashionable ideas and thinking their work is more important than others’ work. Theories and concepts fade in and out of use and may be tossed aside when new evidence challenges their truth. Part of the challenge in your research projects will be finding what you believe about an issue, rather than summarizing what others think about the topic. Good science, just like good social work practice, is authentic. When we see students present their research projects, those that are the strongest deliver both passionate and informed arguments about their topic area.

Good science is also open to ongoing questioning. Scientists are fundamentally skeptical. As such, they are likely to pursue alternative explanations. They might question the design of a study or replicate it to see if it works in another context. Scientists debate what is true until they arrive at a majority consensus. If you’ve ever heard that 97% of climate scientists agree that global warming is due to human activity[1] or that 99% of economists agree that tariffs make the economy worse,[2] you are seeing this sociology of science in action. This skepticism will help to catch situations in which scientists who make the oh-so-human mistakes in thinking and reasoning reviewed in Section 1.1.

Skepticism also helps to identify unethical scientists, as with Andrew Wakefield’s study linking the MMR vaccination and autism. When other researchers looked at his data, they found that he had altered the data to match his own conclusions and sought to benefit financially from the ensuing panic about vaccination (Godlee, Smith, & Marcovitch, 2011).[3] This highlights another key value in science: openness.

Openness

Through the use of publications and presentations, scientists share the methods used to gather and analyze data. The trend towards open science has also prompted researchers to share data as well. This in turn enables other researchers to re-run, replicate, and validate analyses and results. A major barrier to openness in science is the paywall. When you’ve searched online for a journal article (we will review search techniques in Chapter 6), you have likely run into the $25-$50 price tag. Don’t despair—your university should subscribe to these journals. However, the push towards openness in science means that more researchers are sharing their work in open access journals, which are free for people to access (like this textbook!). These open access journals do not require a university subscription to view.

Openness also means engaging the broader public about your study. Social work researchers conduct studies to help people, and part of scientific work is making sure your study has an impact. For example, it is likely that many of the authors publishing in scientific journals are on Twitter or other social media platforms, relaying the importance of study findings. They may create content for popular media, including newspapers, websites, blogs, or podcasts. It may lead to training for agency workers or public administrators. Regrettably, academic researchers have a reputation for being aloof and disengaged from the public conversation. However, this reputation is slowly changing with the trend towards public scholarship and engagement. For example, see this recent section of the Journal of the Society of Social Work and Research on public impact scholarship.

Exploration, description, and explanation

Social science is a big place. Looking at the various empirical studies in the literature, there is a lot of diversity—from focus groups with clients and families to multivariate statistical analysis of large population surveys conducted online. Ultimately, all of social science can be described as one of three basic types of research studies: exploration, description, explanation, evaluation, and construction of measurement instruments. In this section we will discuss exploration, description, explanation. As you develop your research question, consider which of the following types of research studies fits best with what you want to learn about your topic. In subsequent chapters, we will use these broad frameworks to help craft your study’s final research question and choose quantitative and qualitative research methods to answer it.

 

Exploratory research

Researchers conducting exploratory research are typically at the early stages of examining their topics. Exploratory research projects are carried out to test the feasibility of conducting a more extensive study and to figure out the “lay of the land” with respect to the particular topic. Usually, very little prior research has been conducted on this topic. For this reason, a researcher may wish to do some exploratory work to learn what method to use in collecting data, how best to approach research subjects, or even what sorts of questions are reasonable to ask.

Often, student projects begin as exploratory research. Because students don’t know as much about the topic area yet, their working questions can be general and vague. That’s a great place to start! An exploratory question is great for delving into the literature and learning more about your topic. For example, the question “what are common social work interventions for parents who neglect their children?” is a good place to start when looking at articles and textbooks to understand what interventions are commonly used with this population. However, it is important for a student research project to progress beyond exploration unless the topic truly has very little existing research.

In my classes, I often read papers where students say there is not a lot of literature on a topic, but a quick search of library databases shows a deep body of literature on the topic. The skills you develop in  Chapter 6 should assist you with finding relevant research, and working with a librarian can definitely help with finding information for your research project. That said, there are a few students each year who pick a topic for which there is in fact little existing research. Perhaps, if you were looking at child neglect interventions for parents who identify as transgender or parents who are refugees from the Syrian civil war, less would be known about child neglect for those specific populations. In that case, an exploratory design would make sense as there is little, if any, literature about your specific topic.

Descriptive research

Another purpose of a research project is to describe or define a particular phenomenon. This is called descriptive research. Findings from descriptive research can include simple counts, or frequencies, of a characteristic within a sample or perhaps the prevalence or incidence of a phenomenon in a population.  They can also be more sophisticated and include results from correlational research, which investigates the associations between two or more variables of interest.

For example, researchers at the Princeton Review conduct descriptive research each year when they set out to provide students and their parents with information about colleges and universities around the United States. They describe the social life at a school, the cost of admission, and student-to-faculty ratios (to name just a few of the categories reported). If our topic were child neglect, we might seek to know the number of people arrested for child neglect in our community and whether they are more likely to have other problems, such as poverty, mental health issues, or substance use.

Social workers often rely on descriptive research to tell them about their service area. Keeping track of the number of parents receiving child neglect interventions, their demographic makeup (e.g., race, sex, age), and length of time in care are excellent examples of descriptive research. On a more macro-level, the Centers for Disease Control provides a remarkable amount of descriptive research on mental and physical health conditions. In fact, descriptive research has many useful applications, and you probably rely on such findings without realizing you are reading descriptive research.

Explanatory research

Lastly, social work researchers often aim to explain why particular phenomena operate in the way that they do. Research that answers “why” questions is referred to as explanatory research. Asking “why” means the researcher is trying to identify cause-and-effect relationships in their topic. For example, explanatory research may try to identify risk and protective factors for parents who neglect their children. Explanatory research may attempt to understand how religious affiliation impacts views on immigration. All explanatory research tries to study cause-and-effect relationships between two or more variables.

There are numerous examples of explanatory social scientific investigations. For example, Dominique Simons and Sandy Wurtele (2010)[4] sought to understand whether receiving corporal punishment from parents led children to turn to violence in solving their interpersonal conflicts with other children. In their study of 102 families with children between the ages of 3 and 7, the authors found that experiencing frequent spanking did in fact result in children being more likely to accept aggressive problem-solving techniques. Another example of explanatory research can be seen in Robert Faris and Diane Felmlee’s (2011)[5] research study on the connections between popularity and bullying. From their study of 8th, 9th, and 10th graders in nineteen North Carolina schools, they found that aggression increased as adolescents’ popularity increased.[6]

Exercises

TRACK 1 (IF YOU ARE CREATING A RESEARCH PROPOSAL FOR THIS CLASS)

  • Thinking about your research interests/topic, which type of research— exploratory, descriptive, or explanatory—best describes your working question?

TRACK 2 (IF YOU AREN’T CREATING A RESEARCH PROPOSAL FOR THIS CLASS)

  • Try writing a research question that fits each type of research (exploratory, descriptive, explanatory)

Science supported by empirical data

Pseudoscience is often doctored up to look like science, but the surety with which its advocates speak is not backed up by empirical data. Empirical data refers to information about the social world gathered and analyzed through scientific observation or experimentation. Theory is also an important part of science, as we will discuss in Chapter 7. However, theories must be supported by empirical data—evidence that what we think is true really exists in the world.

There are two types of empirical data that social workers should become familiar with. Quantitative data refer to numbers and [  id=”724″]qualitative data[/pb_glossary]  usually refer to word data (like a transcript of an interview) but can also refer to pictures, performances, and other means of expressing oneself. Researchers use specific methods designed to analyze each type of data. Together, these are known as research methods, or the methods researchers use to examine empirical data.

Objective truth

In our vaccine example, scientists have conducted many studies tracking children who were vaccinated to look for future diagnoses of autism (see Taylor et al. 2014 for a review). This is an example of using quantitative data to determine whether there is a causal relationship between vaccination and autism. By examining the number of people who develop autism after vaccinations and controlling for all of the other possible causes, researchers can determine the likelihood of whether vaccinations cause changes in the brain that are eventually diagnosed as autism.

In this case, the use of quantitative data is a good fit for disproving myths about the dangers of vaccination. When researchers analyze quantitative data, they are trying to establish an objective truth. An objective truth is always true, regardless of context. Generally speaking, researchers seeking to establish objective truth tend to use quantitative data because they believe numbers don’t lie. If repeated statistical analyses don’t show a relationship between two variables, like vaccines and autism, that relationship almost certainly does not exist. By boiling everything down to numbers, we can minimize the biases and logical errors that human researchers bring to the scientific process. That said, the interpretation of those numbers is always up for debate.

This approach to finding truth probably sounds similar to something you heard in your middle school science classes. When you learned about gravitational force or the mitochondria of a cell, you were learning about the theories and observations that make up our understanding of the physical world. We assume that gravity is real and that the mitochondria of a cell are real. Mitochondria are easy to spot with a powerful microscope, and we can observe and theorize about their function in a cell. The gravitational force is invisible but clearly apparent from observable facts, such as watching an apple fall. If we were unable to perceive mitochondria or gravity, they would still be there, doing their thing, because they exist independent of our observation of them.

Let’s consider a social work example. Scientific research has established that children who are subjected to severely traumatic experiences are more likely to be diagnosed with a mental health disorder (e.g., Mahoney, Karatzias, & Hutton, 2019).[7] A diagnosis of post-traumatic stress disorder (PTSD) is considered objective, and may refer to a mental health issue that exists independent of the individual observing it and is highly similar in its presentation across clients. The Diagnostic and Statistical Manual of Mental Disorders (DSM-5, 2017)[8] identifies a group of criteria which is based on unbiased, neutral client observations. These criteria are based in research, and render an objective diagnosis more likely to be valid and reliable. Through the clinician’s observations and the client’s description of their symptoms, an objective determination of a mental health diagnosis can be made.

Subjective truth(s)

For those of you are skeptical, you may ask yourself: does a diagnosis tell a client’s whole story? No. It does not tell you what the client thinks and feels about their diagnosis, for example. Receiving a diagnosis of PTSD may be a relief for a client. The diagnosis may suggest the words to describe their experiences. In addition, this diagnosis may provide a direction for therapeutic work, as there are evidence-based interventions clinicians can use with each diagnosis. On the other hand, a client may feel shame and view the diagnosis as a label, defining them in a negative way and limiting their potential (Barsky, 2015).[9]

Imagine if we surveyed people with PTSD to see how they interpreted their diagnosis. Objectively, we could determine whether more people said the diagnosis was, overall, a positive or negative event for them. However, it is unlikely that the experience of receiving a diagnosis was either completely positive or completely negative. In social work, we know that a client’s thoughts and emotions are rarely binary, either/or situations. Clients likely feel a mix of positive and negative thoughts and emotions during the diagnostic process. How they incorporate a diagnosis into their life story is unique. These messy bits are subjective truths, or the thoughts and feelings that arise as people interpret and make meaning of situations. Importantly, looking for subjective truths can help us see the contradictory and multi-faceted nature of people’s thoughts, and qualitative data allows us to avoid oversimplifying them into negative and positive feelings that could be counted, as in quantitative data. It is the role of a researcher, just like a practitioner, to seek to understand things from the perspective of the client. Unlike with objective truth, this will not lead to a general sense of what is true for everyone, but rather what is true in a particular time and place.

Subjective truths are best expressed through qualitative data, like conversations with a client or looking at their social media posts or journal entries. As a researcher, we might invite a client to tell us how they felt after they were first diagnosed, after they spoke with family, and over the course of the therapeutic process. While it may look different from what we normally think of as science (e.g. pharmaceutical studies), these stories are indeed a rich source of data for scientific analysis. However, it is impossible to analyze what this client said without also considering the sociocultural context in which they live. For example, the concept of PTSD is generated from Western thought and philosophy. How might people from other cultures understand trauma differently?

In the DSM-5 classification of mental health disorders, there is a list of culture-bound syndromes which appear only in certain cultures. For example, susto describes a unique cluster of symptoms experienced by Latin Americans after a traumatic event (Nogueira, Mari, & Razzouk, 2015).[10] Susto involves more physical symptoms than a traditional PTSD diagnosis. Indeed, many of these syndromes do not fit within a Western conceptualization of mental health because they differentiate less between the mind and body. To a Western scientist, susto may seem less real than PTSD. To someone from Latin America, their symptoms may not fit neatly into the PTSD framework developed in Western nations. Science has historically privileged knowledge from the United States and other nations in the West and Global North, marking them as objectively true. The objectivity of Western science as universally applicable to all cultures has been increasingly called into question as science has become less dominated by white males, and interaction between cultures and groups becomes broadly more democratic. Clearly, what is true depends in part on the context in which it is observed.

In this way, social scientists have a unique task. People are both objects and subjects. Objectively, you could quantify how tall a person is, what car they drive, how many adverse childhood experiences they had, or their score on a PTSD checklist. Subjectively, you could understand how a person made sense of a traumatic incident or how it contributed to certain patterns in thinking, negative feelings, or opportunities for growth, for example. It is this added dimension that renders social science unique to natural science (like biology or physics), which focuses almost exclusively on quantitative data and objective truth. For this reason, this book is divided between projects using qualitative methods and quantitative methods.

There is no “better” or “more true” way of approaching social science. Instead, the methods a researcher chooses should match the question they ask. If you want to answer, “do vaccines cause autism?” you should choose methods appropriate to answer that question. It seeks an objective truth—one that is true for everyone, regardless of context. Studies like these use quantitative data and statistical analyses to test mathematical relationships between variables. If, on the other hand, you wanted to know “what does a diagnosis of PTSD mean to clients?” you should collect qualitative data and seek subjective truths. You will gather stories and experiences from clients and interpret them in a way that best represents their unique and shared truths. Where there is consensus, you will report that. Where there is contradiction, you will report that as well.

Mixed methods

In this textbook, we will treat quantitative and qualitative research methods separately. However, it is important to remember that a project can include both approaches. A mixed methods study, which we will discuss more in Chapter 8, requires thinking through a more complicated project that includes at least one quantitative component, one qualitative component, and a plan to incorporate both approaches together. As a result, mixed methods projects may require more time for conceptualization, data collection, and analysis.

 

Finding patterns

Regardless of whether you are seeking objective or subjective truths, research and scientific inquiry aim to find and explain patterns. Most of the time, a pattern will not explain every single person’s experience, a fact about social science that is both fascinating and frustrating. Even individuals who do not know each other can create patterns that persist over time. Those new to social science may find these patterns frustrating because they may believe that the patterns describing their sex, age, or some other facet of their lives don’t represent their experience. It’s true. A pattern can exist among your cohort without your individual participation in it. There is diversity within diversity.

Let’s consider some specific examples. You probably wouldn’t be surprised to learn that a person’s social class background has an impact on their educational attainment and achievement. You may be surprised to learn that people select romantic partners that have similar educational attainment, which in turn, impacts their children’s educational attainment (Eika, Mogstad, & Zafar, 2019).[11] People who have graduated college pair off with other college graduates, as so forth. This, in turn, reinforces existing inequalities, stratifying society by those who have the opportunity to complete college and those who don’t.

People who object to these findings tend to cite evidence from their own personal experience. However, the problem with this response is that objecting to a social pattern on the grounds that it doesn’t match one’s individual experience misses the point about patterns. Patterns don’t perfectly predict what will happen to an individual person. Yet, they are a reasonable guide that, when systematically observed, can help guide social work thought and action. When we don’t investigate these patterns scientifically, we are more likely to act on stereotypes, biases, and other harmful beliefs.

A final note on qualitative and quantitative methods

There is not one superior way to find patterns that help us understand the world. As we will learn about in Chapter 7, there are multiple philosophical, theoretical, and methodological ways to approach scientific truth. Qualitative methods aim to provide an in-depth understanding of a relatively small number of cases. They also provide a voice for the client. Quantitative methods offer less depth on each case but can say more about broad patterns because they typically focus on a much larger number of cases. A researcher should approach the process of scientific inquiry by formulating a clear research question and using the methodological tools best suited to that question.

Believe it or not, there are still significant methodological battles being waged in the academic literature on objective vs. subjective social science. Usually, quantitative methods are viewed as “more scientific” and qualitative methods are viewed as “less scientific.” Part of this battle is historical. As the social sciences developed, they were compared with the natural sciences, especially physics, which rely on mathematics and statistics to come to a truth. It is a hotly debated topic whether social science should adopt the philosophical assumptions of the natural sciences—with its emphasis on prediction, mathematics, and objectivity—or use a different set of tools—contextual understanding, language, and subjectivity—to find scientific truth.

You are fortunate to be in a profession that values multiple scientific ways of knowing. The qualitative/quantitative debate is fueled by researchers who may prefer one approach over another, either because their own research questions are better suited to one particular approach or because they happened to have been trained in one specific method. In this textbook, we’ll operate from the perspective that qualitative and quantitative methods are complementary rather than competing. While these two methodological approaches certainly differ, the main point is that they simply have different goals, strengths, and weaknesses. A social work researcher should select the method(s) that best match(es) the question they are asking.

Key Takeaways

  • Scientific inquiry accounts for cognitive biases by applying an organized, logical way of observing and theorizing about the world.
  • Social work is informed by science.
  • Social science is concerned with both objective and subjective knowledge.
  • Social science research aims to understand patterns in the social world.
  • Social scientists use both qualitative and quantitative methods, which, while different, are often complementary.

Post-awareness check (Emotion)

How do you feel about applying qualitative and quantitative methods? Do you find yourself leaning toward one or the other?

Exercises

TRACK 1 (IF YOU ARE CREATING A RESEARCH PROPOSAL FOR THIS CLASS)

  • Examine a pseudo scientific claim you’ve heard on the news or in conversation with others. Why do you consider it to be pseudo scientific? What empirical data can you find from a quick internet search that would demonstrate it lacks truth?

  • Consider a topic you might want to study this semester as part of a research project. Provide a few examples of objective and subjective truths about the topic, even if you aren’t completely certain they are correct. Identify how objective and subjective truths differ.

TRACK 2 (IF YOU AREN’T CREATING A RESEARCH PROPOSAL FOR THIS CLASS)

You are conducting research on on-campus housing and supports for LGBTQIA+ college students:

  • Examine a pseudo scientific claim you’ve heard about LGBTQIA+ college students on the news or in conversation with others. What empirical data can you find from a quick internet search that would demonstrate it lacks truth?

  • Provide a few examples of objective and subjective truths about on-campus housing and supports for LGBTQIA+ college students, even if you aren’t completely certain they are correct. Identify how objective and subjective truths differ.

  1. See: https://climate.nasa.gov/faq/17/do-scientists-agree-on-climate-change/
  2. See: http://www.igmchicago.org/surveys/import-duties
  3. Godlee F.Smith J., & Marcovitch H. (2011) Wakefield’s article linking MMR vaccine and autism was fraudulent. British medical journal, 342, 64-66.
  4. Simons, D. A., & Wurtele, S. K. (2010). Relationships between parents’ use of corporal punishment and their children’s endorsement of spanking and hitting other children. Child Abuse & Neglect, 34, 639–646.
  5. Faris, R., & Felmlee, D. (2011). Status struggles: Network centrality and gender segregation in same- and cross-gender aggression. American Sociological Review, 76, 48–73. The study has also been covered by several media outlets: Pappas, S. (2011). Popularity increases aggression in kids, study finds. Retrieved from: http://www.livescience.com/11737-popularity-increases-aggression-kids-study-finds.html
  6. This pattern was found until adolescents reached the top 2% in the popularity ranks. After that, aggression declined.
  7. Mahoney, A., Karatzias, T., & Hutton, P. (2019). A systematic review and meta-analysis of group treatments for adults with symptoms associated with complex post-traumatic stress disorder. Journal of affective disorders243, 305-321.
  8. American Psychiatric Association. (2017). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC
  9. Barsky, A. (2015). DSM-5 and the ethics of diagnosis. New social worker. Retrieved from: https://www.socialworker.com/feature-articles/ethics-articles/dsm-5-and-ethics-of-diagnosis/
  10. Nogueira, B. L., Mari, J. D. J., & Razzouk, D. (2015). Culture-bound syndromes in Spanish speaking Latin America: the case of Nervios, Susto and Ataques de Nervios. Archives of Clinical Psychiatry (São Paulo), 42(6), 171-178.
  11. Eika, L., Mogstad, M., & Zafar, B. (2019). Educational assortative mating and household income inequality. Journal of Political Economy, 127(6), 2795-2835.
definition

License

Doctoral Research Methods in Social Work Copyright © by Mavs Open Press. All Rights Reserved.

Share This Book