https://pressbooks.txst.edu/3402kelemen/chapter/drawing-conclusions-and-reporting-the-results/

Drawing Conclusions

Since statistics are probabilistic in nature and findings can reflect type I or type II errors, we cannot use the results of a single study to conclude with certainty that a theory is true. Rather theories are supported, refuted, or modified based on the results of research.

If the results are statistically significant and consistent with the hypothesis and the theory that was used to generate the hypothesis, then researchers can conclude that the theory is supported. Not only did the theory make an accurate prediction, but there is now a new phenomenon that the theory accounts for. If a hypothesis is disconfirmed in a systematic empirical study, then the theory has been weakened. It made an inaccurate prediction, and there is now a new phenomenon that it does not account for.

Although this seems straightforward, there are some complications. First, confirming a hypothesis can strengthen a theory but it can never prove a theory. In fact, scientists tend to avoid the word “prove” when talking and writing about theories. One reason for this avoidance is that the result may reflect a type I error.  Another reason for this avoidance is that there may be other plausible theories that imply the same hypothesis, which means that confirming the hypothesis strengthens all those theories equally. A third reason is that it is always possible that another test of the hypothesis or a test of a new hypothesis derived from the theory will be disconfirmed. This difficulty is a version of the famous philosophical “problem of induction.” One cannot definitively prove a general principle (e.g., “All swans are white.”) just by observing confirming cases (e.g., white swans)—no matter how many. It is always possible that a disconfirming case (e.g., a black swan) will eventually come along. For these reasons, scientists tend to think of theories—even highly successful ones—as subject to revision based on new and unexpected observations.

A second complication has to do with what it means when a hypothesis is disconfirmed. According to the strictest version of the hypothetico-deductive method, disconfirming a hypothesis disproves the theory it was derived from. In formal logic, the premises “if A then B” and “not B” necessarily lead to the conclusion “not A.” If A is the theory and B is the hypothesis (“if A then B”), then disconfirming the hypothesis (“not B”) must mean that the theory is incorrect (“not A”). In practice, however, scientists do not give up on their theories so easily. One reason is that one disconfirmed hypothesis could be a missed opportunity (the result of a type II error) or it could be the result of a faulty research design. Perhaps the researcher did not successfully manipulate the independent variable or measure the dependent variable.

A disconfirmed hypothesis could also mean that some unstated but relatively minor assumption of the theory was not met. For example, if Zajonc had failed to find social facilitation in cockroaches, he could have concluded that drive theory is still correct but it applies only to animals with sufficiently complex nervous systems. That is, the evidence from a study can be used to modify a theory. This practice does not mean that researchers are free to ignore disconfirmations of their theories. If they cannot improve their research designs or modify their theories to account for repeated disconfirmations, then they eventually must abandon their theories and replace them with ones that are more successful.

The bottom line here is that because statistics are probabilistic in nature and because all research studies have flaws there is no such thing as scientific proof, there is only scientific evidence.

Reporting the Results

The final step in the research process involves reporting the results. As described in the section on Reviewing the Research Literature in this chapter, results are typically reported in peer-reviewed journal articles and at conferences.

The most prestigious way to report one’s findings is by writing a manuscript and having it published in a peer-reviewed scientific journal. Manuscripts published in psychology journals typically must adhere to the writing style of the American Psychological Association (APA style). You will likely be learning the major elements of this writing style in this course.

Another way to report findings is by writing a book chapter that is published in an edited book. Preferably the editor of the book puts the chapter through peer review but this is not always the case and some scientists are invited by editors to write book chapters.

A fun way to disseminate findings is to give a presentation at a conference. This can either be done as an oral presentation or a poster presentation. Oral presentations involve getting up in front of an audience of fellow scientists and giving a talk that might last anywhere from 10 minutes to 1 hour (depending on the conference) and then fielding questions from the audience. Alternatively, poster presentations involve summarizing the study on a large poster that provides a brief overview of the purpose, methods, results, and discussion. The presenter stands by their poster for an hour or two and discusses it with people who pass by. Presenting one’s work at a conference is a great way to get feedback from one’s peers before attempting to undergo the more rigorous peer-review process involved in publishing a journal article.

https://open.oregonstate.education/qualresearchmethods/chapter/chapter-20-presentations/

Interpreting Qualitative Results

It is only now, after the laborious process of coding is complete, that you can actually move on to generate and present findings about your data. Many beginning researchers attempt to skip the middle work and get straight to writing, only to find that what they say about the data is pretty thin. The quality of qualitative research comes from the entire analytical process: open and closed coding, writing analytical memos, identifying patterns, making comparisons, and searching for order in the voluminous transcripts and fieldnotes.

But let’s say that you have followed all the steps so far. You have done multiple rounds of coding—refining, simplifying, and ordering your codes. You’ve looked for patterns. You think you have seen some master concepts emerge, and you have a good idea of what the important themes and stories are in your data. How do you begin to explain and describe those themes and stories and theories to an audience? Chapter 20 will go into further detail on how to present your work (e.g., formats, length, audience, etc.), but before we get to that, we need to talk about the stage after coding but before writing. You will want to be clear in your mind that you have the story right, that you have not missed anything of importance, and that you have searched for disconfirming evidence and not found it (if you have, you have to go back to the data and start again on a new track).

Begin with your research question(s), either as originally asked or as reformulated. What is your answer to these questions? How have your underlying goals (see chapter 4) been addressed or achieved by these answers? In other words, what is the outcome of your study? Is it about describing a culture, raising awareness of a problem, finding solutions, or delineating strategies employed by participants? Perhaps you have taken a critical approach, and your outcome is all about “giving voice” to those whose voices are often unheard. In that case, your findings will be participant driven, and your challenge will be to present passages (direct quotes) that exemplify the most salient themes found in your data. On the other hand, if you have engaged in an ethnographic study, your findings may be thick, theoretically informed descriptions of the culture under study. Your challenge there will be writing evocatively. Or to take a final example, perhaps you undertook a mixed methods study to find the best way to improve a program or policy. Your findings should be such that suggest particular recommendations. Note that in none of these cases are you presenting your codes as your findings! The coding process merely helps you find what is important to say about the case based on your research questions and underlying aims and goals.

The gold star of qualitative research presentation is the formulation of theory. Even for those not following the Grounded Theory tradition, finding something to say that goes beyond the particulars of your case is an important part of doing social science research. Remember, social science is generally not idiographic. A “theory” need not be earth shattering, as in the case of Freud’s theory of Ego, Id, and Superego. A theory is simply an explanation of something general.^[3] It is a story we tell about how the world works. Theories are provisional. They can never be proven (although they can be disproven). My description of Loyalists and Renegades is a theory about how college students from the working class manage the problem of class identity when their class backgrounds no longer match their class destinations. While qualitative research is not statistically generalizable, it is and should be theoretically generalizable in this way. Loyalists and Renegades are strategies that I believe occur generally among those who are experiencing upward social mobility; they are not confined solely to the twenty-one students I interviewed in 2005 in a college in the Pacific Northwest.

What is the story your research results are telling about the world? That is the ultimate question to ask yourself as you conclude your data analysis and begin to think about writing up your results.

1. Saldaña (2014) lists five essential characteristics of a social science theory: “(1) expresses a patterned relationship between two or more concepts; (2) predicts and controls action through if-then logic; (3) accounts for parameters of or variation in the empirical observations; (4) explains how and/or why something happens by stating its cause(s); and (5) provides insights and guidance for improving social life” (349). ↵