So we're going to go back to where we started that first week and discuss how you can be a critical consumer of the research and the media's interpretation of that research. In your reading this week, I asked you to review a research article, Eavesdropping on Happiness, and then a popular press article reviewing it. We'll use this example to talk about how to be a critical consumer of the research and the media. Let's talk a bit about the study first. The authors in this Eavesdropping on Happiness study were interested in whether conversation type was related to well-being. They observed 79 students naturally without direct interference. They did this by using a digital audio recorder which allowed for unobtrusive tracking of 300 waking recordings per student. And they did this by 30 second recordings every 12.5 minutes over four days. Then they coded conversations and differentiated between different types of conversation. So they looked at alone time, talking to others, small talk and substantive conversations. Then they assessed students' well-being using self reports, informant reports and the satisfaction with life scale. Students also took the big five inventory twice to capture information on personality. Scores were averaged with informants ratings of students on the same inventory. The results of this study demonstrated that higher well-being was associated with more conversations and less alone time. Higher well-being was also associated with more substantive conversations and less small talk. These results remained consistent even when controlling for personality type and whether data was collected on weekdays or weekends. So the happiest participants were alone 25% less of the time. And spent about 75% more time talking. So how should we go about assessing this research study, and the media's summary of it? Well, remember the concept of validity, which means truth? As John Keats puts it, beauty is truth, truth beauty. That is all you know on earth and all you need to know. We've talked about three types of validity which are critical to evaluating research. Internal validity, external validity, measurement validity. And today we'll introduce a fourth type, consequential validity. First, internal validity. Internal validity, as you'll recall, captures whether the study can demonstrate that the independent variable causes the dependent variable. And there are two main questions to consider when looking at internal validity. First, the direction of the effect. So can you demonstrate that the independent variable leads to the dependent variable, or could it be the other way around? And second, could there be other variables that could explain the outcome that you didn't actually measure? So in the case of the Eavesdropping on Happiness article, did having more conversations and less alone time actually lead to higher well-being? Or was it the other way around? Did having higher well-being actually lead you to have more substantive conversations? Or was there another variable, altogether, that the authors didn't measure that led to well-being. Now, the authors did control for some other personality variables captured through the big five but, certainly not all of them. So there could have been another variable. I'm sure you all could think of many that could lead to increased well-being that this study didn't assess. And the authors can't account for other factors or the direction of the effect because there wasn't random assignment to groups. So this means we should be interpreting these results as correlational not causal. So how well did the media article do? Well, in the opening lines of this popular press article the author states that, new research suggests that less small talk and more substantive conversation causes increased happiness. Later he does go on to state that correlation doesn't necessarily prove causation. Which was an important observation, but he does use misleading language throughout. So he uses language like conversation may galvanize a previously unexamined life. Implying causation. Now this is one of the most important things to look for in a popular press article. As the media will often overstate findings as causal when the study was only correlational. Why you might ask? Well because it certainly makes for a flashier article and better headlines. The second type of validity to consider is external validity. External validity captures whether the results will translate to other populations. So, for example, we need to consider if and how results will translate to different settings, people, places, and times. The Eavesdropping on Happiness article only looked at undergraduates, which means there are limitations in external validity. So the question remains, could this even be applied to non-undergraduates? Or is there something about the types of conversations you have in college that are associated with your well-being that wouldn't apply outside of college? Now this is something that the media article doesn't address at all and is oftentimes another one of the limitations that you'll find in these types of media articles. The media like to make it seem like the findings of an article are universal and would apply in any setting, when often they are quite context specific. The third type of validity we've discussed is measurement validity. How well does the study capture its intended constructs? The study uses validated measures with high reliability like the satisfaction of life scale. The big five inventory and measures of well-being, which is the positive aspect of the study. They also use multiple measures which increases the convergent validity. Remember, no one measure is perfect, but using multiple measures is a solution to get at that imperfection and reduce the error of any one measure alone. However, the measures are still self report and informant measures, which as we talked about in week two, have several limitations. Perhaps there was social desirability bias and students answered questions in a certain way. Or perhaps there was reference bias in this group of undergraduates meaning that they had a different reference point about what well-being meant that wouldn't apply in other settings. Now these limitations weren't touched on at all in the popular media article. So as a critical consumer of the research it's something you need to look for. And the final type of validity is consequential validity, which refers to the positive or negative consequences of a measure or study. We need to consider how measures should and shouldn't be used based on their limitations. So while this study was interesting and useful, it's not clear how much we should make of the results. We can't demonstrate the direction of the effect, the study was only conducted with undergraduates, and there are limitations in self report and informant measures. So we shouldn't go as far as to the popular media article suggestion and conclude that real conversation might be the necessary stimulant to galvanize a previously unexamined life. It might be, but we'd need more research to demonstrate this was the case. So this bring us to the end of the final video introducing key concepts for this week. There are a few other components for this week. Again, researcher videos featuring Angela Duckworth applying these concepts to her work. Practitioner videos featuring practitioners applying these concepts to their work. A short quiz on key concepts, and two optional extension activities. A peer review activity where you discuss what kind of research design you would use to test your research question and hypothesis. And explore the strengths and weaknesses associated with that design. And an advanced activity where you read the popular media article on true grit and discuss how effectively you think the author captured the key findings from the study.
