P-values vs. Confidence Intervals (CIs): Which would you choose?
TSHS Webinar Summary, by Heather Hoffman, PhD, The George Washington University.
Ms. Hilary Watt of Imperial College London presented the practical relevance of p-values and confidence intervals, explaining how a clear choice of words and images can foster conceptual understanding and help steer away from misconceptions, in this webinar.
Concern for dividing results based on p-value<0.05
Ms. Watt began by presenting the following major misconceptions with statistical interpretation in health science research.
50% of research articles say they found no association based purely on p-value>0.05, some say this proves no association
Many imply results are important solely because p-value<0.05 even when associations are weak and clinically irrelevant
Some interpret CIs according to whether they imply p-value<0.05
Some believe CIs contain 95% of observations
Ms. Watt then explained it is good practice to focus on CIs and assess calculated results and the clinical relevance of each end of the CI, while only assessing the p-value evidence on a continuous scale in a minor supporting role if at all. Ms. Watt encouraged us to check out Amrhein, Greenland, and McShane’s article Scientists rise up against statistical significance, Nature (2019).
She then presented an example using a systematic review of streptokinase for myocardial infarction to exhibit the consequences of ignoring CIs and focusing on p-value<0.05. In doing so, she illustrated the importance of conducting a meta-analysis. The lesson learned is that if you ignore CIs and focus on p-value<0.05, then you lose the ability to sensibly summarize results across studies.
Why do people focus on p-value<0.05 when it is a known problem?
Ms. Watt questioned whether focusing on p-values was due to poor education, lack of understanding, or politics. She reminded us of Dr. Ronald Wasserstein’s 2016 p-value statement, “Teach p<0.05 because it’s the focus of researchers, it’s the focus because it’s taught. Call for better stats education.”
Ms. Watt proceeded to acknowledge the terminology issue, explaining that “significant” implies important but “non-significant” implies irrelevant. She recommended repeatedly defining the population and random sampling assumption and clarifying when referring to participants and when referring to a population. She said the population concept is often misunderstood and referred us to Lu’s paper Are statisticians cold-blooded bosses? A new perspective on the ‘old’ concept of statistical population.
Ms. Watt emphasized the importance of using clear language and provided the following useful interpretations of p-values.
Avoid dichotomizing according to p-value<0.05
Use graduated p-value interpretations
Strength of evidence to reject null hypothesis
Strength of evidence in favor of association in population
Better to express the level of compatibility with a random selection of participants from a population where this association does not exist
Focus on CIs
Similarly, Ms. Watt encouraged us to be clearer with interpretations of CIs, and she illustrated this with an example on mean cholesterol. She referred us to her paper Reflection on modern methods: Statistics education beyond ‘significance’: novel plain English interpretations to deepen understanding of statistics and to steer away from misinterpretations for more information.
She also used an example on blood pressure to show us how she encourages students to focus on CIs instead of p-value<0.05 and engages students by clinically interpreting CIs with such examples. She emphasized a fundamental misconception in thinking 95% of observations lie within a 95% CI and referred us to the following publications.
Finally, Ms. Watt presented some figures you can use to visually explain p-values and CIs to students in a more understandable way. She shared feedback from students and faculty, which were positive overall.
Explore and Learn More
We encourage you to check out her publications related to this talk.
We welcome your thoughts in the comments below.
Commenti