Work Like a Producer, Think Like a Consumer
TSHS Webinar Summary, by Heather Hoffman, PhD, The George Washington University.
Dr. Lucy D’Agostino McGowan of Wake Forest University presented six design principles for data analysis and explained how to think about them in a mathematical way to improve the alignment between producers and consumers of the analysis in this webinar: Design Principles for Data Analysis.
Statistical Thinking
Dr. McGowan began by defining basic terminology. She explained that statisticians typically start with data and then apply a method to get an answer to the question at hand. Dr. McGowan presented the flow chart way of thinking where the type of data (input) leads to a specific test (output), which can easily be conducted by a computer. She then emphasized that it’s the person who must consider the design part. Dr. McGowan described this as a convergent process.
Design Thinking
Dr. McGowan contrasted statistical thinking with design thinking, which is a divergent process where the researcher uses multiple methods to answer the question. Dr. McGowan presented a model on design thinking proposed by the Hasso-Plattner Institute of Design at Stanford, where the analyst starts by considering the end user who will be consuming the product (i.e., the data analysis). She elaborated on the five steps used in the model, which include:
Empathize
Define
Ideate
Prototype
Test
Dr. McGowan described this as an iterative process between the various steps.
Why think about analyses in terms of design?
Dr. McGowan explained that the benefits of thinking about analyses in terms of design include:
Providing a common language
Improving pedagogy
Improving alignment between data analysis producers (i.e., data analysts) and consumers (e.g., clinicians, other statisticians, general public)
Design Principles
Dr. McGowan then presented six design principles for data analysis with real world examples.
Data matching: How well does the available data match the data needed to investigate a question?
Exhaustive: Are specific questions addressed using multiple, complementary, methods, tooling or workflows?
Skeptical: Are multiple related explanations considered using the same data (i.e., sensitivity analyses)?
Second-order: Does the analysis include anything that does not directly address the primary question but gives important context to the analysis?
Clarity: Does the analysis summarize data in a way that is influential in explaining how the underlying data connects to the conclusions?
Reproducible: Could someone who is not the original producer take the published code and data and compute the same results?
Quantitative/Mathematical Thinking
Dr. McGowan gave a more quantitative presentation on ways to improve the alignment between producers and consumers, and she showed the mathematical models used for analytic negotiation between the producer/analyst and consumer. Dr. McGowan clarified that at baseline an analyst and consumer will weight the principles, proceed to have a negotiation, and then reach a resolution. In words, Dr. McGowan explained why certain choices were made regarding the analysis:
choose an analyst from your field,
agree on resources dedicated to the analysis, and
have a discussion between analyst and consumer.
Explore and Learn More
You can review Dr. McGowan’s slide presentation on her website: lucymcgowan.com/talk
Dr. McGowan conducted this work in collaboration with Dr. Roger Peng and Dr. Stephanie Hicks of Johns Hopkins Bloomberg School of Public Health.
Learn more about Analytic Design Theory on their website: analyticdesigntheory.org
Check out their paper in the Journal of Computational and Graphical Statistics:
We welcome your thoughts in the comments below.