I've been thinking about the use of statistics in public sociology--using research to inform people outside of academics. It occurs to me that the statistics that make a difference in the public forum are very different than those published in the academic journals. Specially, they are much more simple (and presented with cooler graphics).
Here's my question then:
Here's my question then:
Has public policy ever been influenced by a statistic other than univariate or bivariate analysis? In other words, do any statistics matter beyond reporting levels of one variable (maybe over time) or two variables (e.g., comparing group means)?
It seems to me that most the policy-influential stats fit this description, but sociologists, even those who want to influence policy, put most their time into much more complicated statistics.
Thoughts?
5 comments:
You asked, "Has public policy ever been influenced by a statistic other than univariate or bivariate analysis? In other words, do any statistics matter beyond reporting levels of one variable (maybe over time) or two variables (e.g., comparing group means)?"
----------------------------------
I say "Yes, related variables matter"! The original argument on whether smoking caused lung cancer revolved around whether there were confounders (3rd variables or factors) that could influence the observed association. Schield (2007) in "Numbers in the News" analyzed 81 news "studes" and found that 9 of these took into account related factors. Schield and Terwilliger (2004) in "Frequency of Simpson's Paradox in NAEP Data" found that 4% to 10% of all statistically-significant differences in state averages could be reversed by taking into account a third factor. Both "The Bell Curve" and "More Guns; Less Crime" focused heavily on the influence of related factors or variables (confounders).
Hello Milo,
Interesting data. I guess that I'm not suggesting that multivariate analysis *shouldn't* be used, rather it rarely *is* used.
As such, it seems researchers should figure out how to tell a complex, multivariate story using uni- and bivariate analyses.
I'm not convinced that the issue is one of ability or capability, so much as playing to the lowest common denominator of "this causes that" story telling. If I'm trying to make a complex case into a black and white situation, the univariate or bivariate analysis works. As soon as you introduce more than a = b it 'muddies' the argumentative waters and makes the story harder to tell in 200 words or less. ;-)
I think you're right, Gordon.
Brad,
You concluded that multivariate analysis is seldom used. If seldom means 10%, I agree. Two studies of news stories (Schield, 2007, 2008) found that about 10% of them involved phrases indicating multivariate analysis. Phrases such as "after controlling for" or "after taking into account." The summary is typically a univariate or bivariate association, but that doesn't mean it is unconditional.
Post a Comment