Tuesday, September 02, 2008

Thoughts on passing along an inaccurate statistic

In thinking about Christine's comments, let me start with some things we agree on and some possible clarifications.

First off, I agree that I make errors. Regarding errors, I've been doing sociology for about 17 years now (Yikes, I'm getting old...), and I continue to be dismayed by how often I get things wrong. Sometimes I do my analyses wrong, sometimes I miss or mangle relevant theories. Because of this, I value working with other people, from whom I learn, public discussion of research, as per this blog, and, especially, the peer-review process for publishing.

Second, I agree that I am biased. I've had a wide range of experiences in Christianity, and they can not help but influence my perception and general understanding of anything to do with Christianity. Furthermore, this type of bias is probably not testable, for we could put two people together who have different worldviews religion and no one "right" answer would emerge.

In recognition of this type of bias, I value the empirical study of Christianity. While not all Christian topics lend themselves well to data, some do and they should be studied rigorously. Data analysis, unlike personal assumptions, does lend itself to discussions of right and wrong, or, at least, better or worse. It is in this vain that I am discussing the statistic introduced here, that Evangelicals have a low reputation second only to prostitutes. I think this is an inaccurate statistic, and I believe that the case that I make against it holds regardless of my personal beliefs.

There are a couple of other issues that I'll develop in later posts. A few quick comments:

CW: "I'm Christine Wicker. I wrote the book you're calling inaccurate."
Me: To be clear, I'm not addressing your book as a whole. I'm only discussing a couple of sentences on the top of page 143. My interest is this particular statistic, how it gets used and modified. It would be unfair to evaluate your book on the basis on this single point.

CW: "You often quote statistics without researching them fully. Or presenting contrary data."
Me: This being a blog, I will sometimes present studies that I find interesting without any critical analysis. In my published writing, however, I try to avoid that.

CW: "I agree with Helen's idea that you could easily find out what the rest of the world thinks about evangelicals."
Me: By virtue of being on the faculty of a public university + teaching a class on the sociology of religion, I end up hearing a lot of people's perspectives about different religions.

CW: "BTW, reporters aren't popular either. But I don't try to kill the messenger for saying so."
Me: My interest is in the use and interpretation of this statistic rather than who in particular uses it. I apologize if you feel like I'm shooting the messenger (i.e., you)...

Part 5 of the series.


4 comments:

Corey said...

I think there's an apples and oranges dynamic going on here:

Brad did what social scientists are trained to do. When he encountered the truth claim about "Christians" being almost as denigrated as prostitutes, he invoked Peter Berger's imperative: "Say's who?" In looking at the full distribution of responses from Barna (a polling organization that has been previously taken to task on this blog for sloppy methodology), we see that infact the rank orderings have a couple of "Christian" groups ranked highly: Ministers and Born Again Christians occupy positions 3 and 4.

Brad points out that the respondents may have been confused as to the implied meaning of evangelical. Perhaps they took the term to mean, one who accosts you with a Schofield reference bible as you try to walk down the street. But who knows.. that's one point of Brad's earlier criticism. This measure is of questionable face validity.

But the Statistic has been used (apparently by Ms. Wicker, among others... I haven't read the book in question) as an indicator that Christianity is dying or universally denigrated.

The point is that this particular assertion is undermined by the measure. Now Ms. Wicker's accusation:

==Start Quote==
You are able to find other statistics that don't back this one up. That doesn't make this one inaccurate. It simply means that you would rather use other studies. Just as you accuse me of being sensationalist, I'd say that you are biased toward evangelicals and Christians.
==end quote==

As a social scientist, that logic really frustrates me. It tantamount to nihilism of the worst sort. Whether or not Brad is biased toward Christians and evangelicals, he has always struck me as a fair and critical examiner of the evidence (even when I disagree with his interpretation). Evidence always has strengths and weaknesses. Good scholarship (as well as good journalism, I hope) takes critical examination of the evidence into account. Just because Brad has called an interpretation of Barna's evidence into question, does not necessarily mean that he's willing to cherry pick his own evidence to support another position... at least, I've never seen him do something that smacks of that. To a social scientist, such an accusation is stinging!

For the record, I'm no longer a believer (but suffered through a long childhood in an evangelical home, my parents are still very active in their community). From my N of 1 perspective, Christians are not the problem... in fact, I like being around most Christian's; it's their hypocritical leaders (I'm looking at you Rick Warren) that are the problem.

Knumb said...

Brad,

I didn't really bother reading your first posts on this topic until this one.

As soon as I read the Evangelicals vs. prostitutes part, I just figured someone out there was writing a hit piece on Christians.

Reading the posts now, it appears to me that you simply questioned the veracity of a statistic.

Pretty remarkable that the author would turn and call you biased for asking a question. That's classic obfuscation with a twist. It's not the crabapples in my cheeks, it's the chestnuts in your fists.

The old rag applies here: there lies, damned lies, and statistics. I've always read that as a clever praise of statistics, but in this case it opens the possibility of three lies in a row, the worst at the end, if the statistics in question are flawed.

If I had to guess, the book title was chosen before the first scrap of "data" was collected.

Unknown said...

After all, you're not exactly a scholar either... :)

Brad Wright said...

All too true, all too true.