Over-interpreting the data?
(Post 8 in an 11-part series)
In my last post, I discussed one of Reveal's two major criticisms of Willow Creek Church, a moderate relationship between church activities and spiritual growth. Here I discuss the second major criticism--that a significant minority of sample members described themselves as spiritually stagnant or were dissatisfied with their church. As shown in this figure, about 1/4 of the sample identified labeled themselves as either dissatisfied or stagnant.
This statistic highlights the problem posed by Reveal not having a comparison group, for we don't know if it's good or bad.
Let's say I told you that a sports player was successful at two-thirds of what they attempted, and then I asked you if this is good. Well, you would need to know how everyone else does. If it's free throws, this isn't very good. If it's completing passes, this is very good. If it's getting on base, it's the best ever.
It may be the nature of Christian life that 1/3 of the people are stalled or dissatisfied.
or
It may be that this number is high, suggesting that Willow needs to improve
or
It may be that this number is low, suggesting that Willow is doing a great job.
There's another wrinkle here: Church involvement is voluntary. So, why would a dissatisfied, stalled person be involved? It suggests that there may be other benefits or ties that come into play.
What if a church found that 100% of its attendees were satisfied and growing. Would this be good? Maybe but maybe not. What if the church didn't attract people unintentionally drove off or otherwise couldn't keep people who were having troubles or were unhappy? That would produce high satisfaction scores, but it might be missing the larger purpose of the Church.
In this sense, Willow might be commended for holding on to these people.
Again, it's not that Reveal did anything wrong in collecting these data or that it collected them badly. Rather, the concern here is over-interpretation. These data should suggest future work, but by themselves they are not sufficient to justify wide scale changes in an organization as successful as Willow Creek.
3 comments:
Hey Brad
I am really enjoying your series on Reveal. I make no secret of the fact that I am a big Willow Creek fan and also a fan of Reveal--especially the quality discussions it is generating on how to do church more effectively (and more importantly) how to BE THE CHURCH more effectively.
I've really appreciated your series and its detailed look at the study from a serious research perspective.
This post makes a lot of sense to me (especially the sports analogy... I'm simple that way). Without comparison, it is hard to tell the significance of the data.
My gut is that the conclusions they are drawing are accurate--but that is only a gut level response. As you know, St. Paul's Collegiate Church (www.stpaulswired.org) will be one of 500 churches participating in the next round of research. I will be interested to see what we learn and how it compares to previous surveys and research we have done.
Great series so far... thanks!
Great stuff... thanks.
I too greatly appreciate Willow, and I agree that good discussions are coming out of Reveal.
The conclusions of Reveal may well be correct, but I'm not sure we can tell from the data themselves...
The gut instinct of pastors such as yourself and Bill Hybels has a lot going for it.
Excellent Post. Willow was clearly disappointed by the results of the survey, but should they have been? What's normal for a church? Maybe they were doing great!
But here's some thoughts for another post or series of posts.
If there's no control data, how can churches accurately access themselves? Or can they only compare themselves to themselves?
We're better than last year! We may have simply moved from horrible to no good or from good to great but we don't have any idea which.
Also, what would be a realistic (or maybe not realistic)sociological study of churches over time across a broad spectrum? In other words how would you do Reveal 2?
And let's bracket the idea of figuring out what are the right questions to ask. That's pretty difficult. It's difficult to distill what an ideal disciple would exhibit and then how to put that into a series of questions. But let's say you figured it out. What then. How then would you ideally proceed?
Post a Comment