Over the weekend, Lyndon Unger posted a tremendous article that lays a fairly good debunking on the “John-MacArthur-is-attacking-the-extreme-fringe-people-in-the-charismatic-movement-he-needs-to-interact-with-Carson-Fee-Grudem-Keener-and-the-vast-majority-of-sound-thinking-charismatics” narrative that has popped up since the Strange Fire conference. Phil and John even mentioned it at the Q&A Sunday night.What Lyndon did is to consider the twitter accounts of both the so-called sound-minded charismatic, continuationists and the so-called wacko individuals who are outright heretics, and count the number of followers for each account. It is not meant to be an exact, scientific statistical analysis, because certainly there are many factors in play to skew data. However, it does demonstrate the profound, deep influence those of the heretical variety have in the broader world contrary to the shrill complaints of the so-called “sound” charismatics.Read his initial post and commentary (comments are good, too) and then his follow-up post a day later. Both of them should provide Steve Hays enough material for like 100 smug rebuttal posts.http://hipandthigh.wordpress.com/2013/11/18/mennoknights-twitter-analysis-post/
Unfortunately, Fred's honesty has gone into hibernation. I hope we're not in for a long winter.
I've never taken a position on the ratio of sound charismatics to unsound charismatics. I've never said that unsound charismatics reflect "the extreme fringe" of the movement.
What I have said is that if you're going to assess the truth or falsity of a position, then you need to engage the most intellectually astute spokesmen for that position. That's entirely distinct from how many people believe it. Has Fred become such a blind partisan that he can no longer distinguish between the truth of a position and the popularity of a position?
Since he brings it up, it's a reflection of Fred's confirmation bias that he puts any stock in Unger's "article." What are Unger's qualifications to do a comparative statistical analysis? Is he a sociologist of religion who specializes in the charismatic movement? Does he have scientific surveys? Why does Fred imagine that Unger's amateur-night exercise in Google scholarship has any scientific merit? By the same token, Phil posting YouTube clips of charismatic nonsense is hardly a scientific sampling.
When Fred and Phil operate with such low intellectual standards, how are they in any position to attack the low intellectual standards in broad swaths of the charismatic movement? It's like watching a competition between rodeo clowns.
As I read Unger's article, I felt my brain cells committing suicide one by one.
ReplyDeleteFred said:
ReplyDeleteIt is not meant to be an exact, scientific statistical analysis, because certainly there are many factors in play to skew data. However, it does demonstrate the profound, deep influence those of the heretical variety have in the broader world contrary to the shrill complaints of the so-called “sound” charismatics.
So do they want these results to mean anything or not? It's like they want the confirmation of a solid study without any of the necessary work.
Why don't they just write "MACARTHUR WAS RIGHT" on 100 index cards, put them in a bag, and randomly pick one card at a time to read out loud. It would be just as meaningful as this Twitter "research."
"I've never taken a position on the ratio of sound charismatics to unsound charismatics."
ReplyDeleteSteve, if you did, what would be your reasoned guess for picking between these two choices:
A). # of unsound charismatics > # of sound charismatics
B). # of sound charismatics > # of unsound charismatics
I don't have an informed opinion on the relative percentages.
DeleteYes! Just 99 to go!
ReplyDeletePhil posting YouTube clips of charismatic nonsense is hardly a scientific sampling
ReplyDeleteThere are a BUNCH of those.
An increase in a bad sample doesn't make the sample better.
DeleteIt means the sample size is really large.
DeleteReally? Let's say there are half a billion charismatics. What fraction of a fraction of fraction of a percentile would even hundreds of YouTube clips sample? Likewise, are those YouTube clips demographically representative? Do they cut across nationalities, ethnicities, economic, and educational strata? Surely you're not naive enough to think that posting YouTube clips is a scientifically accurate way to sample a vast and diverse religious movement?
DeleteAt best, that would be a way of *illustrating* a widespread problem, the scope of which you previously established by more reliable means. That's not a way of proving the scope of the problem.
What fraction of a fraction of fraction of a percentile would even hundreds of YouTube clips sample?
DeleteWhen each of the clips shows hundreds of people...
A few hundred here, a few hundred there, times a hundred... pretty soon you're talking real life.
are those YouTube clips demographically representative?
I don't know, but I don't think it's particularly relevant.
Do they cut across nationalities, ethnicities, economic, and educational strata?
Mbewe handled that part.
Surely you're not naive enough to think that posting YouTube clips is a scientifically accurate way to sample a vast and diverse religious movement?
Not scientific, no. Close enough for government work, though.
You know what would be awesome? Some big name charismatics throwing out blanket disaffirmations of such behavior. But that would be unreasonable, wouldn't it? ;-)
that would be a way of *illustrating* a widespread problem
Sounds great!
Like I said, increasing a bad sample does nothing to make the sample better.
DeleteA YouTube video represents a media creation outlet generally accessible only to the moderately wealthy in countries with sufficient infrastructure. Such media are also generally only produced by certain kinds of individuals or groups, among other relevant sociological factors.
Even if it were useful, it would be challenging to get a proper statistical sample of charismatic YouTube videos. To obtain a decent confidence level/interval given the enormous population size, you'd need to watch over 1,500 videos and you'd likely need to conduct relevant polling and demographic surveys. It would require hundreds of hours to conduct such research.
Whatever happened to the Reformed notion of loving God with your mind? The science of statistical sampling isn't something that can be ignored just because we like a certain stereotype that fits a particular theological agenda.
The other problem is that Phil's YouTube methodology reflects the schizophrenia of MacArthurites on the role of experience in evaluating religious claims. MacArthurites keep playing both sides of that fence. MacArthurites cite negative examples of charismatic experience to disprove charismatic theology, but preemptively discount positive examples of charismatic experience.
DeleteThis is one of the glaring systematic contradictions running through the MacArthurite indictment. They need to settle on a consistent position regarding the evidential value (or not) of religious experience, and stick with it.
Whatever happened to the Reformed notion of loving God with your mind?
DeleteI got slain in the Spirit one too many times.
"When each of the clips shows hundreds of people...A few hundred here, a few hundred there, times a hundred..."
DeleteHundreds here and hundreds there in relation to a half billion?
Oah, and that wouldn't be "times a hundred," but hundreds plus hundreds plus hundreds. I assume you know the difference between addition and multiplication.
And who said it wasn't talking about "real life"? Did I suggest these YouTube clips were staged? Fake? No.
The question is not whether that really happens, but whether that's an accurate say of *surveying* the phenomenon.
You're resorting to the tactic of liberal pundits who brand the entire Tea Party movement as "racist" based on a statistically infinitesimal sample of allegedly racist individuals.
"I don't know, but I don't think it's particularly relevant."
Which means you're not making a good faith effort. If you're going to use YouTube clips to sample a huge movement, then the clips need to be demographically representative.
"Mbewe handled that part."
So he's an expert the charismatic movement in India and Latin America? You're not making a good faith effort.
Even on the African scene, what makes you think he's better informed than Kwabena Asamoah-Gyadu?
"Not scientific, no. Close enough for government work, though."
What does "close enough for government work" even mean? Does gov't rely on YouTube clips? If so, is that a reliable indicator?
"You know what would be awesome? Some big name charismatics throwing out blanket disaffirmations of such behavior. But that would be unreasonable, wouldn't it? ;-)"
Yes, that would be beneficial. It would also be beneficial if partisans like you were honest enough to be even-handed in your criticisms. But that would be unreasonable, wouldn't it?
"Sounds great!"
If you think that sounds great, then when are folks like you going to get serious about statistically accurate polling on the charismatic movement?
Oah, and that wouldn't be "times a hundred," but hundreds plus hundreds plus hundreds. I assume you know the difference between addition and multiplication.
DeleteA hundred clips with a hundred barkers each. That would be multiplication.
If you're going to use YouTube clips to sample a huge movement, then the clips need to be demographically representative.
That has never been my argument, actually.
Even on the African scene, what makes you think he's better informed than Kwabena Asamoah-Gyadu?
B/c there are a lot of people in Africa.
And b/c maybe this latter man has some face to save?
What does "close enough for government work" even mean?
It's a colloquial phrase.
Yes, that would be beneficial.
You murdered many thousands of pixels a few weeks ago arguing that such a thing wasn't necessary. Strange to see you say this now.
It would also be beneficial if partisans like you were honest enough to be even-handed in your criticisms.
Since you're not addressing my actual argument, this swings wide of the mark.
when are folks like you going to get serious about statistically accurate polling on the charismatic movement?
Haha, I promise you I have better things to do with my time than that.
But, again, since that was never my argument, I'll bow out here.
Alan,
DeleteI'm tired of your adversarial mentality. You're like a lawyer or political operative who thinks you only have to present your side of the argument.
As a rule, it's a good practice to mentally argue both sides of an issue. To anticipate objections and counterexamples.
You're not doing that. You're not making a serious effort to see potential weaknesses in the MacArthurite position or potential strengths in the noncessationist position. You always wait for someone else to bring those up. I don't have time for that behavior. That's substandard for Tblog teammates. If MacArthur wants to host a totally one-sided event, that's his prerogative. But that's not good enough for Tblog.
"You're not doing that. You're not making a serious effort to see potential weaknesses in the MacArthurite position or potential strengths in the noncessationist position."
DeleteSteve, what would you consider to be your best serious effort to see potential weaknesses in the Keenerite, Piperite, Grudemite positions, or potential strengths in the cessationist position?
If you have to ask, you haven't been reading my posts. It's all there, in detail.
DeleteNot scientific, no. Close enough for government work, though.
DeleteHi Rhology,
Just a quick note if I may please. As we know, the federal gov't (not to mention local gov't and state gov't) does make health policy and recommend health guidelines for example. Of course, the gov't doesn't base their health policy or guideliness on willy nilly whims or somesuch. Rather they generally base their health policy or recommended guidelines on medical and scientific studies. Not just any medical or scientific studies either, but on ones which have been undertaken responsibly, are valid, have no significant biases, are generalizable, etc. Anyway, point being "government work" doesn't always preclude "scientific" standards, I don't think.
I've been reading a lot of them. And I'm fairly sure that there are a lot more anti-MacArthurite posts than anti-Keenerite, anti-Piperite, anti-Grudemite posts.
ReplyDeleteWhat would you consider to be your strongest anti-continuationist post?
Don't be simplistic. I don't write for readers with a Twitter attention span.
DeleteAnd this has nothing to do with numerical matching. That would only apply if cessationist and noncessationism had comparable strengths and weaknesses.
Lyndon Unger said:
ReplyDeleteYup. Agreed. There are plenty of methodological flaws. All I was going for was the general tenor of breadth of influence.
1. At the risk of stating the obvious, it's not possible to do a proper study (let alone a good study) if there are "plenty of methodological flaws." Indeed, if there are "plenty of methodological flaws," then how could it be a useful study in the first place? If we're building a house, and if there are plenty of flaws in the house's frame or foundation, then how can we expect to build a house that will stand? It'd be like building a house on sand rather than rock.
2. Hence when Unger ends by saying, "All I was going for was the general tenor of breadth of influence," I'm sorry but this is no better than saying something like: "Sure, what I've written has plenty of problems. But I'm just going for a ballpark figure. A rough guesstimate. Just my general impression of the situation. Hopefully I got close enough, but who knows? No guarantees though."
3. In any case, sample size determination can involve quite sophisticated statistics.
Sample size is closely related to statistical power, which in turn can likewise involve complex statistics.
To say nothing of other relevant and associated concepts like statistical distribution, confidence intervals, type I and type II errors, etc.
Take sampling bias for instance. How can we better ensure subjects are representative of the general population? We may have to do something like (a) blind the study participants. And not just the subjects or participants in the study, but also the researchers undertaking the study since researchers could be biased in selecting particular subjects even without knowing they're doing so (e.g. if a researcher has an affinity for someone of his or her own social group or ethnic group or gender). At the same time (b) we may have to randomize the groups since ideally groups should not differ in a systematic way, otherwise the results may be biased. And so on and so forth.
If the subjects aren't representative of the general population, then whatever results we obtain won't be generalizable.
Anyway, all this to say sample size determination and all it entails is nothing to sneeze at. But too many people including Unger (sorry to say) seem to treat it as if it's a stroll in the park.
Sorry, but I'm afraid this isn't intellectually responsible.