Should non-experts shut up? The skeptic's catch-22
I'm not sure I can get behind Massimo's broader argument that non-experts should defer to the expert consensus in a field. The main problem, as I see it, is that while we can usually count on the process of peer criticism to check the research within a field for validity, I'm not sure we can count on it to check the validity of the field itself. I'm talking here about the methodology and the philosophical assumptions that underlie a field, the usually-unstated pillars on which all its findings rest.
Why? First of all, while there are strong incentives for a researcher to find errors in other work in the field, there are strong disincentives for her to challenge the field's foundational assumptions. It will be extremely difficult for her to get other people to agree with her if she tries, and if she succeeds, she'll still be taking herself down along with the rest of the field. Second of all, fields naturally select for people who accept their foundational assumptions. People who don't accept those assumptions are likely not to have gone into that field in the first place, or to have left it already.
In my profession I can’t afford to be ‘radically skeptical’ of the acknowledged experts, or I wouldn’t know what to do about almost anything. However there are significant reasons not to swallow unquestioningly every pronouncement from academia.
It should be immediately obvious to everyone with wit enough to be held responsible for their actions that, human nature being what it is, review of some kind is necessary. Outright fraud happens. Unfortunately, one can almost guarantee that peer review will not be completely successful in disclosing it. Beyond that there is the natural tendency to want one’s research to be successful. This almost certainly leads in some cases to tweaking of the data, without overt ill intent. Then there is the ‘echo chamber’ effect, wherein nearly everyone in a particular discipline assumes certain things to be so, and tends to discount anything that tends to disprove them.
I submit two items of my own experience that lead me to be skeptical of the pronouncements of ‘experts’. In my senior year as a biology student I had the opportunity to take a graduate level course with a professor whom I had come to respect (at least partly due to his willingness to openly admit his and his colleagues’ ignorance in certain areas). The course was on critically reading research papers. He would have us read various papers. I can’t remember with certainty, but I think these were published papers. He would invite us to critique them on methodological grounds alone, without reference to other research in the field. All the papers seemed fine to me. He would proceed to rip them to shreds, again on methodological grounds alone, demonstrating how they had failed to control for this, had done their statistics wrong about that, etc.
From that experience it seemed, and still seems to me that critically reading research is a discipline in itself, and one that one does not necessarily acquire from being an ‘expert’ in one’s field.
From my own field of expertise, I submit another experience: sometime around 1970, the American College of Obstetrics and Gynecology, the acknowledged experts in the field in the US, and the NIH, the federal government’s medical arm, together announced the following: that henceforth the word ‘pregnancy’ would mean ‘that state that begins with implantation’. Why would they do that? Consider: there are multiple methods of birth control that either only partially block conception or do not prevent it at all. In addition to blocking conception, or instead of it, these methods may prevent the implantation of the developing embryo onto the uterine wall. The embryo will then die of lack of nourishment and be flushed out with the next menstrual period, the mother being all unaware. These methods include the intrauterine device and the hormonal methods (which includes ‘the Pill’). Many women are understandably concerned about using a method of birth control should they become pregnant anyway. Some at least want to make sure that, if they do become pregnant, the birth control method will not harm the baby. Changing the definition of ‘pregnancy’ now allows physicians to say with a straight face, “O no, Ms Smith, if you get pregnant this method won’t interfere at all with the pregnancy.” Of course he isn’t likely to emphasize to her that his definition of ‘pregnancy’ is different than hers.
Knowing, then, as I do, that political considerations can and do affect the pronouncements of ‘experts’, I am all the more emboldened to be at least preliminarily skeptical of them.
"From that experience it seemed, and still seems to me that critically reading research is a discipline in itself, and one that one does not necessarily acquire from being an ‘expert’ in one’s field."
For better or for worse, I think in medical education today, such as at my med school, we have to learn how to "critically read research" as part of the curriculum and as part of the bigger picture of evidence-based medicine.
There seem to be a lot of med students undertaking research too. Of course, many seem to opt for clinical research, but a lot do lab-based research. From benchside to bedside and everything in between (e.g. "translational" research).
Again, I don't know if this is necessarily a good or bad thing overall. As a mere med student, I don't have the proper perspective to say one way or the other how well this contributes to our future medical education.
But I certainly agree "critically reading research is a discipline in itself." I suppose that's why to attempt to do good research necessitates not only physicians or scientists, but also a strong team including statisticians and others trained in how to best appraise research.
It'd be great to abide by guidelines like the PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses, but I doubt most research teams do.
Cochrane tries to vet the research on a particular topic or field, and they're great at that, but they're not comprehensive.
At least insofar as I understand all this.
"Knowing, then, as I do, that political considerations can and do affect the pronouncements of ‘experts’, I am all the more emboldened to be at least preliminarily skeptical of them."
In addition, I've read the change in definition started out as a marketing strategy to sell the oral contraceptive pill in the late 1960s or early 1970s. Albert Rosenfeld realized the pill could possibly prevent implantation from occurring, so he and others thought it'd be best to change the definition of pregnancy from fertilization (the traditional definition) to implantation. Otherwise the public might not buy the pill because it might cause an abortion. So there was a financial incentive in the definitional change.
Of course, now that there's an entire market built around contraceptives including the pill as well as providing and defending the provision of abortion (e.g. if I'm not mistaken Planned Parenthood made over $1 billion gross last year and over $100 million net profit), and now that abortion is no longer widely vilified like I assume it was in the 1960s and 1970s, then there's strong social and many other pressures including as you've pointed out political pressures to becloud the judgments of even the most objective "experts."
It's not merely experts we need today, but honest experts.
RwithH: Thanks for your comments. Always glad to hear of others in the profession who are not 'sold out' to the pro-abortion perspective. As you say, we need honest experts; perhaps moreover we might say we need Godly experts.
"Godly experts" would, of course, be ideal! If only there were more.
I should say I'm just a mere med student though and not yet part of the profession. I appreciate the mentorship relationships between attendings, residents, interns, med students, etc., but I realize I'm the lowest guy on the totem pole! But, God willing, soon...
By the way, sorry to everyone else since this is off-topic, but as I don't see contact details, I don't suppose I could ask you as a seasoned veteran Christian physician whether you might have any advice for Christian med students in choosing a particular specialty, particularly in light of Obamacare?
rocking: Like you I hope we're not inappropriately hijacking the comments here. But at the risk of doing so... Twain had a saying: "No one is safe when Congress is in session." Anything anybody thinks they know about 'Obamacare', even if currently accurate, could change at any time. Given that, and knowing the propensities of unConstitutional legislators (which means just about all of them), I suspect the primary care disciplines will fare about as well as any. However one must give due weight to one's own talents and drives; if a particular field fascinates you, you're likely to do better (for yourself, your family, and your patients) in it than another.
In my profession I can’t afford to be ‘radically skeptical’ of the acknowledged experts, or I wouldn’t know what to do about almost anything. However there are significant reasons not to swallow unquestioningly every pronouncement from academia.
ReplyDeleteIt should be immediately obvious to everyone with wit enough to be held responsible for their actions that, human nature being what it is, review of some kind is necessary. Outright fraud happens. Unfortunately, one can almost guarantee that peer review will not be completely successful in disclosing it. Beyond that there is the natural tendency to want one’s research to be successful. This almost certainly leads in some cases to tweaking of the data, without overt ill intent. Then there is the ‘echo chamber’ effect, wherein nearly everyone in a particular discipline assumes certain things to be so, and tends to discount anything that tends to disprove them.
I submit two items of my own experience that lead me to be skeptical of the pronouncements of ‘experts’. In my senior year as a biology student I had the opportunity to take a graduate level course with a professor whom I had come to respect (at least partly due to his willingness to openly admit his and his colleagues’ ignorance in certain areas). The course was on critically reading research papers. He would have us read various papers. I can’t remember with certainty, but I think these were published papers. He would invite us to critique them on methodological grounds alone, without reference to other research in the field. All the papers seemed fine to me. He would proceed to rip them to shreds, again on methodological grounds alone, demonstrating how they had failed to control for this, had done their statistics wrong about that, etc.
From that experience it seemed, and still seems to me that critically reading research is a discipline in itself, and one that one does not necessarily acquire from being an ‘expert’ in one’s field.
From my own field of expertise, I submit another experience: sometime around 1970, the American College of Obstetrics and Gynecology, the acknowledged experts in the field in the US, and the NIH, the federal government’s medical arm, together announced the following: that henceforth the word ‘pregnancy’ would mean ‘that state that begins with implantation’. Why would they do that? Consider: there are multiple methods of birth control that either only partially block conception or do not prevent it at all. In addition to blocking conception, or instead of it, these methods may prevent the implantation of the developing embryo onto the uterine wall. The embryo will then die of lack of nourishment and be flushed out with the next menstrual period, the mother being all unaware. These methods include the intrauterine device and the hormonal methods (which includes ‘the Pill’). Many women are understandably concerned about using a method of birth control should they become pregnant anyway. Some at least want to make sure that, if they do become pregnant, the birth control method will not harm the baby. Changing the definition of ‘pregnancy’ now allows physicians to say with a straight face, “O no, Ms Smith, if you get pregnant this method won’t interfere at all with the pregnancy.” Of course he isn’t likely to emphasize to her that his definition of ‘pregnancy’ is different than hers.
Knowing, then, as I do, that political considerations can and do affect the pronouncements of ‘experts’, I am all the more emboldened to be at least preliminarily skeptical of them.
Just to reinforce what Doc has said:
Delete"From that experience it seemed, and still seems to me that critically reading research is a discipline in itself, and one that one does not necessarily acquire from being an ‘expert’ in one’s field."
For better or for worse, I think in medical education today, such as at my med school, we have to learn how to "critically read research" as part of the curriculum and as part of the bigger picture of evidence-based medicine.
There seem to be a lot of med students undertaking research too. Of course, many seem to opt for clinical research, but a lot do lab-based research. From benchside to bedside and everything in between (e.g. "translational" research).
Again, I don't know if this is necessarily a good or bad thing overall. As a mere med student, I don't have the proper perspective to say one way or the other how well this contributes to our future medical education.
But I certainly agree "critically reading research is a discipline in itself." I suppose that's why to attempt to do good research necessitates not only physicians or scientists, but also a strong team including statisticians and others trained in how to best appraise research.
It'd be great to abide by guidelines like the PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses, but I doubt most research teams do.
Cochrane tries to vet the research on a particular topic or field, and they're great at that, but they're not comprehensive.
At least insofar as I understand all this.
"Knowing, then, as I do, that political considerations can and do affect the pronouncements of ‘experts’, I am all the more emboldened to be at least preliminarily skeptical of them."
In addition, I've read the change in definition started out as a marketing strategy to sell the oral contraceptive pill in the late 1960s or early 1970s. Albert Rosenfeld realized the pill could possibly prevent implantation from occurring, so he and others thought it'd be best to change the definition of pregnancy from fertilization (the traditional definition) to implantation. Otherwise the public might not buy the pill because it might cause an abortion. So there was a financial incentive in the definitional change.
Of course, now that there's an entire market built around contraceptives including the pill as well as providing and defending the provision of abortion (e.g. if I'm not mistaken Planned Parenthood made over $1 billion gross last year and over $100 million net profit), and now that abortion is no longer widely vilified like I assume it was in the 1960s and 1970s, then there's strong social and many other pressures including as you've pointed out political pressures to becloud the judgments of even the most objective "experts."
It's not merely experts we need today, but honest experts.
RwithH: Thanks for your comments. Always glad to hear of others in the profession who are not 'sold out' to the pro-abortion perspective. As you say, we need honest experts; perhaps moreover we might say we need Godly experts.
DeleteThanks, Doc! I appreciate your comments as well!
Delete"Godly experts" would, of course, be ideal! If only there were more.
I should say I'm just a mere med student though and not yet part of the profession. I appreciate the mentorship relationships between attendings, residents, interns, med students, etc., but I realize I'm the lowest guy on the totem pole! But, God willing, soon...
By the way, sorry to everyone else since this is off-topic, but as I don't see contact details, I don't suppose I could ask you as a seasoned veteran Christian physician whether you might have any advice for Christian med students in choosing a particular specialty, particularly in light of Obamacare?
Thanks in advance!
rocking:
ReplyDeleteLike you I hope we're not inappropriately hijacking the comments here. But at the risk of doing so...
Twain had a saying: "No one is safe when Congress is in session." Anything anybody thinks they know about 'Obamacare', even if currently accurate, could change at any time. Given that, and knowing the propensities of unConstitutional legislators (which means just about all of them), I suspect the primary care disciplines will fare about as well as any. However one must give due weight to one's own talents and drives; if a particular field fascinates you, you're likely to do better (for yourself, your family, and your patients) in it than another.