Confirmation bias

Coram Deo asks a good question here:

I don't know if either Hawk or steve (or another active T-blogger) would be willing to take up the question, but both here and elsewhere I'm seeing what I perceive as rampant confirmation bias swirling amidst the current coronavirus pandemic and I'm wondering if it's possible to implement guards or filters in our critical thinking to avoid this to some extent.

This assumes of course that folks wish to avoid or mitigate this feature, indeed many seem to rush to pick out this or that expert, model, or theory which bolsters their preconceived notions.

Thanks, CD. That's a good point and a good question! What you've said is important to keep in mind. If I can offer my random thoughts:

Some prefatory comments about my own position

1. Admittedly, I've probably been guilty of confirmation bias. Quite possibly more than a few times. Alas! Still, my goal is to follow the evidence where it leads, even or perhaps especially when I fail.

2. That said, I think my general position has (more or less) been that the coronavirus is a serious threat (e.g. transmission seems significantly higher than the seasonal flu, virulence is another factor) and that our response to the coronavirus could be an equally serious threat if we aren't vigilant about safeguarding our way of life (e.g. people panicking, overreaching government mandates). And (I believe based on the evidence) I regard the positions "the coronavirus is overblown" and "the coronavirus is the fourth horseman of the apocalypse" as a kind of Scylla and Charybdis to sail in-between.

3. Hence why many of my posts have been attempts to urge reasonable thinking and acting, proper preparation and safety, level-headed policies and measures, information about what infectious disease experts (e.g. physicians, scientists) are doing to combat the coronavirus (to hopefully give readers some light at the end of the tunnel), and so on.

4. Of course, I don't claim to be anyone special. I'm just a "nobody" as I think I've said several times in the past. That's why I've cited numerous infectious disease experts, physicians, scientists, epidemiologists, medical scientific literature, and other relevant resources in my posts. This way, people won't depend on me or anything I say, but instead they can see and learn for themselves what the experts and medical scientific journals and so on are saying. At most, I want to be nothing more than a conduit for people to find good information for themselves. And in that vein I even made a post titled "Which medical experts do I follow?".

5. And I agree with what Steve has argued as well (e.g. his posts about damaging our economy, his criticisms about church closures).

We all make mistakes

1. I think it might be good to interject the following at this point. I don't fault people who are attempting to do their level best but who fail and have to retract what they've said. In fact, I think that probably reflects personal honesty and scientific integrity.

2. After all, we all make mistakes even in the best of times let alone in the midst of a pandemic. It's like we're trying to fight an eight-headed hydra. We're trying to fight the coronavirus at the same time we're trying to learn about it. And the coronavirus doesn't stand still and wait for us to catch up before it attacks, but it itself is constantly evolving, even as we are, and attempting to fight back against us.

3. What I do fault - or at least must question - is people who hold a position, are offered reason or evidence which weakens if not defeats their position, but refuse to concede their position, even refuse to engage the evidence or arguments they've been shown, and even keep persisting in repeating the same lines that others have already responded to. To me, it seems like they're sticking their fingers in their ears in order to remain in their own imaginary world.

Some ideas about dealing with confirmation bias

1. These are just some of my own ideas about how to deal with confirmation bias. I'm sure there are more and even better ideas out there if anyone else wishes to comment.

2. I think one help in mimimizing confirmation bias is to make sure to understand the arguments from both or all sides in a debate. It doesn't help to only read one side and ignore the other side. That's a recipe not only for confirmation bias, but all sorts of other biases too.

3. There's something to be said for steelmanning the other side's arguments so that they're presented in the best and brightest light possible.

4. Similarly it might be a helpful exercise to criticize our own positions. To test for any and every weakness in what we believe. To know our position better than the harshest critics know our position. To be an even harsher critic of ourselves than our harshest critics.

5. I think one issue with confirmation bias is that it doesn't look at the evidence in toto. Rather it tends to look at some evidence in isolation of other evidence. It may even cherrypick the evidence. That is, it picks out evidence that is favorable to its arguments, but ignores evidence to the contrary. Hence a good way to try to avoid confirmation bias is to look at the total or complete evidence. Or at least look at a sample of the evidence that fairly represents the complete evidence.

6. That said, as we all know, the coronavirus pandemic is an ongoing and evolving global crisis. The evidence itself is constantly changing. That makes it harder to pin down evidence. It's like trying to take hold of Jell-o in our hands without having it fall apart. Indeed, the evidence is so dynamic and rapidly evolving that even the experts are sometimes proven wrong after they make this or that pronouncement. Some may even have to retract what they've said. As such, I don't mean to imply we should look at the literal "complete evidence" in our pandemic, for that would be impossible inasmuch as we're still in the thick of it. What I mean is the evidence thus far. Likewise I think we have to make allowance that we are in a dynamic and evolving situation, which again makes it difficult to pin down the evidence.

7. Perhaps another issue in confirmation bias is that people don't know what's strong evidence and what's weak evidence. On the one hand, that's completely understandable because responding to the coronavirus involves dealing with technical matters across a spectrum of disparate fields. In fact, even among experts, many only know the ins and outs of their own areas of expertise, but not other specialized fields. They may know a lot about the virus itself, but they don't have a clear idea about how it works to harm human beings, let alone how to properly treat and manage it. They may know a lot about statistics and epidemiological models, but they don't fully appreciate what public health officials need to balance in their policies. They may know a lot about how government and public health can help respond to an epidemic, but they don't know how drugs are researched and developed. So if this is difficult for the experts to grasp, how can non-experts ever hope to understand it?

On the other hand, one doesn't need to be experts in order to understand the basic or fundamental elements in responding to the coronavirus pandemic. For one thing, much of what makes evidence sound or unsound, or good or bad, is fairly intuitive. For example, we know if a study only has maybe 20 people in it, then in general 20 people are less likely to be representative of (say) the entire population of the US let alone the world. Another example, we know if the 20 people are all from the same background (e.g. same sex/gender, same race/ethncity, same social background, same age), then it's again less likely to be representative of (say) a diverse nation like the US let alone the world as a whole. A third example, we know if we are looking at a study that isn't randomly picked, but participants are handpicked for certain incomparable features, then it's less likely to be a fair comparison. A fourth example, we know if a study includes people who know exactly what drug they're getting in advance, then that could bias them toward or against the drug. A fifth example, we know if a study doesn't include at least two groups of people, say Group A and Group B, but a study only has one group of people, then all things equal it's less likely to be a fair comparison, since we need to be able to compare how the drug works in one group that has taken the drug vs. another group that has taken a placebo or conventional treatment. Anyway, we could go on and list more, but I think these suffice to get the point across.

Moreover, we live in an extraordinary time. Thanks to the coronavirus pandemic in the information age, the future is now! In a way, everyone is receiving a free education in infectious disease simply by listening to experts talk on social media. In fact, colleges and universities have moved to online education, and many seem to find it's at least equal to if not better than attending lectures and classes in person. Academic institutions and scientific journals have made all their resources and publications on the coronavirus free for the general public. Bleeding-edge medical research that might take years to be approved has been fast-tracked. Of course, it takes good judgment and discernment to sort the wheat from the chaff, but all I'm saying is much of the information is available for free if anyone wishes to make an effort to learn.

8. I think another issue in confirmation bias is that people wouldn't have confirmation bias if they didn't make any confirmations in the first place! What I mean is it's perfectly fine and good to say "I don't know" or even to say nothing. However, the problem is there are a lot of people today who always want to give their opinion on something, even if they don't really have any idea what they're talking about. Prov 17:28 comes to mind: "Even a fool who keeps silent is considered wise; when he closes his lips, he is deemed intelligent."

A corollary to this is people need to know their limitations when they do give their opinions. They don't need to speak with confidence over something they're unsure about. Rather they tailor their opinions in accordance with what they know and what they don't know.

Now, I'm sure there are people (including me) who have been guilty of saying or implying more than they know (e.g. I've been cavalier in dismissing some drug studies). However I'm not talking about people who are guilty of this, admit they're wrong, but try to do what's right. As I've said, we all make mistakes, and all the more so with regard to a rapidly moving object like the coronavirus. Rather I'm talking about people who persistently keep making one confident statement after another without their statement measured to the state of the evidence. That sort of thing.

9. I suppose we should distinguish between evidence and predictions based on evidence. As I've mentioned in the past, I'm no epidemiologist, but my impression is predictions are sort of like weather forecasts. They might be generally accurate, but there's a not insignificant degree of inaccuracy too. Maybe less so when the timeframe is only hours or days away, but if we stretch out the timeframe to weeks or months, then predictions tend to become fuzzier. Or so it seems to me. As far as the coronavirus, of course the best predictions are based on the best reasoning and evidences, but still there's an element of uncertainty even in our best predictions.

10. I think a helpful place to go is an organization known as the Cochrane colloboration which publishes the Cochrane reviews. These reviews look at all the literature on a particular topic. Such as what does all the literature say about how effective drug x is in disease y. As far as I know, Cochrane hasn't done any review of the present coronavirus, because it's such a novel virus. But that's not why I recommend them. Rather my point is simply that Cochrane is really good at sniffing out biases including confirmation biases. That doesn't mean they're immune to bias, of course, but they do strive to meet certain criteria when it comes to their reviews. In that vein I'd recommend their recently updated Cochrane handbook as a helpful guide to avoiding all sorts of biases including confirmation bias. I realize it's somewhat technical, but one could simply use it as a reference work to read up on various types of biases in studies.

11. I'm sure much more can be said, and I hope others feel free to weigh in with their own thoughts, but this is what comes to mind for me right now.