Confirmation Bias and Its Limits

By Yuval Levin

Wednesday, April 09, 2014

The last few days have provided both a good laugh and some food for thought on the important question of confirmation bias—people’s tendency to favor information that confirms their pre-existing views and ignore information that contradicts those views. It’s a subject well worth some reflection.

The laugh came from a familiar source. Without (it seems) a hint of irony, Paul Krugman argued on Monday that everyone is subject to confirmation bias except for people who agree with him. He was responding to anessay Ezra Klein wrote for his newly launched site, Vox.com, which took up the question of confirmation bias and the challenges it poses to democratic politics. Krugman acknowledged the research that Klein cites but then insisted that his own experience suggests it is actually mostly people he disagrees with who tend to ignore evidence and research that contradicts what they want to believe, while people who share his own views are more open-minded, skeptical, and evidence driven. I don’t know when I’ve seen a neater real-world example of an argument that disproves itself. Good times. 

Klein’s actual essay (which Jonah ably took up yesterday, as did the always wise David Harsanyi at the Federalist), is more serious and interesting, though.

To some extent Klein’s piece, too, is an example of confirmation bias in action — as is a great deal of the work of all of us who write about politics for a living, I’m sure. He opens with what he presents as a critique of the assumptions underlying our politics, but is probably better understood as a critique of the assumptions underlying vox.com:

    There’s a simple theory underlying much of American politics. It sits hopefully at the base of almost every speech, every op-ed, every article, and every panel discussion. It courses through the Constitution and is a constant in President Obama’s most stirring addresses. It’s what we might call the More Information Hypothesis: the belief that many of our most bitter political battles are mere misunderstandings. The cause of these misunderstandings? Too little information — be it about climate change, or taxes, or Iraq, or the budget deficit. If only the citizenry were more informed, the thinking goes, then there wouldn’t be all this fighting.

I wonder if most people involved in politics in America really have so little respect for people who disagree with them as to imagine that the disagreement is just a misunderstanding. Many people seem, rather, to assume that differences are grounded in different priorities and worldviews that are rooted in moral and philosophical differences. That assumption is surely the source of much of the passion and intensity of our politics, and sometimes also of its depth and seriousness.

Above all, though, I can’t imagine why Klein would think that this idea “courses through the Constitution” in particular. It seems to me the Constitution is built on a far more sophisticated grasp of human limitations and on a sense of the permanence of parties, each of which can only ever hope to be partially right.

Some members of the founding generation believed political differences were rooted in inborn dispositions and temperaments and so were in a sense natural to human societies. Thomas Jefferson often expressed a form of this view (in terms most agreeable to his own disposition, of course), writing for instance in an 1802 letter to Joel Barlow:

    The division into Whig and Tory is founded in the nature of man; the weakly and nerveless, the rich and the corrupt, seeing more safety and accessibility in a strong executive; the healthy, firm, and virtuous, feeling confidence in their physical and moral resources, and willing to part with only so much power as is necessary for their good government; and, therefore, to retain the rest in the hands of the many, the division will substantially be into Whig and Tory.

Others suggested such differences had more to do with interests, including material interests. James Madison, for instance, offered a highly materialist version of this idea in Federalist 10, writing: “From the protection of different and unequal faculties of acquiring property, the possession of different degrees and kinds of property immediately results; and from the influence of these on the sentiments and views of the respective proprietors, ensues a division of the society into different interests and parties.” There are other causes of factional differences, Madison acknowledged, “but the most common and durable source of factions has been the various and unequal distribution of property.”

Whatever we may think of the materialism of Madison’s view of this question (and I think it is excessive), the constitutional system he helped design is surely built on the notion that society would always be riven by some set of profound differences that would not be readily resolved by better information. Instead of assuming a position above society from which different claims can always be objectively adjudicated, the system seeks to counterbalance those differences and to channel them through complicated institutions, to prevent any faction from gaining too much power for too long, and to force any party that seeks to deploy significant public power to construct a coalition broad enough as to most likely restrain its worst excesses.

I personally think the sources of our party differences lie at least as much in different philosophical outlooks as in different material interests or natural dispositions (though those do matter, surely). I think our Left and Right, very broadly understood, are each implicitly attached to a different set of ideas about human perfectibility and human limits, different notions of what the shape and purpose of society are, and different understandings of what kinds of knowledge could be available to us to address social problems. I wrote a book on that subject recently, and won’t belabor it here. But I think these different outlooks incline us to emphasize different kinds of questions and prioritize different kinds of goods in the effort to improve our society, and that this means the Left and Right often talk past one another in our political debates. It also means that both are very deeply vulnerable to confirmation bias.

My view of that subject leads me to think that arguments (and facts and figures and other information) do matter a great deal in politics, though maybe not as much as any of us might like. But it also leads me to think that the permanence of the limits of human reason and the fact that every party only sees part of the whole means no one — not even people who agree with me — is ever likely to be entirely right when it comes to big, complicated social questions. In terms of the structure of our governing institutions, it therefore leads me to the same conclusion Madison reached, which is that we need a system that keeps all sides from getting too much power and forces them to confront and compromise with one another in practical terms. However much we might regret it, this is a system that assumes we will never fully persuade one another in politics, and indeed that assumes we are probably all wrong — which we probably are. It is therefore a political system that makes us less stupid, not (as Klein suggests in the title of his piece) more so.

But the fact that among the roots of our political differences is a difference about epistemology — about what we can know in politics and how we can know it—means that this Madisonian conclusion is often itself one of the points of debate between Left and Right.

American progressives have long contended that as social science enables us to overcome some of the limits of what we know, it should also be permitted to overcome the constitutional limits on what government may do. They take themselves to be an exception to the rule that all parties see only parts of the whole, and therefore an exception also to the ubiquity of confirmation bias, and so they demand an exception to the rule that no party should have too much raw power. This is basically what Paul Krugman was getting at.

But the progressives’ understanding of how social science can come to know society and of how such knowledge might be put into effect has itself been a point of great contention with conservatives — who tend to think that a society’s knowledge exists mostly in dispersed forms and therefore that public policy should work largely by enabling the dispersed social institutions of civil society, local community, and the market economy to address problems from the bottom up through incremental trial-and-error learning processes. This is a view of public policy that is generally compatible with the limits the constitutional system places on government, while the progressive preference for consolidated knowledge and centralized action tend to be far less so, and not by coincidence.

Many serious people on the left don’t believe this disagreement about the proper way to obtain and act on social knowledge is a legitimate difference, or rather they treat the technocratic attitude of the modern Left as common sense and therefore as not requiring justification. The Left is concerned with ends, they say — the betterment of the poor, the improvement of living conditions — and is purely pragmatic about means. I’m sure they believe this quite genuinely, but the means of politics and policy can only be separated from the ends (which is to say, the means can only be left unlimited) this way if you take for granted the worldview of the modern Left and its understanding of how people thrive and how societies work. It is also no coincidence, therefore, that people who claim that progressivism is pragmatism strongly incline to centralized technocratic approaches to policy — which leave little room for experimentation, make it difficult to evaluate success and failure, and create programs that are very hard to change or discard when they fail, and therefore aren’t very pragmatic at all.

That progressive preference for centralized expertise and authority ultimately assumes the possibility of a vantage point outside society from which the social scientist and social manager can view the whole and not just parts. Klein quotes Dan Kahan, the Yale professor whose research about confirmation bias he highlights, articulating the ideal of such detachment: “My hypothesis is we can use reason to identify the sources of the threats to our reason and then we can use our reason to devise methods to manage and control those processes.” This has long been the hope of progressives seeking to use social science to overcome the limits of politics, and indeed the hope of rationalist philosophers throughout the history of the West.

But understanding human limitations does not mean we can overcome them. It only means we can’t pretend they don’t exist. It should point us toward humility, not hubris. And in politics and policy, understanding the limitation that Klein highlights should point us away from technocratic overconfidence and toward an idea of a government that enables society to address its problems through incremental, local, trial-and-error learning processes rather than centrally managed wholesale transformations of large systems.

Klein, to his credit, seems genuinely skeptical about whether Kahan’s ambitions are plausible. For one thing, unlike Krugman, he acknowledges the vulnerability of everyone in our politics to confirmation bias. He accepts the proposition that it’s as prevalent on the Left as on the Right, though he does seem to have some trouble finding examples on his own side of the aisle. They are not so hard to come by. He points out, for instance, that confirmation bias can be especially problematic regarding issues “where action is needed quickly to prevent a disaster that will happen slowly.” Madison pointed to the very same problem. And while Klein can only think of conservative attitudes toward climate change when looking for an example of this, he could easily have found another instance much closer to home if he were not inclined to ignore it.

But in the end, it is not clear if Klein accepts Madison’s notion that these limits on human knowledge and on its effectiveness are permanent and universal, and so accepts the need for a system that limits the damage that hubris might cause while making room for a constructive politics and an effective government. Rather, he suggests that these limits on our knowledge are a function of systems set up to keep our minds closed. “Washington is a bitter war between two well-funded, sharply-defined tribes that have their own machines for generating evidence and their own enforcers of orthodoxy,” Klein writes. “It’s a perfect storm for making smart people very stupid.”

“The silver lining,” Klein continues, “is that politics doesn’t just take place in Washington.” And he suggests that outside the gilded capital, people should be far better able to judge policies by their outcomes and remain free of partisan sentiments and of confirmation bias. Here again is the progressive assumption that, free of the nefarious influence of landed interests, the people can in fact overcome what only seem to be the limits of our reason and knowledge. And so, having come around at last to the peculiar mix of populism and technocracy that has always characterized American progressivism, Klein ends up suggesting that it is after all our system of government that is the problem here. He began his essay by wrongly attributing to the Constitution the view that political differences are just misunderstandings so that he could conclude by stepping up to defend a version of that view himself, and could offer his political vision as a vindication of the constitutional order rather than a rejection of it.

But a rejection of it is what it seems to be. “If American politics is going to improve, it will be better structures, not better arguments, that win the day,” Klein concludes.

My own sense is that if American politics is going to help American society improve, it will be better policies that make that possible, and the constitutional structures we have — precisely because they are built upon a realistic understanding of human limits and a sense that government’s purpose is to sustain the space in which society can function and to enable everyone to benefit from what happens there — are very well suited to allowing for that kind of politics of bottom-up improvement. The liberal welfare state is not. It is the task of conservatives in the coming years to make to the public—concretely, issue by issue, with evidence and argument, as both a political vision and a policy agenda — the case for the former over the latter, secure in the conviction that arguments matter . . . at least up to a point.