Kahneman, Tversky and the CIA – Grappling with Biases and Heuristics
By Andrew Grenville, Chief Research Officer | August 18, 2020
In the world of market research, we really have one fundamental job: generating insights that help our organizations thrive. It sounds simple, but it is not.
We, however, are not the only professionals who must generate insights. The intelligence community have much the same task—only the stakes are much higher. We have much to learn from their approach to generating insights, while grappling with innate biases and limitations.
The journey to an insight is not straightforward. We must dodge avalanches of information, clamor over mounds of cognitive biases, and try to avoid tumbling down the chute of tunnel vision. The intelligence community learned this early on and drew on the pioneering research of Daniel Kahneman and Amos Tversky starting in the late 1970s. The story of how they developed techniques to combat biases and heuristics traces back to World War II and a Yale historian with a specialization in 19th century French politics named Sherman Kent.
Roots of Intelligence Analysis
Just prior to the U.S. joining World War II, Kent was recruited to join a group of scholars in the newly formed Research & Analysis Branch (R&A) of the Office of Strategic Services in Washington. In the spirit of serving the war effort, they called themselves the “Bad Eyes Brigade.” Their purpose was to provide intelligence on foreign countries and potential theaters of battle. The people best versed in this information were academics: historians, political scientists, geographers, anthropologists, economists, and psychologists. The problem was, no one had ever done this before, and there was no model for how to do analysis.
There were few in Washington, Kent writes, “who could give any guidance as to how to go about the business at hand. What intelligence techniques there were, ready and available, were in their infancy. Intelligence was to us at that period really nothing in itself; it was, at best, the sum of what we, from our outside experience, could contribute to a job to be done.”
Kent was pondering how the analysis was to be done because he had a keen interest in methodology. Just prior to joining the intelligence effort he had published Writing History, a book that espoused an approach to analysis that emphasized the importance of the scientific method, intellectual plasticity, and deep skepticism of the evidence at hand. The approach was “akin to the method of science which Francis Bacon put forth in the early seventeenth century,” he writes.
“It consists of gathering facts…[and] forming hypotheses on the basis of these facts, of testing these hypotheses for traces of one’s own ignorance or bias, of cleansing them if possible. The goal of research is to build better hypotheses than already exist and to establish them as relatively more true: it is to reveal a sharp picture of what happened and make a closer approach to actuality than anyone has yet contrived.” A colleague later noted, “In many of the passages [of Writing History] one need only substitute the words ‘intelligence officer’ for ‘historian.’” Or, indeed, insights analyst.
As academic and author J. Peter Sclobic writes, “When Kent joined the Research and Analysis Branch, U.S. intelligence analysis was a haphazard affair. By the time he left the CIA, it was an orderly process staffed by career analysts who hewed to a strict methodology that prioritized objectivity in the face of ambiguity and neutrality in the face of ideology. Kent’s insistence on disinterested analysis gave ONE a degree of independence from Washington politics, and his reverence for the scientific method legitimized its work on prediction.”
Kent was insistent on questioning assumptions and biases and favored generating multiple hypotheses, but he did not prescribe a specific set of rules or exact methods for extracting insights from information.
As much as his methodology was “akin” to science, it was not science per se.
The next generation took a step closer. Enter CIA methodologist Richards J. Heuer Jr., whose hugely influential The Psychology of Intelligence Analysis helped propel the intelligence community toward a more scientific approach to analysis.
Into the psychology of intelligence analysis
Richards “Dick” Heuer, a philosophy student, was recruited out of university to join the CIA. He worked in operations, which sounds kind of dull. But here is how the CIA describes the role of an operations officer on the careers page of its website: “you will focus on clandestinely spotting, assessing, developing, recruiting, and handling non-US citizens with access to foreign intelligence vital to US foreign policy and national security decisionmakers. You will be expected to build relationships based on rapport and trust using sound judgment, integrity, and the ability to assess character and motivation.” In other words, Dick Heuer spent 24 years as a spy. Then he moved on to methodology.
Heuer describes it this way: “In 1975 I arranged to shift from the operations side of CIA to the analysis side in order to work in a new Analytic Methodology Division that had recently been created in direct response to criticism by prominent academics that the agency’s analytic methods were way out of date.” At the 1977 International Studies Association (ISA) conference, Heuer chaired a panel entitled “Quantitative Approaches to Political Intelligence: The CIA Experience.” There he met an Israeli officer who was tasked with the same challenge of improving intelligence analysis methodology in his country.
“Immediately after that ISA presentation, I was approached by a man with a foreign accent who said, ‘Vee need to talk. Zee answer ees not in zee numbers, it’s in zee head,’” Heuer recounts. “That sounded interesting, so we had lunch, and he told me about Kahneman and Tversky’s path-breaking work in cognitive psychology.”
“I then read Kahneman and Tversky’s work, and that was the beginning of my continuing interest in cognitive psychology. I then began writing a series of papers and lecturing to CIA training courses on the cognitive problems and to a lesser extent the group process and organizational problems one encounters in doing analysis. One of the lessons I learned during that period is still very applicable today. If you want to change how analysis is done, you need to show analysts how you can help them. I could do that after my research in cognitive psychology taught me about how the mind works.”
In Psychology of Intelligence Analysis he collected together the learnings from these papers and lectures. The essence of his premise is that “Intelligence analysts should be self-conscious about their reasoning processes. They should think about how they make judgements and reach conclusions, not just about the judgements and conclusions themselves.”
Heuer, the former philosophy student, opens the book with a section about “thinking about thinking” before reviewing various perceptual and cognitive biases from the literature. He builds on Kent’s encouragement to consider multiple hypotheses and to be skeptical, and caps the book off with a description of a technique he devised to encourage a more scientific and systematic approach to the consideration of alternative hypotheses.
ACH—a structured analytic technique
Heuer devised a technique called Analysis of Competing Hypotheses, known as ACH. It’s a fascinating approach to coming to an insight. It forces people to look beyond the data and to consider why things are as they are. And, cognizant of our biases, it attempts to make us grapple with our cognitive limitations.
ACH also encourages people to step outside their own perspectives and connect with others in the journey to insight. It chronicles people’s thinking and the steps they took—so that others can retrace those steps and see how conclusions were arrived at.
Heuer lays out his process in eight steps. Each step is more like a guideline than a specific, prescribed process. This makes the use of ACH more accessible and open to interpretation. But this variability also has the effect of making it difficult to measure the effectiveness of using ACH. Nonetheless, ACH provides great food for thought when thinking about how we get to an insight.
Here is Heuer’s “Step by Step Outline of the Analysis of Competing Hypotheses.”
“1. Identify the possible hypotheses to be considered. Use a group of analysts with different perspectives to brainstorm the possibilities.
2. Make a list of significant evidence and arguments for and against each hypothesis.
3. Prepare a matrix of hypotheses across the top and evidence down the side. Analyze the ‘diagnosticity’ of the evidence and arguments—that is, identify which items are most helpful in judging the relative likelihood of the hypotheses.
4. Refine the matrix. Reconsider the hypotheses and delete evidence and arguments that have no diagnostic value.
5. Draw tentative conclusions about the relative likelihood of each hypothesis. Proceed by trying to disprove the hypotheses rather than prove them.
6. Analyze how sensitive your conclusion is to a few critical items of evidence. Consider the consequences for your analysis if that evidence were wrong, misleading, or subject to a different interpretation.
7. Report conclusions. Discuss the relative likelihood of all hypotheses, not just the most likely one.
8. Identify milestones for future observation that may indicate events are taking a different course than expected.”
Learning from ACH
The worlds of intelligence analysis and insights are similar, but not the same. So, the approach of ACH is not a direct fit with the process of generating an insight, but there is much to be learned from its example. “A principal advantage of the analysis of competing hypotheses is that it forces you to give a fairer shake to all the alternatives,” Heuer states.
This is but one of the structured analytic techniques (SAT) Heuer and others devised, but it is the most influential and the most enduring. You can learn about it in some detail in Heuer’s book The Psychology of Intelligence which is available for free as a pdf on the CIA’s website.
This look at the approach of the CIA and the forces that pushed them toward adopting a more rigorous and consistent approach to analysis is instructive. The world of insights needs to move away from approaching problems in isolation, ignorant of our biases and with a tendency to embrace the easy solution and get sucked into tunnel vision.
The approaches of Kent and Heuer are inspiring and suggest some strategies that insights could learn from. And while ACH may be a bit involved for a small tactical decision, it makes a great deal of sense when tackling bigger questions like informing resource allocation.
But a key question remains: is there proof it is effective? More on that in our next article.
This article has been adapted from Eureka! the science and art of insights, which is available now.