Seymour thought for a while. Moments earlier, he had estimated it would take about two more years. Faced with Kahneman’s question about other teams, he said he had never even thought to compare this instance to separate projects, but that about 40 percent of the teams he’d seen never finished at all, and not a single one he could think of took less than seven years.
Kahneman’s group was not willing to spend six more years on a curriculum project that might fail. They spent a few minutes debating the new opinion, and decided to forge ahead trusting the about-two-years wisdom of the group. Eight years later, they finished, by which point Kahneman was not even on the team or living in the country, and the agency that asked for the curriculum was no longer interested.
Our natural inclination to take the inside view can be defeated by following analogies to the “outside view.” The outside view probes for deep structural similarities to the current problem in different ones. The outside view is deeply counterintuitive because it requires a decision maker to ignore unique surface features of the current project, on which they are the expert, and instead look outside for structurally similar analogies. It requires a mindset switch from narrow to broad.
For a unique 2012 experiment, University of Sydney business strategy professor Dan Lovallo—who had conducted inside-view research with Kahneman—and a pair of economists theorized that starting out by making loads of diverse analogies, Kepler style, would naturally lead to the outside view and improve decisions. They recruited investors from large private equity firms who consider a huge number of potential projects in a variety of domains. The researchers thought the investors’ work might naturally lend itself to the outside view.
The private equity investors were told to assess a real project they were currently working on with a detailed description of the steps to success, and to predict the project’s return on investment. They were then asked to write down a batch of other investment projects they knew of with broad conceptual similarity to theirs—for instance, other examples of a business owner looking to sell, or a start-up with a technologically risky product. They were instructed to estimate the return for each of those examples too.
In the end, the investors estimated that the return on their own project would be about 50 percent higher than the outside projects they had identified as conceptually similar. When given the chance at the end to rethink and revise, they slashed their own initial estimate. “They were sort of shocked,” Lovallo told me, “and the senior people were the most shocked.” The investors initially judged their own projects, where they knew all the details, completely differently from similar projects to which they were outsiders.
This is a widespread phenomenon. If you’re asked to predict whether a particular horse will win a race or a particular politician will win an election, the more internal details you learn about any particular scenario—physical qualities of the specific horse, the background and strategy of the particular politician—the more likely you are to say that the scenario you are investigating will occur.
Psychologists have shown repeatedly that the more internal details an individual can be made to consider, the more extreme their judgment becomes. For the venture capitalists, they knew more details about their own project, and judged that it would be an extreme success, until they were forced to consider other projects with broad conceptual similarities. In another example, students rated a university a lot better if they were told about a few specific science departments that were ranked in the top ten nationally than if they were simply told that every science department at the university was ranked among the top ten. In one famous study, participants judged an individual as more likely to die from “heart disease, cancer, or other natural causes” than from “natural causes.” Focusing narrowly on many fine details specific to a problem at hand feels like the exact right thing to do, when it is often exactly wrong.
Bent Flyvbjerg, chair of Major Programme Management at Oxford University’s business school, has shown that around 90 percent of major infrastructure projects worldwide go over budget (by an average of 28 percent) in part because managers focus on the details of their project and become overly optimistic. Project managers can become like Kahneman’s curriculum-building team, which decided that thanks to its roster of experts it would certainly not encounter the same delays as did other groups. Flyvbjerg studied a project to build a tram system in Scotland, in which an outside consulting team actually went through an analogy process akin to what the private equity investors were instructed to do. They ignored specifics of the project at hand and focused on others with structural similarities. The consulting team saw that the project group had made a rigorous analysis using all of the details of the work to be done. And yet, using analogies to separate projects, the consulting team concluded that the cost projection of £320 million (more than $400 million) was probably a massive underestimate. When the tram opened three years late, it was headed toward £1 billion. After that, other UK infrastructure projects began implementing outside-view approaches, essentially forcing managers to make analogies to many outside projects of the past.
Following their private-equity-investor experiment, the outside-view researchers turned to the movie business, a notoriously uncertain realm with high risk, high reward, and a huge store of data on actual outcomes. They wondered if forcing analogical thinking on moviegoers could lead to accurate forecasts of film success. They started by giving hundreds of movie fans basic film information—lead actor names, the promotional poster, and a synopsis—for an upcoming release. At the time, those included Wedding Crashers, Fantastic Four, Deuce Bigalow: European Gigolo, and others. The moviegoers were also given a list of forty older movies, and asked to score how well each one probably served as an analogy to each upcoming release. The researchers used those similarity scores (and a little basic film information, like whether it was a sequel) to predict the eventual revenue of the upcoming releases. They pitted those predictions against a mathematical model stuffed with information about seventeen hundred past movies and each upcoming film, including genre, budget, star actors, release year, and whether it was a holiday release. Even without all that detailed information, the revenue predictions that used moviegoer analogy scores were vastly better. The moviegoer-analogies forecast performed better on fifteen of nineteen upcoming releases. Using the moviegoers’ analogies gave revenue projections that were less than 4 percent off for War of the Worlds, Bewitched, and Red Eye, and 1.7 percent off for Deuce Bigalow: European Gigolo.
Netflix came to a similar conclusion for improving its recommendation algorithm. Decoding movies’ traits to figure out what you like was very complex and less accurate than simply analogizing you to many other customers with similar viewing histories. Instead of predicting what you might like, they examine who you are like, and the complexity is captured therein.
Interestingly, if the researchers used only the single film that the movie fans ranked as most analogous to the new release, predictive power collapsed. What seemed like the single best analogy did not do well on its own. Using a full “reference class” of analogies—the pillar of the outside view—was immensely more accurate.