When I asked Garry Kasparov, perhaps the greatest chess player in history, to explain his decision process for a move, he told me, “I see a move, a combination, almost instantly,” based on patterns he has seen before. Kasparov said he would bet that grandmasters usually make the move that springs to mind in the first few seconds of thought. Klein studied firefighting commanders and estimated that around 80 percent of their decisions are also made instinctively and in seconds. After years of firefighting, they recognize repeating patterns in the behavior of flames and of burning buildings on the verge of collapse. When he studied nonwartime naval commanders who were trying to avoid disasters, like mistaking a commercial flight for an enemy and shooting it down, he saw that they very quickly discerned potential threats. Ninety-five percent of the time, the commanders recognized a common pattern and chose a common course of action that was the first to come to mind.
One of Klein’s colleagues, psychologist Daniel Kahneman, studied human decision making from the “heuristics and biases” model of human judgment. His findings could hardly have been more different from Klein’s. When Kahneman probed the judgments of highly trained experts, he often found that experience had not helped at all. Even worse, it frequently bred confidence but not skill.
Kahneman included himself in that critique. He first began to doubt the link between experience and expertise in 1955, as a young lieutenant in the psychology unit of the Israel Defense Forces. One of his duties was to assess officer candidates through tests adapted from the British army. In one exercise, teams of eight had to get themselves and a length of telephone pole over a six-foot wall without letting the pole touch the ground, and without any of the soldiers or the pole touching the wall.* The difference in individuals’ performances were so stark, with clear leaders, followers, braggarts, and wimps naturally emerging under the stress of the task, that Kahneman and his fellow evaluators grew confident they could analyze the candidates’ leadership qualities and identify how they would perform in officer training and in combat. They were completely mistaken. Every few months, they had a “statistics day” where they got feedback on how accurate their predictions had been. Every time, they learned they had done barely better than blind guessing. Every time, they gained experience and gave confident judgments. And every time, they did not improve. Kahneman marveled at the “complete lack of connection between the statistical information and the compelling experience of insight.” Around that same time, an influential book on expert judgment was published that Kahneman told me impressed him “enormously.” It was a wide-ranging review of research that rocked psychology because it showed experience simply did not create skill in a wide range of real-world scenarios, from college administrators assessing student potential to psychiatrists predicting patient performance to human resources professionals deciding who will succeed in job training. In those domains, which involved human behavior and where patterns did not clearly repeat, repetition did not cause learning. Chess, golf, and firefighting are exceptions, not the rule.
The difference between what Klein and Kahneman documented in experienced professionals comprised a profound conundrum: Do specialists get better with experience, or not?
In 2009, Kahneman and Klein took the unusual step of coauthoring a paper in which they laid out their views and sought common ground. And they found it. Whether or not experience inevitably led to expertise, they agreed, depended entirely on the domain in question. Narrow experience made for better chess and poker players and firefighters, but not for better predictors of financial or political trends, or of how employees or patients would perform. The domains Klein studied, in which instinctive pattern recognition worked powerfully, are what psychologist Robin Hogarth termed “kind” learning environments. Patterns repeat over and over, and feedback is extremely accurate and usually very rapid. In golf or chess, a ball or piece is moved according to rules and within defined boundaries, a consequence is quickly apparent, and similar challenges occur repeatedly. Drive a golf ball, and it either goes too far or not far enough; it slices, hooks, or flies straight. The player observes what happened, attempts to correct the error, tries again, and repeats for years. That is the very definition of deliberate practice, the type identified with both the ten-thousand-hours rule and the rush to early specialization in technical training. The learning environment is kind because a learner improves simply by engaging in the activity and trying to do better. Kahneman was focused on the flip side of kind learning environments; Hogarth called them “wicked.”
In wicked domains, the rules of the game are often unclear or incomplete, there may or may not be repetitive patterns and they may not be obvious, and feedback is often delayed, inaccurate, or both.
In the most devilishly wicked learning environments, experience will reinforce the exact wrong lessons. Hogarth noted a famous New York City physician renowned for his skill as a diagnostician. The man’s particular specialty was typhoid fever, and he examined patients for it by feeling around their tongues with his hands. Again and again, his testing yielded a positive diagnosis before the patient displayed a single symptom. And over and over, his diagnosis turned out to be correct. As another physician later pointed out, “He was a more productive carrier, using only his hands, than Typhoid Mary.” Repetitive success, it turned out, taught him the worst possible lesson. Few learning environments are that wicked, but it doesn’t take much to throw experienced pros off course. Expert firefighters, when faced with a new situation, like a fire in a skyscraper, can find themselves suddenly deprived of the intuition formed in years of house fires, and prone to poor decisions. With a change of the status quo, chess masters too can find that the skill they took years to build is suddenly obsolete.
* * *
• • •
In a 1997 showdown billed as the final battle for supremacy between natural and artificial intelligence, IBM supercomputer Deep Blue defeated Garry Kasparov. Deep Blue evaluated two hundred million positions per second. That is a tiny fraction of possible chess positions—the number of possible game sequences is more than atoms in the observable universe—but plenty enough to beat the best human. According to Kasparov, “Today the free chess app on your mobile phone is stronger than me.” He is not being rhetorical.
“Anything we can do, and we know how to do it, machines will do it better,” he said at a recent lecture. “If we can codify it, and pass it to computers, they will do it better.” Still, losing to Deep Blue gave him an idea. In playing computers, he recognized what artificial intelligence scholars call Moravec’s paradox: machines and humans frequently have opposite strengths and weaknesses.
There is a saying that “chess is 99 percent tactics.” Tactics are short combinations of moves that players use to get an immediate advantage on the board. When players study all those patterns, they are mastering tactics. Bigger-picture planning in chess—how to manage the little battles to win the war—is called strategy. As Susan Polgar has written, “you can get a lot further by being very good in tactics”—that is, knowing a lot of patterns—“and have only a basic understanding of strategy.”