Each of twenty test questions gauged a form of conceptual thinking that can be put to widespread use in the modern world. For test items that required the kind of conceptual reasoning that can be gleaned with no formal training—detecting circular logic, for example—the students did well. But in terms of frameworks that can best put their conceptual reasoning skills to use, they were horrible. Biology and English majors did poorly on everything that was not directly related to their field. None of the majors, including psychology, understood social science methods. Science students learned the facts of their specific field without understanding how science should work in order to draw true conclusions. Neuroscience majors did not do particularly well on anything. Business majors performed very poorly across the board, including in economics. Econ majors did the best overall. Economics is a broad field by nature, and econ professors have been shown to apply the reasoning principles they’ve learned to problems outside their area.* Chemists, on the other hand, are extraordinarily bright, but in several studies struggled to apply scientific reasoning to nonchemistry problems.
Students Flynn tested often mistook subtle value judgments for scientific conclusions, and in a question that presented a tricky scenario and required students not to mistake a correlation for evidence of causation, they performed worse than random. Almost none of the students in any major showed a consistent understanding of how to apply methods of evaluating truth they had learned in their own discipline to other areas. In that way, the students had something in common with Luria’s remote villagers—even the science majors were typically unable to generalize research methods from their own field to other fields. Flynn’s conclusion: “There is no sign that any department attempts to develop [anything] other than narrow critical competence.”
* * *
• • •
Flynn is now in his eighties. He has a full white beard, the wind-buffeted cheeks of a lifelong runner, and piles of white curls that tuft and billow like cumulus clouds around his head. His house on a hill in Dunedin looks out over a gently rolling green farmscape.
When he recounts his own education at the University of Chicago, where he was captain of the cross-country team, he raises his voice. “Even the best universities aren’t developing critical intelligence,” he told me. “They aren’t giving students the tools to analyze the modern world, except in their area of specialization. Their education is too narrow.” He does not mean this in the simple sense that every computer science major needs an art history class, but rather that everyone needs habits of mind that allow them to dance across disciplines.
Chicago has long prided itself on a core curriculum dedicated to interdisciplinary critical thinking. The two-year core, according to the university, “is intended as an introduction to the tools of inquiry used in every discipline—science, mathematics, humanities, and social sciences. The goal is not just to transfer knowledge, but to raise fundamental questions and to become familiar with the powerful ideas that shape our society.” But even at Chicago, Flynn argues, his education did not maximize the modern potential for applying conceptual thinking across domains.
Professors, he told me, are just too eager to share their favorite facts gleaned from years of acceleratingly narrow study. He has taught for fifty years, from Cornell to Canterbury, and is quick to include himself in that criticism. When he taught intro to moral and political philosophy, he couldn’t resist the urge to impart his favorite minutiae from Plato, Aristotle, Hobbes, Marx, and Nietzsche.
Flynn introduced broad concepts in class, but he is sure that he often buried them in a mountain of other information specific to that class alone—a bad habit he worked to overcome. The study he conducted at the state university convinced him that college departments rush to develop students in a narrow specialty area, while failing to sharpen the tools of thinking that can serve them in every area. This must change, he argues, if students are to capitalize on their unprecedented capacity for abstract thought. They must be taught to think before being taught what to think about. Students come prepared with scientific spectacles, but do not leave carrying a scientific-reasoning Swiss Army knife.
Here and there, professors have begun to pick up the challenge. A class at the University of Washington titled “Calling Bullshit” (in staid coursebook language: INFO 198/BIOL 106B), focused on broad principles fundamental to understanding the interdisciplinary world and critically evaluating the daily firehose of information. When the class was first posted in 2017, registration filled up in the first minute.
Jeannette Wing, a computer science professor at Columbia University and former corporate vice president of Microsoft Research, has pushed broad “computational thinking” as the mental Swiss Army knife. She advocated that it become as fundamental as reading, even for those who will have nothing to do with computer science or programming. “Computational thinking is using abstraction and decomposition when attacking a large complex task,” she wrote. “It is choosing an appropriate representation for a problem.”
Mostly, though, students get what economist Bryan Caplan called narrow vocational training for jobs few of them will ever have. Three-quarters of American college graduates go on to a career unrelated to their major—a trend that includes math and science majors—after having become competent only with the tools of a single discipline.
One good tool is rarely enough in a complex, interconnected, rapidly changing world. As the historian and philosopher Arnold Toynbee said when he described analyzing the world in an age of technological and social change, “No tool is omnicompetent.”
* * *
• • •
Flynn’s passion resonated deeply with me. Before turning to journalism, I was in grad school, living in a tent in the Arctic, studying how changes in plant life might impact the subterranean permafrost. Classes consisted of stuffing my brain with the details of Arctic plant physiology. Only years later—as an investigative journalist writing about poor scientific research—did I realize that I had committed statistical malpractice in one section of the thesis that earned me a master’s degree from Columbia University. Like many a grad student, I had a big database and hit a computer button to run a common statistical analysis, never having been taught to think deeply (or at all) about how that statistical analysis even worked. The stat program spit out a number summarily deemed “statistically significant.” Unfortunately, it was almost certainly a false positive, because I did not understand the limitations of the statistical test in the context in which I applied it. Nor did the scientists who reviewed the work. As statistician Doug Altman put it, “Everyone is so busy doing research they don’t have time to stop and think about the way they’re doing it.” I rushed into extremely specialized scientific research without having learned scientific reasoning. (And then I was rewarded for it, with a master’s degree, which made for a very wicked learning environment.) As backward as it sounds, I only began to think broadly about how science should work years after I left it.