In Defense of Academia
Identifying the baby in the bathwater
There are many reasons to criticize modern academia: bureaucratic insanity, replication crises, absurd postdoc holding patterns, and detachment from the real world to name only a few.1 Sometimes it feels like everybody is down on academia, including academics.
But we may all be taking it for granted.
There are many societally valuable things that academia does incredibly well; very few proposals for new institutions (including my own!) have good ideas about how to replace those niches in the broader innovation ecosystem
.
Digging down to academia’s “core competencies” will hopefully serve several purposes:
Give people who want to make academia better a set of north stars to optimize around.
Give people who want to burn it all down a Chesterton’s Fence that they must contend with and hopefully some appreciation for what they would be destroying.
Help give people within the system cynicism-banishing reasons to keep doing great work.
A quick note on definitions: the academia I’m talking about isn’t just universities. While many academics are associated with universities, it’s not a perfect correlation: institutions like HHMI Janelia Farm and National Labs are still incredibly academic, despite having different structures and incentives than Universities. Instead, academia is a fuzzy-boundaried institution characterized by prioritizing knowledge-generation and engaging in research, training new researchers in the form of graduate students and postdocs (who also provide a lot of the hands-on labor), scientific publishing, principal investigators, and many other intertwined activities.
Training people who work on the knowledge frontier. The discovery and invention that drives economies, military prowess, and human well being depends on people with fingertip-knowledge of the deeply unintuitive research enterprise.2 Training these folks takes a lot of time and resources — training a PhD student costs roughly $300k. Involving trainees (which is what grad students are!) in the research process is not the most efficient way to do research! However, the mentor-apprentice relationship that ideally characterizes the interactions between a professor and a grad student may be the only way to incept the thought patterns of top scientists. Even outside this ideal scenario, spending years struggling in a lab seems to be the best way to develop deep research skills and intuitions. Perhaps there’s a faster or better way, but I haven’t seen any compelling alternatives yet.
Academia is one of the only places where active, cutting edge research labs are incentivized to train novices. A rule of thumb is that the average grad student is a net drag on productivity until their third year. No other institution has developed structures to support an unproductive worker for three years until they become productive. Someone who took that long to be net positive would be quickly fired in a market-driven setting. Even the great industrial labs of the past depended on universities to train their researchers.
In large part, grad student training as part of the academic research process exists through sheer cultural pressure to pay it forward. I haven’t seen any other institution even attempt to create this combination of culture where it’s expected to take a productivity hit for the sake of future generations and frontier research environment.
Disciplinary mixing. There are few other places where different disciplines can so thoroughly mix as academia. It’s shockingly low friction for a top-tier (say) chemist, geologist, and historian to just grab lunch. There are no other institutional settings where people are paid to be there just because they’re the best in their discipline (who also opted to stay in academia), regardless of what that discipline is. Some companies foster cross-disciplinary mingling but only when some particular mixtures of disciplines that happen to support that company’s broader goals (ie. Math, materials, and electronics at Bell Labs). But when no companies exist that need a specific disciplinary mixture, university campuses are the only places where that can happen. The many studies3 and anecdotes suggesting that amazing things come out of cross-disciplinary mixing are convincing: from molecular biology being the result of physicists bringing their tools to bear on biology to the transistor needing physicists, electrical engineers, and metallurgists all in close proximity. that I’m convinced this is an important thing for humanity.
Generating Open Knowledge. Prestige based on publications and the institution of academic publishing more generally have dramatically increased the flow of technical knowledge. Before scholarly journals, people like DaVinci and Galileo literally wrote their discoveries in cyphers so that they could horde their discoveries and potentially write a magnum opus near the end of their life. Keeping discoveries close to the vest is not a thing of the past: companies (and most non academic technical institutions) have strong incentives to keep as much secret as they possibly can. The counterfactual to the current publication system might not be a utopia of free-flowing micro publications; it might be a return to the old system where everybody hoards as much information as possible.
The academic culture of open publication in academia puts pressure on people outside of academia to follow suit. Researchers in various industries still see publishing as high status both because academia is high status and publishing is what academics do and also because they were often trained in academia and have peers who are still there.
Talent pools. Academic institutions create a pool for obscure talents and knowledge outside of market forces. Academic research (in the “arcane”/not-immediately-useful sense) enables not-particularly-in-demand ideas and talents to survive. Market forces are great for efficiency! But part of efficiency is weeding out useless skills and capabilities. However, we’ve seen over and over how arcane areas of knowledge are utterly useless until they’re not. Neural networks and quantum computing — considered purely academic topics for years — are two salient examples.
Tacit knowledge pervades the knowledge frontier, so just because it’s written down somewhere doesn’t actually mean that the civilizational “we” can do it. Academia is one of the few places that can keep these communities of practice alive.
International community. Academia is one of the few truly borderless institutions. There aren’t many institutions where people across the world genuinely come together for a single purpose. The cross-border letter writing, lecture circuits, and conferences that are the glue of academia have existed for hundreds of years. Few other institutions have normalized a system of inviting people from other organizations to just come and hang out and exchange ideas — it’s very special.
Taking ideas seriously. With its roots in philosophy, academia genuinely values ideas for their novelty and depth rather than just their production value. There are few other places where people will grapple seriously with other people’s ideas. In most other worlds, people will only engage with someone else’s ideas insofar as it will get them to an instrumental outcome like convincing a third party or unlocking a skill. Academics will actually dig into the minutiae of ideas and the why behind them. (This could be because of genuine curiosity or the desire to be right but either way ideas get explored in ways they wouldn’t otherwise.)
Orthogonal incentive systems. Most of the world outside of academia incentivizes doing things because they’re profitable or useful. This is actually a great way to allocate most of our resources. But academia’s orthogonal systems that incentivize doing things because they’re novel or awesome enables us to explore parts of idea space that would be inaccessible otherwise. There are few other lasting institutions that support these orthogonal incentives.
Actually basic research Academia is one of the few places that support genuine inquiry into how the world works without thought of how it might be relevant to most people. (Again, the root of academia is arcane!) Knowing what people ate 7000 years ago or how stars form may never raise GDP by a single dollar, but it enriches our civilization nonetheless.
Actionable? Conclusion
I’m aware that each of these points has a dark side. Academia certainly needs a lot of fixing and has been asked to do too much in the modern world.
But before we give in to cynicism or say “burn it all down and start from scratch” it’s important to step back and see how, at its core, academia is a uniquely valuable institution. Identifying the bright lights at its core is how we can start to fix it.
The principle of comparative advantage is a powerful driver of progress: when individuals and organizations focus on what they’re best at and offload other roles, everybody benefits. Competition drives companies to do this or die, but Universities and other academic institutions have aggregated a massive number of roles, with many of those roles falling on the professors and scientists. More roles have led to more tradeoffs between them, more context switching for scientists, and generally poorer performance across the board. In the same way that you don’t ask a neuroscientist to do accounting just because they’re smart enough to hack it, we’re asking universities and professors to do far too many things.
By asking “what is academia great at?” we can start to identify which things we should be asking academia (and by extension professors) to do and which things we should unbundle into other institutions (more on this soon). These new institutions could be everything from new ways of structuring large research efforts (like Convergent Research or Future House) to unconstrained pursuit of ideas (like Science House4) to building a home for pre-commercial technology research (like we’re doing at Speculative Technologies). It’s obviously optimistic, but identifying what academia is uniquely good at and then doubling down on those things may be how we return academia to its position as a beacon of discovery, curiosity, and progress.
Thanks to Jessica Alfoldi, Marissa Weichman, Adam Marblestone, and Boris Hanin for discussions that contributed to this piece.
See https://eiko-fried.com/antidotes-to-cynicism-creep/#1_So_Many_Problems for one of many descriptions of these problems if you’re not familiar.
See The Knowledge Machine for a book-length treatment of how unintuitive the modern scientific enterprise is.
See https://www.newthingsunderthesun.com/pub/vqahzl0l/release/17 for a great synthesis of the literature.
So many houses!
Another positive externality is that academia furnishes a source of labor for peer review. The vast majority of labor, both for reviewing papers and for organizing conferences, comes from researchers in academia. Industry researchers are not in a cultural milieu where reviewing (or serving on a program committee, if you're in ML) is expected. Of course, whether peer review provides societal value is now being questioned. I've previously argued that it does have value, in that it increases recall (sensitivity) to new ideas, particularly from people without prior fame or prestigious affiliations: https://calvinmccarter.substack.com/p/peer-review-worsens-precision-but-improves-recall
Well, this is certainly thought provoking and needed, speaking as one of the "burn it all to the ground" guys. Ironically, far better written and more accessible than an academic paper would be. In no particular order:
- It's not really the case that companies don't train people from scratch, is it? Historically that was the only way people got trained, as the vast majority never went to university. Many people still do. Apprenticeship schemes are found widely throughout Europe. And I should say that in my own field of computing, most people teach themselves. CS degrees are often worthless at actually teaching programming from scratch, so self-teaching carries the load. It's not at all clear that universities are required for training.
- International community?! Conferences, international meetups and international collaboration are hardly academic! Why do you think only academia does this? The average programmer has been to way more industry conferences than academic, assuming they go to such events. I've been to both types but industry conferences were far bigger, more useful and in fact more international. The software industry routinely organizes hundreds of thousands of international collaborations in (open source) projects. Other industries also work through international orgs, often standards or research related (e.g. MPEG). This argument is very weak.
- Generating open knowledge. I think this is one of the best arguments. Though patents do the same thing with the difference that the peer reviewers are actually incentivized properlyC. The main problem with patents is that because patent rights are overly strong and penalties (in the USA) triple for wilful infringement, most people in industry don't read them. They might be useful, but you're better off independently reinventing what's in them for financial/legal reasons. But then again most people don't read academic papers either.
- Cross discipline collaboration. No, this is silly. Academics are notorious for totally failing at cross-discipline collaboration, to the extent that corporate labs routinely smoke them just by fixing this one thing. Look at how easily DeepMind crushes entire academic research fields through the magic of pairing up AI researchers with researchers in other fields. Academic fraud involving stats and programming tasks is absolutely notorious, often rooted in cockups due to a refusal to hire people with the right skills. Why? Because those skills are more valuable than theirs are, and they hate the idea of spending their grant money on expensive programmer/data scientist types rather than more people inside their own field, so they wing it and end up publishing nonsense. This is not an argument to keep academia.
- Obscure talent pools / orthogonal incentives. Quantum computing isn't a good example here because IIUC it's mostly driven by corp labs at IBM, Google and some startups. I don't think academia has done much to develop talent there, and anyway the promise of the tech hasn't panned out. But the wider argument you make is that we need systems that aren't optimized by capitalism because it might optimize away something useful. The problem with this nice sounding idea is that it leaves academia without any way to prune the genuinely and offensively useless. This is one of the biggest weaknesses of the whole enterprise and why so many people are turning against it. The refusal to defund useless or outright harmful work has led to academia incubating dangerously extreme and idiotic ideologies. It also floods the literature with garbage that nobody has the time to wade through, which obviates quite a few of the other suggested advantages.
I am one of the relatively rare types who works in a corporate research lab, and so I read scientific papers as part of my job. I've done this for years. I mostly avoid academic papers these days because I've learned the hard way that the ROI just isn't there. I wasted years of my life studying a purely academic sub-field that corporate labs were ignoring, because I believed the claims made by the academics writing the papers. Then one day I went to an obscure conference with them and learned that the way they presented their work in their talks was very different to how they presented it in papers and press releases. Major problems were revealed that somehow hadn't made it into the actual published works, and after I got some of them drunk one of the most celebrated researchers told me they would never actually use their own research for anything! What a fool I was. There were good reasons nobody tried to apply their output after all. And yet this sub-field churns out hundreds of papers a year. Other sub-fields are hardly better. The best papers come from other big corporate labs, and that's where the time is best spent. Or just, y'know, doing research and pushing the boundaries forward a little bit at a time. Blue sky stuff is rarely the right way to go.