đź§ We Need Smart People Who Give a Shit
Something I learned a long time ago, something that's been proven to me time and again, is that the world is held together by smart people who give a shit. This truth always comes into stark focus when I learn how a complex system functions—or, rather, how it barely functions.
When I worked at Stripe I learned a lot about various banking and money movement systems, and it's terrifying how flawed and fragile these systems are. ACH is a great example. It's a complex system we rely on every day that is just barely good enough to function. It's old, riddled with flaws, and encumbered with legacy baggage. And yet, despite all of those problems, ACH usually works great and is generally considered to be safe, cheap, and reliable.
The reason there's a mismatch between the technical reality of the ACH system and how it functions in practice is due to a single factor: smart people who give a shit. Every problem with ACH is fixed, patched, bypassed, overridden, routed around, or manually handled by people who have two things:
- Intimate knowledge of the system
- The drive and determination to force the system to work
Those two things feed into each other. You have to care about a complex system to dive deep enough to learn how it works, and when you know a lot about how a complex system works you care more about it, especially if the system is important or critical infrastructure.
Now, you can replace "ACH" above with a myriad of other complex systems—everything from huge infrastructure to nice-to-have apps—and everything I said remains true, both about the systems themselves and the people who keep them running. The world runs on these systems, and they depend on smart people who give a shit. Take those people away, and the systems collapse.
And collapse is what we're starting to see. Some recent examples of various failures from just this past week or so: Microsoft 365, Air Traffic Control, United Airlines' systems, the NSA's network, many major banks, Apple's systems, and more. We're told these failures and problems are not related. They're not the result of a cyberattack. They're not the fault of generative AI.
I guess that depends on how you look at it.
I don't have inside information about what's going on, but my experience dealing with complex systems tells me there's a common through line to all of these disruptions: there aren't enough smart people who give a shit.
So where did they go?
Capitalism has never been particularly kind to smart people who give a shit (or people in general... but I digress). Capitalism does, however begrudgingly, recognize that smart people who give a shit are required to build and maintain complex systems that serve capitalist goals. Thus, those people have been accommodated and incorporated into the system by virtue of their necessity and, in many cases, their rarity.
Now, though, generative AI is here. To the people running the show—leadership and management—it seems like they can get the same or better value (and even more output!) from a machine. The machine is cheaper, works longer, doesn't take sick days or vacations, doesn't ask for any benefits, and never pushes back against a bad idea.
People who don't do the actual work—the people who aren't in the trenches—have a hard time telling the difference between a flawed system held aloft by the drive and determination of smart people who give a shit and slop provided by generative AI. To them it's all just output: words you can read, code that runs, and pixels that look nice to their eye. It's all the same to them, so why does it matter where it comes from?
It matters because generative AI isn't smart, and it doesn't give a shit. What seems like intelligence is an illusion propped up by a massive amount of data strip-mined from the commons with weights that nudge the output toward a generally desirable shape. What seems like care, concern, and attention is just generative AI doing what it's designed to do: telling you what you want to hear. The math and logic that power LLMs are as cold, unfeeling, and indifferent as the laws of physics.
Smart people who give a shit notice things. They connect dots. They notice when things don't feel right. They know when something is about to go off the rails, and they know how to get things back on track. They know how to handle edge cases. They know how to manually handle a unique situation. They know what the right thing to do is, and they know when they need to put in extra effort or time to make something happen. Generative AI falls far short in these areas, but this is the glue that holds complex systems together. More prompting and more output isn't a substitute for this.
It's hard for capitalist leadership to internalize any of that, though. It would be such a relief for them, such a burden lifted, to replace all those smart people who give a shit with generative AI. It would be so much easier to win at capitalism if generative AI lived up to the hype, and there's so much pressure for leadership and management to believe that the hype is real.
Thus, a lot of the smart people who give a shit have been laid off, and the ones that remain are hollowed out. Telling people to stop doing the job they signed up for in favor of babysitting LLMs so they can do that job instead is a great way to make people less smart about the system they used to work on and give less of a shit about it. Every prompt moves them one step further away from the system they used to know inside and out. Every generated output that becomes part of the system means those people understand less about how the system actually works. Those people are robbed of the things that make them smart (the struggle of building and learning) and the things that made them gave a shit (a sense of ownership and close connection with the system).
We've moved from a world dependent on complex and fragile systems held together by smart people who give a shit to a world of complex and fragile systems run by generative AI. The people running the generative AI are not as smart, nor do they give as much of a shit. There are, of course, exceptions (which is why a lot of things are still humming along), but we are on a course where the number of smart people who give a shit are decreasing, and that's a big problem. As long as the number continues to go down, more systems will fail.
I think what we have here is a hot stove moment for capitalism. When you tell a young kid a stove is hot and they shouldn't touch it, you'll find there are two kinds of kids. For one kind of kid, the warning is the lesson, and they don't touch the stove. For the other kind of kid, the only lesson that works for them is being burned. They have to touch the stove at least once to learn the lesson. Capitalism was told the stove was hot, but their hand is on it right now.
Unfortunately, we're all getting burned.