#256: The "world" according to AI pushers
Reflections on the viral article
There is currently an alarmist post going viral on X, also published on Fortune, about how AI is about to render all our jobs obsolete and everything we know about life on Earth. As of writing, it has 80 million views on X alone and it’s been reshared and dissected widely. The “essay” was written by an AI company founder named Matt Shumer (and co-authored, very clearly, by his LLM of choice). Shumer, it turns out, has previously been documented as exaggerating the performance of his models for personal gain, but I read it before I knew that, laying in bed, my chest tightening as I went.
Mostly he employs a whistle-blower’s tone, positioning himself as someone who feels morally obligated to warn the public. But when, about halfway through, his 4,500-word warning turned into a full-throated pitch for adapting AI immediately, I felt a different kind of despair. X, owned by anti-social edgelord Elon Musk, slowly enshittifying as users get paid for going viral, makes for an increasingly perilous town square. Who is Matt Shumer? How does he stand to benefit from the world embracing AI in the ways he encourages? What’s the difference between a sales pitch and an op-ed? These are questions traditional publishers used to be tasked with answering before awarding an idea visibility.
I don’t mean to suggest his entire post is wrong or dishonest. He’s probably right on certain fronts, like that the average shittalker who’s still trotting out claims of chatbots “hallucinating” has an outdated understanding of AI capabilities. I believe him when he says the latest LLMs would surprise many of us and that they may fundamentally change the nature of white-collar work. But while reading his tidily formatted and actionable opus, which appears to be a sweeping piece about our imminent future, I noticed the world he described felt utterly weightless and abstract—everything of importance reduced to how it could be done online and quickly. It’s not a life I recognize.
Despite spending a lot of time on the internet and having a computer job, my life is still meaningfully shaped in the physical realm: taking care of myself and my daughter, preparing and eating meals, spending time with people I love, taking walks, feeling the air. Quotidian as it may sound, these things comprise the texture of my existence. This is what makes my life feel real and consequential; shifts in these areas would fundamentally change my human experience. I’m not trying to be dense, I understand that AI will first impact jobs and the economy and that those will have tangible downstream effects. But when tech guys try to tell us what life is like, or is going to be like, this tension between their myopic view of “the world” and the world most of us are living in feels important.
This gap reminds me of a newsletter I came across last year by Hollis Robins on how to tell if something is AI-written. She explains that the primary difference between LLM and human communication is that the former mainly communicates in concepts whereas the latter is rooted in imagery, memory, and experience. When a person tells us a story or explains an idea, they might include little anecdotes from their life or comparisons to physical things. Their language may be symbolic, but it’s typically referring back to something that actually exists. LLMs, meanwhile, mainly gesture at symbols themselves: everything is a signifier of another signifier. As Sam Kriss described it, “All they can do is pile concepts on top of one another until they collapse.” If the language doesn’t conjure anything for you—if nothing springs to mind when you read it—it’s probably AI, Robins writes. LLMs may well improve on this front, but their lack of humanity will always be felt, if only in our implicit understanding that no one is really there. To many of us, this matters.
When it comes to AI discourse, this gap between the abstract and the tangible transcends language. Sweeping statements on “the future” by people like Shumer always seem to paint a world populated only by concepts: efficiency, productivity, accuracy, capability, technology, progress, incentives, outcomes, predictability. At first blush, this stuff feels important and relevant—this is the language we’ve come to associate with bosses, experts, and academics—and on second blush, it feels unnervingly disconnected from what feels important about being alive. “To be fair to the machines,” Kriss wrote, “they have a serious disability: They can’t ever actually experience the world. This puts a lot of the best writing techniques out of reach.” What’s Shumer’s excuse?
In March 2023, I asked whether it was more concerning when AI sounded like us or when we sounded like AI. For me, the question still stands. Most things LLMs excel at are things that are (or had become) formulaic for humans, too: coding, contracts, data analysis, collating research, public statements, professional communication, customer service, rational strategic reasoning, imitation. This encompasses a lot of the tangible outputs of the white-collar labor force, and it’s awe-inspiring that machines can now do this work instantly. Obviously I believe many of these things still benefit from a human touch (more than many enthusiasts appreciate), but even so, I’m more impressed by LLMs than the average skeptic, and sometimes personally find them useful.
Part of the reason it’s hard to take the most prolific AI pushers seriously is because their claims that AI can (or will soon be able to) do everything a human can do, but better, seem to betray a belief that almost all human output and activity is formulaic. Shumer suggests that, as a software engineer, he is simply “the first” to be replaced, but he fails to reckon with the fact that his work is uniquely poised to be done by a machine. For years, engineers have been compensated as if they’re the most skilled people alive, so if they can be replaced, everyone else must be, too. But outside of an engineering context, their arguments weaken. When Sam Altman claimed that a new model of Chat GPT was “good at creative writing” and published a short story it had written as proof, I was stunned by what he considered “good.” It might have hit some marks from a technical perspective, but it was utterly empty and meaningless.
I think it’s helpful to keep in mind the politics held by people who believe that everything done on a computer—such as “reading, writing, analyzing, deciding, communicating”—will cease to be done by humans in the “medium term.” I don’t disagree because I think LLMs aren’t impressive, I disagree because I do not see these activities as inherently transactional. Even if we take at face value everything Shumer has to say about mass employment overhauls1 (which some / people / doubt), note that his advice for weathering this does not include policy change or anything truly expansive for the collective. It’s simply: start paying for AI and practice using it so you can get ahead of the rest of the losers who didn’t read my amazing post. (Also, now that you’re sufficiently terrified, follow me for more!) The salient parts of his argument notwithstanding, this reads like neoliberal propaganda.
It’s both alarming and comforting that AI is in the hands of people who hold such a shallow view of human value. On the one hand, I don’t trust these people to build and disseminate these tools responsibly. On the other, I don’t believe the world is theirs to shape—not to the extent they think it is.
ICYMI, last week I answered 12 more questions from readers about being a parent/having a toddler (covering: current highs and lows, preschool, tantrums, sleep, baby vs toddler parenting, future excitement, having kids before/after your friends, future worry, how to be a good aunt, food challenges, postpartum advice, mother rage, and my village).
Friday’s 15 things are here. The recs of the week were unconventional lullabies to sing to your kid at night and what to watch in the Olympics! Taking notes.
Hope you have a nice Sunday,
Haley
One of the more compelling responses to Shumer’s post that I found was Will Manidis’s “Tool Shaped Objects,” in which Manidis describes the AI industry as a market for feeling productive. He writes that the problem begins when “the feeling of work becomes the product, sold as work itself.” Where are the tangible outputs of LLMs? “The dominant narrative about AI is not what it has built, but the rate at which people are consuming it,” he writes. “AI is everywhere in consumption and almost nowhere in output. We are spending unprecedented sums to acquire, configure, deploy, and operate these systems, and the primary product of that spending is the experience of spending it.”
I feel like I could make fun of “Something Big is Happening” forever. Sure, the tasteless loser who would choose that grotesquely basic header image written in a fake handwriting font that looks like the text generated by a digital document signature software WOULD believe machines can develop taste.
As a teacher I was most offended by “knowledge is basically free now, for only $20 a month” line about an “infinitely patient” tutor who can “explain at any level” - one of many fantasies in this piece and based in a belief that teaching is merely “explaining.” Also very rich to think you will somehow distinguish yourself professionally in a positive way by using AI tools because “no one is doing this” when in fact AI tools are ubiquitously available because they have been forcefully embedded with no opt-out in almost every part of our digital landscape. Even the examples he mentioned about law fail to mention that lawyers are routinely being sanctioned for citing case law that doesn’t exist because they or their staff used AI to write a brief. 404 media has good journalistic coverage of these incidents and other impacts of AI on workplaces, like “workslop” submitted by employees using AI having the effect of creating more work for others on their team, who have to try to parse and clean up the AI generated mess.
Like you mentioned, considering the politics of AI boosters is important. They almost never acknowledge the intellectual theft at mind boggling scale or the low wage training labor in places like Kenya that these models were built on. They fail utterly to consider the ethical, emotional, and material realms of being human. People who believe human labor will shortly be rendered useless by AI are usually not doing much of the labor that sustains their lives themselves.
On robots doing physical labor - I recommend the 2008 film “Sleep Dealer” - very prescient sci-fi that also has a lot to say about immigration.
I'm a software engineer, and the last few months have been pretty miserable. It's not that AI can now do my entire job and I'm sitting around doing nothing, but there's been a significant shift in the expectations of my role that has sucked the fun, creativity, and satisfaction out of solving complex problems or building something. The people going off on twitter always have an agenda, they’re rarely the ones in the weeds working in extremely complex codebases or connected to the problems that often arise when you try to ship code that hasn’t been thoughtfully considered. To act like AI tools aren’t still making basic mistakes and don’t need to be guided and instructed makes me feel like i’m being gaslit!!
Of course there are aspects of coding that are formulaic, and AI tools can be given more and more context and clearer guidelines for what’s good/what’s bad. Humans often make mistakes too. But the people controlling where things are headed are the “progress at any and all cost” kind of dudes, with 0 consideration for the real costs this technology will have on people and society (not to mention $$$) and it feels like we’re all at the mercy of their choices.