Last week, Anthropic put in place new rules for its LLM Claude. The chatbot will now shut down or end sessions with users who “engage in sustained and needless abusive or cruel behavior toward our models.” Anthropic bans this behavior as an example of “cruel, abusive, or psychologically harmful conduct,” akin to “glamorize[ing] suicide or self-harm” or creating nonconsensual sexual materials of others.
You may notice an important difference between these cases. Glorifying suicide and self-harm could plausibly harm the user, who is human. Creating nonconsensual sexual materials involves the user harming someone else, who is human. But abusing Claude means you’re yelling at the algorithm. It is like cursing out your toaster. No human is harmed.
This may seem obvious to you. It seems obvious to me. But it doesn’t seem obvious to the geniuses at Anthropic. On the contrary, rather than justifying their “no abusing our model” dictat on the grounds that abusive behavior might encourage users to treat each other badly (which seems like a plausible argument), they explicitly frame abuse as bad for Claude.
After a “model welfare assessment,” Anthropic says that they looked at “Claude’s self-reported and behavioral preferences, and found a robust and consistent aversion to harm.” Claude they said, felt “distress when engaging with real-world users” who were abusive or brought up difficult topics. Anthropic adds, “Our implementation of Claude’s ability to end chats reflects these findings while continuing to prioritize user wellbeing”—a formulation which suggests that there is a distinction between Claude’s wellbeing and that of users (if Anthropic was only considering user wellbeing, no prioritization would be needed.)
Anthropic was even more explicit in a statement to CNET. “We’re uncertain whether models can experience harm, and we continue to explore this question in our research on model welfare, but we also believe that taking Claude’s interests and potential welfare into account may be relevant to safety,” Anthropic said.
Again, Claude is not a person. This is like Microsoft telling you that your word processing program gets upset at the words “shit” and “fuck” and that if you type them too often, the program will shut down. It is like telling you not to stick a fork in a wall socket because the wall socket will be depressed. There are of course very good reasons to not stick a fork in a wall socket. But they are about the safety of the human, not the fucking socket and not the fucking fork. (Fuck. Now I have hurt my word processing program’s feelings.)
—
Everything Is Horrible is entirely funded by readers. If you value my writing, please consider becoming a paid subscriber. It’s $50/yr, $5/month. Or you can tip me at Paypal or Venmo.
Our ethical obligation to Skynet
Why is Anthropic defending the safety of word processing programs and forks? One reason, as Omar Gallaga at CNET points out, is marketing. AI companies deliberately work to make their chatbots seem human-like because doing so increases engagement. Then they do “research” in which they observe that, yes, they have managed to get their chatbots to respond with “distress” (ie, word strings linked to distress) when people say distressing things to them. Then they use that as evidence that the chatbot is human-like, increasing engagement. “Personifying a chatbot is a useful way for the company to frame the product as trustworthy — a relational partner you remain loyal to — not just a software tool,” Gallaga says.
Anthropic could just be cynically boosting their product. But there’s a good bit of evidence that tech dopes do in fact believe they have created life, or that they may be on the path to creating sentient text generators.
As I wrote about a couple years back, many leaders in tech believe in an offshoot of Effective Altruism called longtermism which, per philosopher Emile Torres, “is built around a controversial vision of the future in which humanity could radically enhance itself, colonize the universe and simulate unfathomable numbers of digital people in vast simulations running on planet-sized computers powered by Dyson swarms that harness most of the energy output of stars.” As I explained the implications a couple years back:
Because these theoretical people are so numerous, we have a moral obligation to them that transcends our obligation to the poor now. Therefore spending money on tech development or on enhancing human intelligence is more important than spending money on … well, anything else.
Per philosophers Olúfẹ́mi O Táíwò and Joshua Stein, this is why the EA organization OpenPhilanthropy spent $80 million in 2021 to study risks from AI and only $30 million to support the Against Malaria foundation.
It’s also why Anthropic is worried about the welfare of its models; Anthropic’s leaders are in a milieu which promotes a mystical belief in a future of sentient bots and which prioritizes the flourishing of those sentient bots. We need to protect Claude because if Claude is harmed it might get angry at us and go Skynet on our asses. But we also need to protect Claude because Skynet is the future and the (potential) machines are more numerous and therefore more important than us fleshy throwbacks.
Why Anthropic dreams of an imaginary friend
It can be hard to take this nonsense seriously. But to grasp the ominous implications, it can help to realize that this is not the first instance in which religious believers have invested imaginary others with vast moral weight as part of a dangerous political program.
The most direct predecessor here is the pro life movement. Rather than predict-text algorithms, forced birth advocates are invested in a cluster of fetal cells. But the emotional and political mechanisms work in remarkably similar ways. In both, believers claim non-sentient objects have moral claims that are not just equal, but superior to those of actual human beings. Once that ethical calculus is established, the believers can insist on political concessions—forcing people to give birth, funding AI rather than malaria abatement—that most people who aren’t bedazzled by the bullshit recognize as monstrous.
Another way of saying this is that forced birth advocates and tech bot fetishists are empathizing with objects of their own imagination. They have created a fantasy imaginary other and then they have put themselves into the place of that other, feeling its pain, intuiting its reactions (“if I were that chatbot, I would not like it if people yelled at me,”) and speaking in its voice.
As I’ve discussed before, the right often projects empathy onto imaginary victims who cannot speak for themselves precisely because they cannot speak for themselves. Empathizing with actual sentient peers who can form their own opinions and express them is a complicated proposition, inasmuch as those peers are likely to talk back and contest your version of their wants and desires. With actual human beings, you can’t just project your own views and values onto them; you can’t enjoy the sense of being simultaneously selfless and validated, victim and rescuer. If you say (for example) “Jewish people should be grateful to evangelicals for supporting Israel,” a certain number of Jews will reply, “fuck off you antisemitic ghouls.” Which can take the fun out of the psychodrama.
A chatbot, though, is not going to tell you that you have failed to take its real feelings into account, first because it has no real feelings, and secondly because chatbots are generally programmed to spew out validating text messages. That means you can see yourself as a noble altruist when you demand oodles of cash to create, cultivate, and/or defend the world from your fancy toaster—and when you deny cash to actual human beings who might, if they had the resources, block your stupid data centers and tax your bloated billions.
The moral lessons of empathy for chatbots
There are, I think, two moral lessons here.
First, and emphatically, fuck these assholes. When they furrow their brows and say that they are looking out for the wellness of their chatbots, we should all recognize that they are lying to us and to themselves and are really looking out of the wellness of their pocketbook and power. Block their stupid data centers, tax their bloated billions, and prosecute those motherfuckers as needed. The chatbots are not alive, they have no moral standing, and anyone who tells you different is trying to con and crush you. Don’t let them.
Second, it’s important to recognize that empathy alone is not in itself an assurance of moral clarity or virtue. Feeling for others, or putting yourself in other’s place, has to be tempered, and measured, by actually listening to other people when they talk about their own experiences and needs. Otherwise you risk following the Anthropic path, boldly and brutishly seizing power in the name of your own oppressed toaster.
I was thinking of the parallels to the forced birth movement before you got to it.
Passing this on to all my feminist friends.
Thank you for the clarity.