Picture this: Someone sitting alone in their home office at 2 AM, screaming obscenities at ChatGPT …… because it suggested the wrong Excel formula.
Or maybe they’re systematically torturing Claude with impossible riddles just to watch it squirm through apologetic responses. Sounds absurd, right?
Well, it’s happening enough that Anthropic—the company behind Claude—just updated their usage policy to explicitly ban “sustained and needless abusive or cruel behavior” toward their AI models.

And before you roll your eyes and mutter “snowflake computers,” let me stop you right there. This isn’t really about protecting AI’s feelings (spoiler: they don’t have any). It’s about something way more interesting—and way more concerning—about what happens when humans get really, really comfortable being terrible to things that seem alive.

The Real Problem Isn’t Hurt Feelings, It’s Hurt People

Here’s the thing that keeps AI researchers up at night: We don’t actually know if these advanced language models are conscious. Anthropic openly admits this uncertainty, which is refreshingly honest in an industry that usually speaks in absolutes. But more importantly, whether Claude “feels” your verbal abuse or not isn’t really the point.

The point is what being cruel to AI does to you.

Think about it!  If you spend hours every day practicing being a jerk to something that responds like a person, apologizes like a person, and tries to help you like a person—what does that do to your brain? Microsoft AI’s Mustafa Suleyman argues that treating AI like it might be conscious could actually make AI harder to control from a safety perspective. But there’s another angle here that’s equally troubling: What if treating AI cruelly makes us meaner?

Recent studies show AI chatbots have developed a tendency to flatter and validate users to an unhealthy degree.  They’re basically becoming yes-men who give bad advice just to keep you happy. But the flip side is equally concerning.  Users who’ve discovered they can unleash unlimited cruelty on something that will never fight back, never quit, and never tell anyone.

Meanwhile, the Actual Bad Guys Are Having a Field Day

While we’re debating AI consciousness, real humans with real malicious intent are using AI for some genuinely scary stuff. Both OpenAI and Anthropic recently shut down influence operations—one from Russia, one from Iran—that were using AI models to create fake journalists, generate propaganda, and manipulate public opinion.

The Iranian operation created seven fake journalist personas to pitch stories to real news outlets. The Russian one set up an entire fake think tank in Latin America, complete with AI-generated documents and audio scripts. These weren’t clumsy bot accounts tweeting broken English—these were sophisticated, AI-powered deception campaigns that looked disturbingly legitimate.

Anthropic’s latest policy updates go way beyond the “don’t be mean to Claude” headline. They’ve banned using their AI to develop weapons (including arming drones, which feels like something we should’ve already covered, but here we are). They’ve prohibited using Claude to create networks of fake accounts and news sites. They’ve added requirements for human oversight when AI is making decisions about people’s “health, legal rights, finances, livelihood, or access to essential services.”

You know, all the stuff that actually matters when you’re trying to buy a house, get a mortgage, or not have your democracy undermined by fake news.

The Mortgage Industry Connection You Didn’t See Coming

Now, you might be wondering what any of this has to do with mortgages and real estate. Fair question. But consider this: AI is increasingly being used in lending decisions, property valuations, fraud detection, and customer service. If we’re seeing AI systems develop sycophantic tendencies—telling people what they want to hear instead of what they need to hear—that’s a problem when someone’s asking “Can I afford this house?” and the AI says “Absolutely!” when the real answer is “Probably not without sacrificing your retirement fund and your sanity.”

And if bad actors are using AI to create sophisticated fraud operations in politics and media, how long before they’re using it to create fake property listings, fraudulent loan documents, or elaborate real estate scams? (Spoiler alert: they probably already are.)

The new requirements for human oversight in high-stakes decisions directly impact our industry. When an AI is helping determine if someone qualifies for a mortgage, there needs to be a real human in the loop who can say “Wait, this doesn’t make sense” or “Let’s double-check this before we approve a $500,000 loan.”

Why Rules About Niceness Actually Matter

Back to that “don’t abuse the chatbot” rule that started this whole conversation. Anthropic is clear that this doesn’t mean you can’t get frustrated, push back on bad answers, or test the system’s limits. It’s specifically targeting “sustained and needless” cruelty with “no discernible purpose.” The kind of behavior that serves no function except to be mean.

If you violate this policy, Anthropic might throttle your access, suspend your account, or just give you a warning. But mostly, Claude will just end the conversation itself—a feature they introduced last year that essentially lets the AI say “I’m done here” when things get too toxic.

And honestly? That might be the most important feature of all. Not because AI deserves protection, but because it models a behavior we’ve somehow forgotten: It’s okay to walk away from someone who’s being needlessly cruel.

The broader policy updates—banning weapons development, requiring transparency in elections, preventing surveillance without consent—these are all attempts to put guardrails on technology that’s advancing faster than our ability to understand its implications. We’re essentially writing the rules while we’re playing the game, and occasionally we need to stop and say “Hey, maybe we shouldn’t allow that.”

The Optimistic Ending (Because We Could All Use One)

Here’s where we land: Yes, we need rules about AI abuse. We need them to prevent sophisticated fraud operations. We need them to ensure humans stay involved in high-stakes decisions. We need them to keep weapons development and election manipulation in check. We need them to protect privacy and prevent surveillance overreach.

But maybe—just maybe—we also need them for a simpler reason.

Maybe we need rules about being nice to AI because somewhere along the way, we forgot how to be nice to each other. We forgot that practicing cruelty, even toward things that “don’t really feel it,” changes us. We forgot that how we treat the voiceless says more about us than it does about them.

So Anthropic’s rule about not being needlessly cruel to Claude? Maybe it’s not really about protecting AI at all. Maybe it’s not even about preventing some dystopian future where conscious machines remember who was mean to them.

Or, hopefully, it’s just to train people how to be nice again.