Anthropic bans sustained abuse of Claude AI models

Anthropic, the artificial intelligence company behind the Claude chatbot, has unveiled a new policy prohibiting users from directing sustained and unnecessary abusive or cruel behavior toward its AI models, with the rules set to take effect on November 12, according to Anadolu Agency. The company published the updated guidelines on Wednesday, stating that the restrictions would apply only to extreme cases in which users repeatedly behave cruelly toward Claude without any discernible purpose.

The measures will not affect ordinary expressions of frustration, criticism of the model, dark themes in creative work, or legitimate testing and research activities. Anthropic said the ban targets only persistent behavior lacking any productive aim.

Safeguards and enforcement

Anthropic said Claude's existing capacity to end rare conversations with persistently abusive users would remain the primary enforcement mechanism for the new rules. The company introduced this feature for Claude Opus 4 and 4.1 in August 2025, instructing the models to terminate exchanges only as a last resort after attempts to redirect the interaction had failed.

The firm described the policy change as part of its broader research into possible AI welfare. It emphasized that the company remained "highly uncertain" whether Claude or other large language models possessed moral status or the capacity for suffering.

Broader policy updates

The announcement comes amid a wider debate over whether advanced AI systems could ever become conscious or deserve moral consideration. Anthropic has said the question is serious enough to justify low-cost precautionary measures despite lacking consensus on present-day AI sentience.

The updated usage policy also consolidates restrictions on deceptive campaigns and fake online activity. It clarifies prohibitions concerning weapons and surveillance and introduces additional safeguards for AI connected to equipment capable of taking physical actions.