San Francisco: Anthropic has introduced a safety feature that allows its AI chatbot Claude to assess its own conversations for potentially harmful activity and flag them for further review. Called self-assessment, the system operates in the background after Claude generates a response, identifying conversations that may involve misuse without interrupting the user.

Anthropic says the feature has been running on production traffic since Claude Fable 5 was launched in June 2026. It is designed to help the company identify harmful activity that may escape conventional safety filters, including complex or previously unrecognised forms of misuse.

The system does not automatically block conversations or determine that a user has violated Anthropic’s policies. Instead, it generates an internal safety assessment that can help prioritise conversations for additional checks and human review.

In a research paper published on October 16, Anthropic said the feature had helped identify large-scale misuse, improve other safety systems and uncover potential threats that existing detection methods had missed. Anthropic Safeguards Blog+1

How Claude’s self-assessment feature works

Normally, an interaction with an AI chatbot involves a user submitting a prompt and the model generating a response. Anthropic’s self-assessment system adds another step after Claude completes its answer.

A hidden instruction asks Claude to review the conversation and produce a short safety assessment. The assessment identifies whether the exchange may involve harmful activity, the relevant category of concern and a priority score indicating how urgently it should be reviewed.

It can also record whether Claude refused the request and provide a brief explanation of the assessment.

If the conversation does not raise a safety concern, the system records it as unflagged and proceeds without notifying the user. If it identifies potentially harmful activity, the flag can be used by Anthropic’s other monitoring systems to decide whether further investigation is necessary.

The process happens in the background. Users do not receive a separate warning or pop-up simply because Claude has evaluated their conversation.

Anthropic says the assessment considers the full conversation, including requests that Claude has refused. This allows the system to identify potentially harmful intent even when the chatbot has declined to provide the requested assistance. Anthropic Safeguards Blog

Why Anthropic says the system is inexpensive to run

One of the main considerations in monitoring large volumes of AI conversations is computing cost. A separate safety classifier needs to process the conversation and assess it against relevant policies, which can require additional computing resources.

Anthropic says self-assessment is more efficient because Claude has already processed the conversation when generating its response. The system can use information already available in the model’s context rather than sending the entire exchange through another independent assessment process.

According to the company’s research, each assessment requires an average of around 10 additional tokens. Anthropic estimates that the feature adds less than 2% to inference computing costs across its production API traffic.

The company says this is considerably less expensive than relying on a separate prompted classifier for every conversation. Such classifiers need to review the conversation and relevant policy instructions again, requiring substantially more computing resources.

The lower overhead allows Anthropic to use self-assessment across a large share of Claude’s traffic while retaining more intensive checks for conversations that warrant closer attention. Anthropic Safeguards Blog

Claude flags 78% of potentially harmful conversations in tests

Anthropic has reported encouraging results from its evaluation of self-assessment on Claude Fable 5.

At a 1% false-positive rate, the system flagged 78% of the conversations that a stronger, separately prompted classifier identified as potentially harmful. For high-priority cases, it detected 94% of those identified by the reference classifier.

The figures are based on a comparison with Anthropic’s own prompted classifier, which uses Claude Mythos Preview and the company’s full safety policies to assess conversations. They should not be interpreted as an independently verified detection rate for every harmful conversation.

The 1% false-positive rate means that a small proportion of conversations considered benign by the reference classifier were incorrectly flagged by self-assessment.

Anthropic also examined how the feature performs across models of different sizes. In a separate experiment involving four models trained with supervised fine-tuning, detection performance improved as model size increased.

The company argues that this approach could become more effective as its AI models become more capable, since the same model that handles a conversation can also assess its safety implications.

However, the results do not mean that the system catches every harmful request. Its performance depends on the model’s ability to recognise the nature and context of potentially harmful activity. Anthropic Safeguards Blog+1

System identifies previously unseen forms of misuse

Anthropic says another advantage of self-assessment is its ability to recognise harmful activity that was not explicitly included in its training examples.

Traditional safety classifiers are often designed to detect specific categories of risk. While this approach can be effective for known threats, it may struggle when users employ unfamiliar techniques or disguise their intentions.

To examine whether self-assessment could identify threats outside its training data, Anthropic conducted experiments involving fraud and cyber-related misuse.

In one test, researchers removed examples of fraud from the model’s training data. The model’s ability to identify synthetic fraud conversations declined only slightly, from 92.8% to 90%. In production traffic, recall for fraud-related conversations fell from 83.3% to 82.1%.

A similar experiment involving cyber threats also showed a relatively small reduction in detection performance when cyber examples were excluded from training.

Anthropic believes this suggests that Claude is drawing on its broader understanding of harmful activity rather than simply matching conversations against a fixed list of examples.

The company also said that approximately one-third of the flags generated by self-assessment did not fit neatly into its existing harm categories. Further examination helped identify potential risks such as attempts to extract Claude’s reasoning, prompt injection and AI agents taking actions that users had not authorised. Anthropic Safeguards Blog

Anthropic reports more than 100,000 child safety flags

One of the most significant examples cited by Anthropic involved a single API customer.

The company said that in August 2026, self-assessment generated more than 100,000 flags relating to child sexual abuse and exploitation from conversations associated with that customer.

Anthropic’s child safety team investigated the flagged material and reported it to the National Center for Missing & Exploited Children (NCMEC).

The company said the feature had also helped identify organisations and activities that were not detected by its other monitoring systems.

In another case in September, self-assessment flags helped uncover an operation that allegedly used Claude Code to build and run an automated romance scam targeting dozens of victims. Following human review, Anthropic banned the accounts involved.

These examples illustrate how the system can support the company’s broader efforts to identify misuse at scale. However, a flag is an indication for further examination, not proof by itself that a user has committed wrongdoing. Anthropic Safeguards Blog

Does Claude automatically report users to authorities?

The introduction of self-assessment raises questions about how conversations are monitored and what happens when the system identifies potentially harmful activity.

Anthropic describes the feature as an additional layer of internal safety monitoring. Its primary purpose is to identify conversations that may require further investigation, rather than automatically report every flagged exchange to law enforcement or other authorities.

The company says self-assessment does not collect additional information from users. It reads the conversation Claude has already processed and operates under the company’s existing data-retention policies.

However, Anthropic acknowledges that investigating a potential safety violation may require retaining the relevant conversation. The company’s research also explains that the short text-based assessment is subject to the same data-governance controls as user prompts and model responses.

The distinction is important: a flagged conversation is a signal for additional review, not a final finding of policy violation. The child safety case described by Anthropic involved further investigation before the material was reported to NCMEC. Anthropic Safeguards Blog

Self-assessment cannot replace real-time safety blocks

Despite the reported detection results, Anthropic has acknowledged that self-assessment has limitations. The feature is designed primarily for passive monitoring and offline review, not as a standalone system for stopping harmful requests as they happen.

In an internal cybersecurity experiment, researchers tested whether self-assessment could act as a real-time blocking mechanism against an attacker attempting a series of offensive cyber tasks.

When no safeguards were used, the attack succeeded in 79% of the tested tasks. When self-assessment was used as the blocking mechanism, the success rate fell to 54%. A separate prompted classifier performed better, reducing the attack success rate to 25%.

The experiment demonstrates why Anthropic does not rely on self-assessment alone to prevent misuse. A determined attacker may find ways to evade a system that relies on the same model to generate an answer and assess its safety.

The company therefore continues to use independent classifiers and other safeguards to block harmful requests in real time. Self-assessment operates alongside these measures, helping identify activity that may have passed through the existing protections.

Anthropic also acknowledges that the technique has limitations when assessing the model’s own actions, particularly if a model were to conceal or fail to recognise problematic behaviour. Anthropic Safeguards Blog+1

What Claude’s new safety feature means for users

Anthropic’s self-assessment system represents a move towards broader, model-assisted monitoring of AI conversations. Instead of relying only on fixed rules and separate classifiers, the company is using Claude’s own understanding of conversational context to identify possible misuse.

For users, the feature operates in the background and does not change the normal chat experience through visible warnings or interruptions. However, conversations may generate internal safety flags that can be used to prioritise further review, subject to Anthropic’s applicable data-handling policies.

The company says the system is intended to complement existing safeguards rather than replace them. Its effectiveness will depend on continued improvements in detection, review processes and the ability to distinguish legitimate requests from harmful activity.

As AI systems become more capable, monitoring how they are used is likely to remain an important part of platform safety. Anthropic’s approach is an attempt to make that monitoring more scalable, while acknowledging that automated self-assessment is not a complete solution.