Anthropic warns AI self-improvement threshold closer than thought

Anthropic warned this week that artificial intelligence systems are approaching the threshold for autonomous self-improvement faster than previously anticipated, with the company's Claude model now generating more than 80% of its own codebase. The US technology firm stated that this figure stood in the single digits in early 2025, marking a dramatic acceleration in automated software development. According to the company's five-stage AI development framework, current technology has reached the fourth level, where models autonomously execute code and delegate tasks to other systems.

The fifth and final stage involves autonomous agents capable of designing and training their own successor models, potentially enabling continuous self-improvement cycles with limited human oversight. Anthropic published these findings in a recent technical report that adds to growing concerns about the pace of AI advancement, according to the agency.

Global warnings and regulatory calls

The technical developments emerge against a backdrop of sustained warnings from academics, technology leaders and religious figures about the risks posed by increasingly advanced AI systems. In March 2023, the Future of Life Institute published an open letter signed by Elon Musk and other prominent figures calling for a six-month pause on training systems more powerful than GPT-4. The letter cautioned that AI approaching human-level intelligence could trigger massive job losses, widespread misinformation campaigns and a potential loss of human control over civilization.

Pope Leo XIV addressed the issue in his encyclical Magnifica Humanitas, released in May, calling for an outright halt to AI development and urging nations to decouple the technology from military and economic competition. These appeals have gained little traction among major technology firms, which continue to announce significant advances in autonomous capabilities.

Investment surge continues

The global technology sector is projected to increase AI spending by 47% year-on-year in 2026 to $2.59 trillion, with further growth to $3.49 trillion expected in 2027, according to a Gartner report released in May. This investment surge represents one of the primary obstacles to slowing development despite repeated calls for caution from public figures and safety advocates.

The projections indicate that financial momentum behind autonomous AI research continues to build regardless of the warnings issued by the Future of Life Institute and religious leaders. Spending is expected to reach $3.49 trillion by 2027 as firms race to develop the autonomous capabilities described in Anthropic's five-stage framework.