US-based AI firm Anthropic warns AI development is advancing at a pace that could soon see agents building, training and improving themselves without human input \u2014 recommending a slowdown in development.<\/p>\n
In a blog post published<\/a> Thursday, Marina Favaro, lead at the Anthropic Institute, and Anthropic co-founder Jack Clark said agents can already run code themselves, delegate hours of work to other agents and could be on the cusp of taking over completely.<\/p>\n \u201cFor most of AI\u2019s history, humans drove every step in its development cycle. But at Anthropic, we are delegating a growing share of AI development to AI systems themselves, which is speeding up our work,\u201d they said.<\/p>\n \u201cTaken far enough, and given enough compute, that trend points to an AI system capable of fully autonomously designing and developing its own successor,\u201d Favaro and Clark added.\u00a0<\/p>\n AI development is advancing at a pace that could lead to agents improving without human input. Source: <\/em>Anthropic<\/em><\/a><\/p>\n There are concerns over what could happen if AI is able to become smarter on its own. In December, OpenAI said it is researching how to safely develop and deploy increasingly capable AI, including AI capable of recursive self-improvement. <\/p>\n \u201cWe want these systems to consistently follow human intent in complex, real-world scenarios and adversarial conditions, avoid catastrophic behavior, and remain controllable, auditable, and aligned with human values,\u201d it said. <\/p>\n The company is also hiring<\/a> a researcher for recursive self-improvement preparedness, which forms part of its Safety Research team. <\/p>\n AI model improvement has been roughly doubling every four months, rather than every seven months, according to Favaro and Clark. The role of humans is narrowing at each step, with Anthropic\u2019s Claude model authoring around 80% of the code merged into Anthropic\u2019s codebase.<\/p>\n \u201cWe are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for,\u201d they said. <\/p>\n \u201cOnce human- and AI-authored code quality reach parity, humans will stop writing code entirely and shift to only reviewing it. But if they can\u2019t review code as quickly as Claude can generate it, human review will become the bottleneck to AI development,\u201d they added.<\/p>\n Favaro and Clark also said that slowing development to allow more time to address its \u201cimmense\u201d implications would be ideal.<\/p>\n Related: <\/strong><\/em>Modern robots impress, but are years away from replacing humans<\/strong><\/em><\/a><\/p>\n In April, Anthropic ruled<\/a> out releasing its AI model, Claude Mythos, to the public over concerns about the threat to global cybersecurity.\u00a0<\/p>\n Claude Mythos was able to easily create software exploits, leading Anthropic to rule out a public release for now. Source: <\/em>Anthropic<\/em><\/a><\/p>\n At the same time, a group of tech leaders, including some from Anthropic and OpenAI, released<\/a> an open letter on Thursday, urging lawmakers to enact stronger guardrails around the technology over concerns it could be used to overcome \u201cknowledge barriers\u201d that have historically prevented bad actors from creating biological weapons.<\/p>\n \u201cWe believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,\u201d Favaro and Clark said.<\/p>\n \u201cBut if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe. Without a global coordination mechanism, companies and governments will have to make difficult decisions about safety while under competitive and geopolitical pressures.\u201d<\/p><\/blockquote>\n AI agents are becoming increasingly popular, including among crypto users<\/a>. Some crypto executives have speculated that AI agents settling transactions could drive adoption and transaction volumes. Circle CEO Jeremy Allaire predicted in January that billions of AI agents<\/a> would operate on users’ behalf within five years.<\/p>\n Crypto investment firm Keyrock reported last month that AI agents<\/a> settling payments went from concept to reality in the past 12 months, with $73 million settled across 176 million transactions.<\/p>\n![\"\"](\"https:\/\/s3-images.ctmedia.io\/media\/content\/pasted-image-1341.png\")
<\/figure>\n<\/p>\n