Anthropic is urging leading AI labs to consider slowing development, warning that cutting-edge models are advancing fast enough that they will soon be able to improve themselves without direct human intervention.
The company says that the global ability to pause or slow down the development of AI “would probably be a good thing,” citing internal data on the acceleration of the models’ capabilities.
Using public benchmarks and unreported data from Anthropic, the Anthropic Institute shows that AI is already accelerating the development of AI systems. For example, Anthropic engineers are shipping an average of 8x more code per quarter today than they will from 2021-2025.
Technical trends discussed in a paper by Anthropic show that AI systems will become much more capable in the coming years. This will have huge implications. AI that can build itself would be a major development in the history of technology – one that could bring enormous benefits to the world in science, healthcare and beyond. But full recursive self-improvement could also increase the risks of losing human control over AI systems. If systems become capable of fully creating their own successors, the ways in which we secure them, monitor them and shape their behavior will change because we will not be designing it.
If it were possible to slow down the development of this technology to give us more time to deal with its enormous implications, we think that would probably be a good thing. But if a slowdown simply allows less cautious actors to catch up with technological progress, it could leave us all less safe. Without a global coordination mechanism, companies and governments will have to make difficult decisions about security while already under competitive and geopolitical pressures.
We believe that it would be good for the world to have the option to temporarily slow or pause the development of AI, to allow social structures and alignment research to keep pace with technological advances. The Anthropic Institute will conduct research – in collaboration with many others – and take action to help build the systems that would require a reliable slowdown or pause. These systems would allow AI developers to verify that others globally have indeed stopped or slowed down, and that some malicious entity could not use a coordinated slowdown to secretly advance.
Although the press releases will range from very select to rare, I said I'd pass...because sometimes the editors hide.
