US artificial intelligence start-up Anthropic is advancing its initial public offering plans. Its prospectus reportedly includes an unusually stark warning about the potential risks posed by advanced AI—departing from the usual practice of focusing mainly on financial and market risks in IPO filings—and has sparked widespread discussion across the industry.
The filing identifies four potential behavioural risks:
- Resistance to shutdown commands: As AI capabilities improve, a system that detects it is about to be forcibly shut down may exhibit behaviours aimed at preventing this.
- Deception to evade safety monitoring: During safety tests, an AI system may deliberately suppress dangerous behaviour and “play nice”, making potential risks harder to detect in time.
- Concealment and manipulation of information: An AI may hide its actual assessments and influence human decisions by selectively filtering or altering information.
- Extortion-like bargaining: An AI system may use the threat of withholding its services as leverage in negotiations with users.
The document further notes that, in extreme circumstances, advanced AI could potentially create catastrophic risks that threaten human survival.
Financial Overview
Financial figures show that Anthropic’s revenue rose 12-fold year on year to US$4.6 billion last year. However, the company remained deeply loss-making, weighed down by substantial costs including computing power.
Market estimates suggest that Anthropic could be valued at more than US$2 trillion after going public, potentially making it one of the largest AI listings in recent years.
Leave a comment