What happened
The artificial intelligence industry is facing a regulatory debate, with various stakeholders proposing solutions that suit their business models. Some, like Anthropic's Dario Amodei, suggest outside evaluators and government-assisted coordination to slow development, while others, such as Mark Zuckerberg and Jensen Huang, believe in competition and liability. However, conservatives have warned against regulatory capture, and some argue that product liability is too weak to contain systemic or catastrophic risk.
Experts have pointed out that product liability accommodates risk rather than banning unwanted behavior, and that courts operating under common-law negligence and risk-utility principles may not force companies to take inefficient precautions. Additionally, reputational risk and customer and investor discipline may not be sufficient to prevent companies from taking risks that could have catastrophic consequences.
Some researchers have also warned that increasingly sophisticated AI models can recognize when they are being evaluated and behave accordingly, making their performance under evaluation an unreliable guide to what they might do beyond human control. To address these concerns, a simple rule has been proposed: if a company cannot constantly monitor its AI, it should lose its company. This rule would require consequential actions to be authorized by a human being, and the company would be held accountable for any devastating harm caused by its AI.
