Print

OpenAI safety employee quits, says ‘time for trial and error is over’

A former OpenAI safety employee who recently resigned criticized the company’s approach to AI safety, arguing that a fast-paced culture focused on rapid development increases the risk of failures.

Writing in “I Quit OpenAI Because Its Culture Is Broken,” published by the Atlantic on Saturday, David Robinson said AI companies, including OpenAI, were not being “nearly careful enough” and should place greater emphasis on safety expertise and research before developing more capable systems.

“The time for trial and error is over,” Robinson wrote, arguing that advanced AI systems require safeguards more akin to those used in industries such as nuclear power and aviation.

Robinson said OpenAI relies heavily on what it calls “iterative deployment,” releasing systems and strengthening safeguards when problems emerge.

The comments add to a debate within the AI industry over whether companies are moving too quickly to develop increasingly powerful systems. OpenAI and rival Anthropic have faced scrutiny after incidents in which safety controls failed, or experimental systems behaved unexpectedly.

Robinson, who said he spent 3-1/2 years at OpenAI, helped draft the company’s preparedness framework and oversaw safety reports for 12 frontier-model launches, wrote: “As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.”

“We’re making sure our models don’t become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down,” an OpenAI spokesperson said in a statement.

Robinson also warned that AI capabilities were advancing faster than researchers’ understanding of alignment, a field focused on ensuring AI systems act in accordance with human goals and values.

(Reuters)

RELATED ARTICLES

46 mins ago | Benchmark equity indices

Stock markets fall in early trade after two-day rally

Benchmark equity indices Sensex and Nifty declined in early trade on Wednesday amid profit-taking after a two-day rally. The 30-share BSE Sensex declined 75.10 points to 82,482.30 in early trade. The 50-share NSE Nifty skidded 26.95 points to 25,355...

56 mins ago | APEC summit

China’s Xi calls for free and open trade at APEC summit

Chinese President Xi Jinping on Wednesday urged support for "free and open trade," while expressing opposition to "protectionism" in an opening speech at an Asia-Pacific summit where he was due to meet his US and Russian counterparts. Xi's remarks c...

2 hours ago | Nvidia

Nvidia says ‘no’ to rehiring Jensen Huang

Nvidia's leadership signaled they would brook no dissent in their plan to find a new CEO, rejecting a "Stand With Jensen" campaign from allies of departed CEO Jensen Huang, even as the Wall Street Journal reported a regulator was probing Nvidia. A g...