Anthropic drops core safety pledge after centering AI safeguards - Morning Brew
Anthropic has abandoned its previous safety pledge to halt AI training if models become potentially dangerous, opting instead to continue development unless it gains a significant lead over competitors. The policy change is reportedly linked to pressure from Defense Secretary Pete Hegseth, who threatened to withdraw a $200 million Pentagon contract and blacklist the company from government work unless it relaxes its military AI rules. The shift also followed the resignation of a top safety leader and reflects concerns about competitive pressures impacting AI safety standards.
AI company that touted safety drops its core safety pledge
The policy change comes as Defense Secretary Pete Hegseth reportedly threatened to pull a $200 million Pentagon contract with Anthropic and blacklist the company from government contracts if it didn’t relax its rules for military usage.
• less than 3 min read
A plot hasn’t veered this hard from its premise since whatever the heck was happening on Lost. Anthropic, a company founded by former OpenAI employees who wanted more AI safeguards, said this week that it’s dumping a flagship promise to stop training AI models if they become potentially dangerous.
This is a “dramatic shift” from the company’s first comprehensive safety policy, per the Wall Street Journal. Now, instead of pausing development when a model’s capabilities outpace safeguards, Anthropic said it’ll keep chugging, unless it thinks it has a “significant lead” over competitors. Why?
- The company implied that its prioritization of human well-being makes it hard to rival other major AI companies, none of which hold themselves to Anthropic’s previous safety standards.
- To that point, Anthropic argued that
everybody’s doin’ itrestraining itself while less careful competitors run free could ultimately make the world less safe.
Impeccable timing: The policy change comes as Defense Secretary Pete Hegseth reportedly threatened to pull a $200 million Pentagon contract with Anthropic and blacklist the company from government contracts if it doesn’t relax its rules for military usage by tomorrow. An Anthropic spokesperson said these negotiations were unrelated to its safeguards switchup, but the policy overhaul did contribute to a top safety leader’s decision to resign from Anthropic earlier this month, sources told the WSJ.—ML
Become smarter in just 5 minutes
Morning Brew delivers quick and insightful updates about the business world every day of the week from Wall St. to Silicon Valley.
Become smarter in just 5 minutes
Morning Brew delivers quick and insightful updates about the business world every day of the week from Wall St. to Silicon Valley.
Comments (0)
No comments yet. Be the first to share your thoughts.
Sign in to leave a comment.