Anthropic Weighs In On Whether To Ban Open-Source AI

Anthropic said it does support banning open-source artificial intelligence (AI) models and instead favors limiting Chinese access to AI chips and mandatory safety testing for AI models.

“Anthropic has never advocated for a ban” on open-source models, Anthropic CEO Dario Amodei wrote on Monday in a blog post. He said that these models serve as a “public good” as long as they do not have dangerous capabilities. The Anthropic chief executive released his stance on open-source AI as Nvidia launched the Open Secure AI Alliance, which includes many leading tech companies, to protect open-source AI.

Chinese AI company Moonshot AI released its latest flagship model, Kimi K3. The new model saw so much demand that it had to limit new subscriptions.

Anthropic referred the Daily Caller News Foundation to Amodei’s blog post announcing its position on open-source AI.

The Anthropic CEO said open-source models have a higher risk of misuse to carry out cyber or biological attacks because they do not have guardrails set by closed-source platforms such as Anthropic and OpenAI. However, he added that banning open-source models would not address this risk.

Banning open-source models would “protect US AI companies from competition, but that has never been my goal,” Amodei wrote.

Instead, the Anthropic founder said America should limit Chinese access to powerful AI chips and AI chipmaking equipment, crack down on alleged Chinese use of American AI to boost Chinese AI models through a process called distillation and require mandatory safety testing for both open-source and closed-source models

Nvidia believes “blanket restrictions” on open-source models would “risk concentrating power, dependence, and vulnerability” in a few closed AI platforms, the company stated in its press release.

“Companies and governments should invest in shared open infrastructure for AI defense — datasets, evaluation frameworks, attack simulators and red-teaming tools — much as past generations invested in open source software,” Nvidia stated, adding that protecting open-source AI could aid “competition.”

The Trump White House is weighing whether to limit access to Chinese open-source AI models, Axios wrote. The Treasury Department could sanction Chinese AI companies for alleged intellectual property theft. The administration now favors limiting access to these models after it previously declined to regulate AI, Axios reported.

Moonshot, a Chinese AI lab, developed “large scale distillation against U.S. models, allowing them to quickly switch between multiple methods of access to avoid detection,” Michael Krastios, the White House director of the Office of Science and Technology Policy (OSTP), wrote on Wednesday.

Policymakers and regulators should recognize open-source AI models as “defensive assets, not liabilities,” the Open Secure AI Alliance wrote. Corporations and governments should invest in shared infrastructure for “AI defense,” the alliance wrote.