Nicholas Carlini, a researcher at Anthropic, recently warned about the dangers of a specific AI risk labeled Mythos. He is now part of a team from the company briefing the White House on safety measures for the latest AI models.
The shift from whistleblower to advisor underscores the growing tension within the AI industry, where the same experts who highlight existential risks are often called upon to guide regulatory policy. Carlini’s role places him at the center of a debate over how quickly powerful systems should be deployed.
Anthropic has positioned itself as a leader in AI safety research, including internal efforts to identify potential hazards before models reach the public. Carlini’s earlier warnings about Mythos were part of that internal process, though the specific nature of the threat remains undisclosed outside the company.
The White House briefings come amid broader federal efforts to understand and regulate frontier AI systems. Carlini and his colleagues are expected to present evidence on both the capabilities and the vulnerabilities of upcoming models, influencing potential executive action.
Critics argue that relying on company insiders to set safety standards risks conflicts of interest, as firms may prioritize commercial release over caution. Anthropic has maintained that its research arm operates independently from product teams.