Who is Dario Amodei, the Anthropic CEO pushing independent AI oversight?

Many of its leading figures have ties to a movement called "effective altruism (EA)," the belief that evidence and careful reasoning can help maximize the good that companies and people can do with their time and resources.

METR describes itself as an AI safety testing laboratory.

"METR evaluates frontier AI models to help companies and wider society understand AI capabilities and what risks they pose," its website reads.

Although its website and public materials don't make mention of effective altruism, its founders have used it as a framing for their work.

Beth Barnes, METR's founder and CEO, worked at OpenAI alongside Amodei as the company developed early versions of ChatGPT.

At an Effective Altruism Global event, she laid out her vision for how to maintain AI safety.

"Our overall plan is - it sort of seems like it would be good if it was someone's job to look at models and decide if they're going to kill us, think through the ways that that might happen, anticipate them, figure out what the early warnings would be, that sort of thing," Barnes said.

Similarly, Paul Christiano, who led the research around OpenAI's efforts to ensure its models followed acceptable strategies for delivering requested results, later founded the first iteration of METR. He too has framed parts of his approach to AI safety as a form of effective altruism.

Figures like Barnes and Christiano provide informal links to Anthropic, a company that received funding from effective altruism's largest supporters when it emerged as a way of thinking among tech moguls. Barnes and Christiano also both worked on evaluations involving Anthropic models, providing safety checks for Anthropic's flagship AI, Claude.

Amodei isn't calling for the rest of the industry's leaders to submit to the thinking of METR, specifically. But in a recent letter, he used them as an example of the guidance he believes the industry needs.

Amodei proposed a plan: Accountability could come through "embedded evaluators" that would supervise AI development companies.

"Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR) whose role it is to verify adherence to safety practices and commitments, report incidents and help assess the alignment of not just completed AI models but training pipelines and processes," Amodei wrote.

"Regardless of what commitments we make, the public deserves to know what is going on. We are still the ones choosing what to include and omit. Embedded evaluators will change this dynamic."

Amodei originally studied biophysics, earning a Ph.D. from Princeton in 2011. He would go on to become a postdoctoral scholar at the Stanford University School of Medicine.

At OpenAI, Amodei continued those themes, eventually becoming vice president of research as the company developed its ChatGPT 2 and ChatGPT 3 models.

In its early stages, the GPTs were asked to fill in blanks to sentences like, "Today, I went to the ______ and bought some milk and eggs," and "I knew it was going to rain, but I forgot to take my ______."

But just as the company began to discover that it could amplify the power of its models through larger and larger language models, Amodei left OpenAI in 2020. He believed the company wasn't doing enough to install guardrails on what he saw as a budding reality of the technology he had long theorized about.

He also didn't know if he could trust the company to set aside its financial interests.

"When you feel that you can't trust someone, when you feel that their values are not what they say they are, when you feel that they're not honest, when you feel that they're not in it for the reasons that they say, when you see disturbing patterns of behavior, dishonesty, that makes it very hard to continue to work with a company, to continue to trust the company," Amodei said in an interview with Bloomberg earlier this year.

Since leaving OpenAI, Amodei helped start Anthropic, a company that has made AI safety a key part of its makeup. Anthropic even has a sort of constitution for its flagship AI; a guiding document laying out boundaries for its research.

Although that tension cost Anthropic a $200 million contract, Amodei believes it's time for the industry to take similar stances, especially as some companies have begun to report trouble controlling their own agents.

Amodei continues to believe that AI capabilities can grow safely, but only if the industry sets standards for how it achieves "safe."

"I continue to believe that AI can enormously improve the quality of human life. My desire to achieve these benefits is undimmed. But the benefits will only be achieved if we build the technology in the right way, and - so long as we use the time we gain well - it is worth taking unusually deliberate care to get it right," Amodei wrote in his letter.