OpenAI Adds a Man Who Fears AI to Its Safety Board

iEXExchanger
OpenAI Adds a Man Who Fears AI to Its Safety Board

Paul Christiano, the former head of alignment research at OpenAI and a co-creator of RLHF, has joined the company's nonprofit board and its Safety and Security Committee amid growing fears about AI risk.

One AI researcher quit this week, calling the race for superintelligence a reckless bet with human lives. Another accepted a seat at the table of the very industry he warns about. OpenAI has added Paul Christiano, a longtime alignment researcher, to the board of its nonprofit foundation.

Christiano isn't a random pick. From 2017 to 2021 he led alignment research at OpenAI and helped build RLHF — reinforcement learning from human feedback, the technique still shaping how ChatGPT and most modern chatbots behave. After leaving, he founded the independent Alignment Research Center. Today he's a senior technical adviser at CAISI, the Commerce Department unit that evaluates AI models and drafts safety guidance for the US government.

The new role gives him three hats at once: a seat on the nonprofit foundation's board, which retained 26% of the restructured for-profit OpenAI Group PBC last year; membership on the Safety and Security Committee chaired by Zico Kolter; and a non-voting observer spot on the for-profit board itself. He'll recuse himself from anything touching his Commerce Department work or model evaluations.

Christiano doesn't soften his warnings. He argues that without slower development, the industry risks a 'catastrophic and irreversible loss of control,' and that systems built to design their own successors could eventually outrun humans' ability to steer them. Back in 2023, he put the odds of an AI takeover killing a large share of humanity at 10 to 20 percent.

The appointment doesn't settle the industry's core tension between speed and caution. But one of its loudest warning voices now sits inside the boardroom instead of shouting from outside it.

Questions and answers

Frequently asked questions about this article

Who is Paul Christiano?

An alignment researcher who led that work at OpenAI from 2017 to 2021 and helped develop RLHF. After leaving, he founded the independent Alignment Research Center. He's now a senior technical adviser at CAISI, part of the US Commerce Department.

What is OpenAI's Safety and Security Committee?

A board-level body that oversees the company's decisions on critical model and infrastructure safety issues. It's chaired by Zico Kolter, a professor at Carnegie Mellon University.

What is RLHF and why does it matter?

Reinforcement learning from human feedback is a training method that shapes how language models respond so their answers match what people expect. It underlies the behavior of ChatGPT and most modern chatbots.

Is this connected to the Anthropic researcher's resignation?

Not directly — they're two separate events at two different companies. But both happened around the same time and reflect the same trend: growing tension in the AI industry between the pace of development and safety demands.