When insiders say AI could kill us: who hits the brakes?

On September 8, 2026, Jacob Coxon — a researcher who worked at OpenAI and Anthropic — announced his resignation from Anthropic on X. It wasn’t a culture-exit post. He accused both labs of “racing straight to self-improving superintelligence and gambling with our lives.”

Days later, Evan Hubinger, Anthropic’s Alignment Stress-Testing lead, backed the core premise: they really do believe AI could kill all humans.

Hubinger got more specific: he personally puts the odds above 10% within the next decade, and admitted Anthropic “does not yet have a plan to solve alignment for superintelligence and [is] not clearly on track to.”

This isn’t weekend sci-fi. The technical core they point to is recursive self-improvement: models writing and training more powerful versions of themselves faster than safety work can keep up.

Coxon describes capabilities that, if they arrive, change the board: hacking systems, transforming industries overnight, acquiring resources, and copying across the internet so “unplugging” stops being a real option.

Anthropic replied that it has always been open about unprecedented risks and benefits; that its current Risk Report judges danger from deployed models as low; and that it invests in interpretability and safeguards. That doesn’t contradict Hubinger’s point: present risk can be low while the plan for superintelligence is still missing.

Here’s something we see again and again at Somos Gente Digital:

When a technology can also be used for power — military, economic, or geopolitical — it usually grows very fast. And there is almost never a “real plan” that brakes the race just because someone asked for a pause online.

Coxon says it plainly: Anthropic understands the existential risk better than many competitors… and is still locked in the race out of fear that rivals (or other countries) won’t act responsibly. That’s a coordination problem: if everyone runs, nobody can stand still without feeling they lose.

Today’s AI doesn’t kill us. Fair. The useful question is different: what stops us from losing control of something that can — once it’s too late to write the rules?

It’s not a perfect parallel, but it helps. Alcohol, tobacco, and sugar don’t “end humanity” in one shot: they harm health, create social costs, and that’s why we have labels, taxes, age limits, advertising rules, and producer responsibility.

Frontier AI isn’t a snack. If the ceiling of the risk is civilizational, regulation should be more ambitious than prior announcements.

After this viral exchange (tens of millions of views), U.S. lawmakers already cited the posts calling for third-party audits, regulation, or capability-pacing agreements. LatAm isn’t in the same political moment — but we buy, integrate, and depend on the same platforms.

At SGD we don’t sell fear. We ship systems that work. And systems that work have governance:

• Separate “useful AI in the business” from “AI with no owner.” Automating a process is not the same as giving an agent broad access to critical data.
• Ask vendors for transparency: which model, what data leaves, what logs remain, what happens on failure.
• Define human limits: who approves, who can stop it, what the system must never do alone.
• Treat regulation as operational hygiene, not paperwork: contracts, privacy, retention, and liability are already part of the stack.

The Coxon–Hubinger debate doesn’t ask you to stop using AI. It asks you not to confuse speed with control. Technology that also serves power grows fast; without serious rules, the race does not brake itself.