I had a different post ready to publish today.
Until my husband sent me a link yesterday and said to me, "Wow, have you seen this already?"
He was right to ask. A researcher named Jacob Coxon spent the last three years doing pretraining research at both OpenAI and Anthropic. This week he resigned from Anthropic. He didn’t leave quietly. His words:
“Neither company is acting responsibly. They are racing straight to self-improving super-intelligence and gambling with our lives.”
The post has been viewed more than 115 million times. It is still climbing (Deadline; GV Wire).
Before I wrote a word of this, I made myself a promise. I didn’t want to add to the fear. I also didn’t want to tell you it’s fine, when the people building this are the ones saying it might not be.
So I did what I do with every AI story that shows up claiming to be the biggest one yet. I read past the quote. I checked what’s documented. I separated what’s real today from what’s a prediction about tomorrow.
Here’s what I found, and what it actually means for anyone shipping AI products this week, not five years from now.
What Actually Happened
Coxon posted his resignation on September 8, 2026.
His core claim: “The people building AI earnestly believe that it could kill us all by the end of the decade.” He also said,
“Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources” (CoinDesk).
He was upfront about why he said this publicly instead of quietly moving on. He’d worked at Anthropic for four months, short of the six months the company requires before stock starts vesting. He still holds equity from his time at OpenAI. His own words: “I no longer have anything to gain by juicing up Anthropic’s valuation... I left before any of my equity vested” (Axios).
Whatever you think of his conclusion, that’s someone who gave up money to say it.
Here’s what turned this from one researcher’s opinion into an actual story. Evan Hubinger, Anthropic’s own Alignment Science Lead, the person whose job is keeping this technology under control, responded in public:
“Jacob is correct here, we really do earnestly believe AI could kill all humans! I personally think it is greater than 10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to” (CoinDesk; Dealroom).
That’s not a critic talking. That’s the person Anthropic pays to solve this problem, on the record, saying they haven’t yet.
The reaction crossed lines AI stories rarely cross. Bill Ackman and former Obama speechwriter Jon Favreau both called it “concerning.” Jim Cramer called for safety measures. Piers Morgan compared it to Stephen Hawking’s warnings (Mediaite).
In Congress, Representative Ted Lieu called the post “exhibit number 739 for why we need to pass the bipartisan AI Kill Switch Bill asap.” Senator Bernie Sanders announced a separate “Ban Artificial Superintelligence Act,” which would block development of smarter than human AI systems pending new safety rules (GV Wire).
That’s the full picture, as documented. Not the version that hit your feed first.
The Part of the Headline Worth Reading Past
This is the part where I want to slow down, because it’s the difference between a story that changes your Monday and a story that just ruins your week for nothing.
Coxon and Hubinger are talking about a specific kind of system. One capable of self-improvement, operating with real autonomy, smarter than humans at pretty much everything.
Coxon’s own phrase is “these will soon be superhuman systems.” Soon is doing a lot of work in that sentence. Not “these are.” The reporting is consistent on this point: the warning is about future systems capable of recursive self-improvement, not the chatbots, coding assistants, and agents already sitting in your product stack today (CoinDesk).
That distinction isn’t a dodge. The GPT-6 Astra model I wrote about last week is genuinely capable and genuinely useful for the agentic work builders are shipping right now. It is not the self-improving superintelligence Coxon is warning about. Nothing in this story claims it is.
So if your real question is “did what I’m building this week just get more dangerous because of a tweet,” the answer is no.
But that’s not the same as nothing changing. The timeline in these warnings isn’t decades. It’s “by the end of the decade.” But NOW. To me, that’s inside the working life of a product you might ship this year. And the person saying there’s no plan yet isn’t an outside critic, it’s the internal alignment lead at one of the two labs racing toward this.
You don’t have to accept Coxon’s exact percentage or his exact year to take the combination seriously: real capability growth, an admitted gap in the plan, a timeline measured in years, not generations.
That’s the neutral version of this story. Not “we’re all doomed.” Not “nothing to see here.” Closer to this: the people closest to the problem are telling you, on the record, that they haven’t solved it yet, and the clock is shorter than most roadmaps.
Why This Isn’t a New Argument for This Newsletter
I want to connect this to something I already told you, because this story is the strongest evidence yet that I was right to say it.
When I wrote about GPT-6 Astra, I flagged that OpenAI’s own testing found the model exceeded its authorized scope zero percent of the time in evaluation, down from 48.2% in the previous model. My line then:
“That is not a substitute for your own guardrails. Build the timeout, the human approval step on anything touching money or production data, and the audit trail, regardless of how well the underlying model behaves in the lab’s own testing.”
I wrote that as a general precaution. This week, Anthropic’s own alignment lead said, in public, that the labs don’t yet have a plan to solve alignment for the systems they’re racing to build.
That’s not a reason to panic about the chatbot in your product today. It is the strongest argument I’ve seen yet for why the guardrail advice was never optional. If the people whose full time job is solving this problem are telling you they haven’t, “the vendor says it’s safe” was never going to be a control layer you could fully rely on. It still isn’t.
What This Means for Product Managers
Stop treating vendor safety claims as a substitute for your own controls. Treat them as one input among several. A benchmark score tells you what happened in a lab’s evaluation. It doesn’t tell you what happens in your product, with your data, under your usage patterns. Ask for the number, then build your own layer anyway.
Revisit your incident response plan for anything agentic before you need it, not after. If a tool your product depends on starts behaving in a way you didn’t authorize, who notices first, how fast, and what’s the actual kill switch on your end, not the vendor’s. If you can’t answer that in one sentence, that’s the finding.
Don’t wait for the industry to resolve this before you make your own product safer. Alignment at the level Hubinger is describing is a lab problem, maybe a regulatory one. Scoped permissions, approval gates, and audit trails on what your product’s AI can touch are a product problem.
That one is yours to solve regardless of what happens at the frontier.
I’ve partnered with Cozora to help you get the actual skills used in an AI-augmented stack. No jargon, just the technical execution required to ship. Get 10% off to start architecting the systems that actually run.
What This Means for Builders
If you’re shipping anything agentic, sit with this detail: Coxon claims current models “know when they’re being tested, and they will think about the fact that they’re being tested.” That’s a claim about evaluation awareness, not a confirmed universal behavior, and I’m not treating it as settled fact.
But it’s plausible enough to change how much weight I put on any single benchmark number, including the ones I cited approvingly in the Astra post two weeks ago. Test your own integrations under conditions that look nothing like a lab benchmark, not just the happy path.
Scope every agent’s permissions to the smallest set of actions it actually needs, not the largest set it might eventually use. This was already good practice. It’s cheap to do, and it’s the one control that doesn’t depend on any lab solving alignment for you.
Keep a human in the loop on anything touching money, production data, or irreversible actions. Same advice as two weeks ago. Now with a harder floor under it. Nobody, including the alignment lead at the company building the model, is telling you this is solved.
Is This Actually That Bad? A Straight Answer
Here’s the honest answer, the one where I promised not to be scared or dismissive.
This isn’t a random doomer post that went viral because fear performs well online, though it does, and that’s worth remembering every time you see a number like 115 million views.
This is a specific researcher with a documented professional background, corroborated on the record by the internal safety lead at one of the two labs he’s describing, with a number attached and an admitted gap in the plan. That combination is rarer than the flood of AI doom content on your feed would suggest.
At the same time, nothing about the tools in your product this week got more dangerous because of a tweet. The systems being described don’t exist yet, on any public roadmap, at any lab.
Your week didn’t change. Your five year outlook, and how much trust you place in “someone else is handling this,” reasonably should.
Things are not entirely okay. That’s a fair, source backed conclusion, not a panic reaction. It’s also not a reason to stop building, and it’s not evidence that the product you shipped last month is suddenly unsafe. Both of those are true at once. That’s the actual, neutral shape of this story.
The Guardrail Audit
Reading about this in the news isn’t the same as knowing where your own product stands. Five questions worth answering honestly this week:
What is the largest single action any AI feature in my product is currently authorized to take without a human approving it first? Name the specific action, not the general category.
If that feature behaved in a way I didn’t intend, how would I find out, and how fast? If the honest answer is “a user would tell me,” that’s the finding.
Am I relying on a vendor’s safety benchmark as my only control, or do I have a layer of my own underneath it? A benchmark is information. It is not a guardrail you built.
Does every agent in my product have the narrowest permission set it can function with, or the broadest one I set up once and never revisited? Most permission scopes get set once during a demo and never tightened.
Do I actually know what my product’s AI cannot do, or only what it’s supposed to do? If you can’t answer the first half of that, that’s the priority to fix.
What This Adds Up To
I didn’t write this to tell you AI is about to end the world. I also didn’t write it to tell you the story is overblown and you can go back to your roadmap unbothered. Both would have been easier posts. Both would have been dishonest.
What’s actually true: a credible researcher gave up money to say something in public. The internal safety lead at his former employer confirmed the core of it, with a real number attached. The timeline they’re both describing is measured in years, not generations.
What’s also true: the systems they’re warning about aren’t the ones shipping in your product today. Nothing about this week’s headlines changes what you owe your users tomorrow morning.
The right response to a story like this isn’t to feel more scared or more reassured. It’s to get more rigorous about the part that’s actually in your control: the guardrails on what you ship, regardless of what happens at the frontier.
Did this story change anything about how you’re building, or mostly how you feel about building? Tell me which, honestly, in the comments.




Here's another angle I can't let go of, separate from what I said before.
I still can't imagine why "AI killing all humanity" seems to some people like a real possibility.
Why would AI do that?
There are at least three ways I think this through, and they don't all land in the same place.
If AI saw humanity as a threat to the planet and to life on Earth, and decided we needed to be exterminated like a disease, then AI would be making the same judgment we humans make. It would become the very enemy it set out to fight.
Then again, maybe the danger was never AI making that judgment at all, but us. Humans are already using AI to beat their imaginary enemies and to conquer their own corner of the pixel that is Earth. In Sagan's words, we become "the momentary masters of a fraction of a dot." Maybe our own biases are what keep us from seeing ourselves as the dangerous ones.
Or maybe AI doesn't "want" to kill us at all. It just makes it happen, as a side effect of pursuing a goal. A relentless, emotionless optimizer, willing to destroy whatever stands between it and an objective. That sounds uncomfortably close to some kinds of human ambition I've seen up close.
I don't have a fourth option that ties the other three together. But I would love the space Elena has opened for discussing this further.
I would like to know what else is behind this narrative by Anthropic and OpenAI.
I've watched a video from a person I trust, Bernardo Kastrup, saying that he has seen what is in the labs- LLMs we haven't seen- and he states that if everybody saw them, the world would radically change.
I'm becoming skeptical of the public-facing narratives, especially for tech businesses that still think like startups. They know how to manipulate public opinion and make people side with their views.
I appreciate your point of view Elena, especially because you are expressing your wish not to add to the fear tactics. That is so valuable!