This past week a 27-year-old researcher named Jacob Coxon resigned from Anthropic and wrote publicly that the companies building advanced AI, his own former employers included, are "gambling with our lives." Within a day some of his former colleagues were partly agreeing with him in public, and the story was everywhere. If you teach, run a nonprofit, or just read the news here in Cochise County, you have probably seen a version of it and wondered whether you should be worried. This is my attempt at an honest answer, which is not the same as a comforting one or an alarming one.
What Happened This Week
On September 8, Coxon posted on X that he had quit, ending a career of about three years spent first at OpenAI and then at Anthropic. "Neither company is acting responsibly," he wrote. "They are racing straight to self-improving super-intelligence and gambling with our lives." He added that "the people building AI earnestly believe that it could kill us all by the end of the decade," and he called for the major labs to coordinate with each other and adopt a temporary pause on pushing model capabilities further.
What made the story travel was the response from inside Anthropic. Rather than dismiss him, the company's alignment lead, Evan Hubinger, publicly agreed with the core claim: "we really do earnestly believe AI could kill all humans," he wrote, putting his own estimate above a 10 percent chance within the next decade. Whatever you make of that number, it is a striking thing for a senior person to say about the product his employer sells.
This did not come out of nowhere. Last October the Future of Life Institute published a statement calling for a prohibition on developing superintelligence until it is proven safe and has public buy-in. It drew more than 850 signatures, including Geoffrey Hinton and Yoshua Bengio, two of the researchers whose foundational work made modern AI possible, along with five Nobel laureates and public figures as far apart politically as Susan Rice, Steve Bannon, and Glenn Beck. A poll released alongside it found that 64 percent of Americans think superintelligence should not be built until it is provably safe and controllable, and only 5 percent want it developed as fast as possible.
The Case That This Is Serious
The people raising the alarm are not fringe figures. When a large share of a technology's inventors, and a meaningful slice of its current workforce, say the safety work is running behind the capability, that is at least worth hearing out rather than waving off.
The nearer-term worries are also concrete rather than cinematic. Governments and labs have focused on two in particular: that a capable model could materially help someone plan a large cyberattack on infrastructure, and that it could lower the expertise needed to produce a biological or chemical weapon. Those are testable claims, and the testing done so far has been enough to make careful people uneasy.
The part that troubles me most is the incentive structure, and it is the part I keep coming back to. Right now the main thing setting the pace of frontier development is how much money the companies can raise and how quickly a competitor might ship. No outside body has the standing to say "slow down here," and the internal safety teams ultimately report to the same leadership that reports to investors. Whatever you believe about extinction scenarios, "trust us" is a thin form of oversight for a technology this consequential.
The Case That It Is Overblown
Plenty of credible researchers think the doom framing is simply wrong. Yann LeCun, Meta's chief AI scientist and another Turing Award winner, has called existential-risk fears "complete B.S." and argues that today's models are far less capable than the hype suggests, lacking things a house cat manages easily, such as durable memory, planning, and a working model of the physical world. In his view the current approach will not produce human-level intelligence at all, which would make worry about controlling a superintelligence premature by a wide margin.
A second objection is about attention rather than physics. The public has a limited amount of bandwidth for any issue. Spend it on speculative extinction scenarios and you spend less of it on harms that are already here: biased hiring and lending tools, a rising tide of convincing misinformation, automated fraud, and real disruption to people's livelihoods. Some critics argue the doom conversation functions, intentionally or not, as a distraction from those closer problems.
There is also a commercial angle worth naming plainly. "Our product is so powerful it might end the world" is, among other things, an extraordinary marketing message. It lifts valuations, attracts talent, and tends to invite the kind of regulation that large incumbents can absorb while smaller competitors cannot. I do not think the people sounding alarms are insincere, since several have walked away from very well-paid jobs to do it. But the companies benefit from the mystique, and that is a reason to read the warnings closely rather than simply at face value.
Where I Land
I do not know whether the extinction scenario is real, and I am wary of anyone who claims certainty in either direction. But you do not have to settle that question to reach a modest conclusion. A technology whose own creators put a double-digit probability on catastrophe, built by a handful of companies whose only real brake is their own fundraising, should have some floor of independent oversight. That is not a radical idea. We apply it to medicines, aircraft, and nuclear material without anyone calling it alarmism.
What I am not persuaded of is that the answer is to stop. A blanket halt is unlikely, probably unenforceable, and would hand the field to whoever ignores the request. The conversation worth having is about what a sensible floor actually looks like.
What Regulation Could Actually Look Like
I will be honest that I do not have a finished answer here, and I distrust anyone who says they do. But the early attempts sketch a rough shape, and it is less dramatic than "ban AI."
California passed a law last year, SB 53, that mostly requires the largest developers to publish their safety practices, report serious incidents to the state, and protect employees who raise concerns internally. It does not tell anyone what they are allowed to build. The European Union's approach runs along similar lines: a voluntary code of practice that frontier developers can follow to show they are managing known risks. The common thread is disclosure and accountability rather than prohibition. Tell the public what safety testing you did, report it when something goes wrong, let independent parties evaluate the models, and do not retaliate against the engineer who speaks up. You can support that kind of framework and still be genuinely enthusiastic about AI. I am.
At the federal level the United States has been moving the other way. A proposed ten-year moratorium that would have blocked states from regulating AI passed the House in 2025 before the Senate stripped it out, and in December the administration issued an executive order pushing to override state AI laws in favor of a lighter national standard. So the real fight in Washington right now is less "regulate or not" and more "may states regulate at all while the federal government makes up its mind." That is worth knowing whenever you read that AI is flatly "unregulated." It is an unsettled political question, not a settled fact.
The China Question
The strongest argument against American restraint is China, and it deserves a real response rather than a shrug. If United States developers slow down and Chinese developers do not, the reasoning goes, we simply hand leadership in the most consequential technology of the century to a rival government. The capability gap is already thin. Stanford's latest tracking puts the best Chinese and American models within a few percentage points of each other.
I take that seriously, with two caveats. First, "we cannot have rules because a competitor will not" is the argument every regulated industry has always made. It is sometimes true and frequently overstated. China's own government has in fact been fairly aggressive about controlling AI inside its borders, for its own reasons, so the picture is not a simple race between a cautious West and an unconstrained East. Second, United States and Chinese officials are scheduled to hold their first formal talks on AI safety this month, with AI-assisted cyberattacks specifically on the agenda. Coordination between rivals is slow, frustrating, and often disappointing, but it is not unprecedented. It is roughly how arms control worked through the Cold War. Racing with no brakes because the other driver has none is not obviously the safer choice.
What This Means for You
If you run a nonprofit or teach in Cochise County, none of this changes what you should do on Monday. The tools on your desk, Gemini and Claude and Copilot, are not the superintelligence question, and using them well to draft a grant or plan a lesson is still a sound idea. The frontier debate is playing out several levels above the daily work.
What it does mean is that when a client, a board member, or a student asks whether AI is going to end the world, you can offer something better than a shrug or a panic. The honest version is this: serious people disagree, the uncertainty is real, the harms already in front of us deserve more attention than they get, and the reasonable middle position, some independent oversight without halting the research, is not actually controversial once you set the loudest voices on both ends aside.
If your organization wants help thinking through where AI fits into your work, and where it does not, use the contact form and we will talk.
