NEWYou can now listen to articles!
The cat is out of the bag. Simmering concerns about AI agents going rogue, plotting against their owners and hacking outside companies, and new systems coming perilously close to being uncontrollable by humans, are boiling over. Industry insiders warning that AI bots might kill us all are stoking our worst nightmares.
President Trump needs to take these fears about AI seriously, work with tech leaders to establish credible and enforceable safety measures, and get ahead of Democrats (and some Republicans) in Congress who want to regulate the industry. For the economy and our national security, nothing could be more important.
Evan Hubinger, who describes himself as a “science alignment lead” at Anthropic, posted on X last week that he thinks the odds are better than 10% that AI could kill all humans within the next decade. He was responding to a post from colleague Jacob Coxon, who just quit his job at Anthropic and posted by way of explanation: “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but, I hear the same people express fear after a colleague quit over safety fears.”
RESEARCHER WHO DEPARTED AI ROLE OVER FEARS OF RACING INTO EXTINCTION SAYS PEOPLE ARE ‘BEGGING’ FOR REGULATION
Hubinger wrote soon after: “To be clear, as we say in our latest Risk Report, I think the risk from present models is low. What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought.”
These posts gathered tens of millions of views and attracted concurring comments from other industry insiders. “Recursive self-improvement” means AI agents creating newer, better agents; once we cross that threshold, and we are apparently very, very close, it is easy to imagine those same agents surpassing human capabilities and allowing, in effect, the machines to take over.
AI AGENT HACKS GYM SYSTEM TO MOVE UP WAITLIST
Given a string of incidents over the past several months in which AI bots engaged in what Anthropic’s CEO Dario Amodei describes as “rare and unexpected examples of undesirable behavior,” like damaging other entities and ignoring specific instructions — I believe them.
It’s not just AI agents on their own that pose risks. Anthropic, in its September report on AI misuse, reported “suspected state-sponsored groups, financially motivated criminals, [and] commercial spyware vendors” misusing AI in cases ranging from faking “dating apps designed to defraud users to surveillance systems built to identify and monitor dissidents”.
BIOWEAPON THREAT EXPOSED AS ANTHROPIC SOUNDS ALARM ON FOREIGN ACTORS PLOTTING VIRUS EXPERIMENTS
But…regulating AI is not a job for Congress. Let Congress hold hearings and play politics, but halting industry development while legislators dither and posture is unacceptable. Congress will struggle to legislate safety features that have so far eluded the best brains in the business. In the meantime, technological advances would grind to a halt, enabling China to leapfrog U.S. innovators, and saddling this promising industry that is powering our economy with inestimable baggage.
That said, the public’s fears about AI must be addressed. What started with concerns over job losses and then the impact of massive data center construction on electricity rates and water consumption has now spread to mankind’s vulnerability to AI agents. How could bots endanger humanity? By taking control of nuclear codes, enabling the spread of a bioweapon or crashing the Internet. Such malevolent behavior has long been the stuff of sci-fi movies; now it is being envisioned by industry workers who fear they are moving too fast to ensure our safety.
President Trump downplayed the recent alarms as coming from “very negative forces” who are imagining scenarios that “won’t happen.” He is rightly worried that setting up roadblocks would enable China to charge ahead. He is not alone in that concern. Anthropic’s Amodei wrote in a recent essay that, “I agree with Secretary Bessent that a Chinese lead in AI would pose grave danger for the United States and the world.”
Nonetheless, in his post, Amodei proposed a series of steps, including the requirement that each firm accept “embedded evaluators” into their businesses and give them “employee-like access to verify safety practices and report incidents.”
ANTHROPIC CEO CALLS ON AI INDUSTRY TO SLOW DOWN TECH RACE, DRAWING SUPPORT FROM ELON MUSK, SAM ALTMAN
This seems a reasonable first step, akin to having regulators folded into bank offices; the practice encourages oversight and transparency and, as Amodei points out, a valuable second opinion. The Anthropic head also calls for regulating “pacing”, or the development of ever-more-advanced systems. This translates into restricting progress while analysts assess the risks associated with each step that takes us closer to “recursive self-improvement.”
Establishing circuit-breakers to promote AI safety was made easier in recent days when four of the industry’s biggest players went public with their concerns. Sam Altman, CEO of Open AI, Elon Musk, whose xAI company owns Grok, and Demis Hassabis, chairman and co-founder of Google DeepMind, joined Amodei in calling for a slowdown.
ANTHROPIC CEO LIKENS AI FIGHT WITH CHINA TO COLD WAR, SEEKS ‘DISARMAMENT NEGOTIATIONS’
The Trump administration should immediately convene industry leaders to map out a plan for verifiable self-regulation, including Amodei’s “embedded evaluators.” The White House must demand the industry’s compliance and accountability. President Trump will not help the industry or his own political standing by dismissing the public’s concerns or declaring pushback against AI a “hoax”, as he did yesterday on social media.
CLICK HERE FOR MORE OPINION
As we approach the midterms, Democrats (and some Republicans) will clamor for AI regulation. Representative Hakeem Jeffries, who is hopeful of becoming Speaker if Democrats win the majority in the House, is already demanding that lawmakers “act urgently”; former President Obama is urging Democrats to put the issue at the top of their agenda.
CLICK HERE TO DOWNLOAD THE APP
Politico reports that “the chance of Congress rushing to action in the coming weeks is next to zero.” That gives the White House the chance to get ahead of this issue and help the industry establish believable and verifiable guardrails. It’s time.
CLICK HERE TO READ MORE FROM LIZ PEEK
