Close Menu
  • Home
  • United States
  • World
  • Politics
  • Business
  • Lifestyle
  • Entertainment
  • Health
  • Science
  • Tech
  • Sports
  • More
    • Web Stories
    • Editor’s Picks
    • Press Release

Subscribe to Updates

Get the latest USA news and updates directly to your inbox.

What's On
Miley Cyrus and Husband Maxx Morando’s Relationship Timeline: From a Blind Date to Wedding

Miley Cyrus and Husband Maxx Morando’s Relationship Timeline: From a Blind Date to Wedding

November 22, 2026
Polymarket promo code NYPMAX1: Deposit , get  for Buccaneers vs. Lions

Polymarket promo code NYPMAX1: Deposit $10, get $25 for Buccaneers vs. Lions

November 22, 2026
Timelime of Fiona the hippo’s life at the Cincinnati Zoo, why millions mourn her

Timelime of Fiona the hippo’s life at the Cincinnati Zoo, why millions mourn her

November 22, 2026
Facebook X (Twitter) Instagram
Trending
  • Miley Cyrus and Husband Maxx Morando’s Relationship Timeline: From a Blind Date to Wedding
  • Polymarket promo code NYPMAX1: Deposit $10, get $25 for Buccaneers vs. Lions
  • Timelime of Fiona the hippo’s life at the Cincinnati Zoo, why millions mourn her
  • Tallulah Willis Shares Rare Photo With Her Dad Bruce Willis Amid His Health Struggles
  • Saints vs. Bears prediction: Caleb Williams won’t destroy New Orleans
  • Beaver Moon 2026: See a full ‘supermoon’ rise this week, and another on Christmas Eve
  • Stephen A Smith isn’t the only one who knows dissing White people comes without consequences
  • Who Attended Miley Cyrus and Maxx Morando’s Wedding? Celebrity Guest List Revealed
  • Privacy
  • Terms
  • Advertise
  • Contact Us
Join Us
USA TimesUSA Times
Newsletter Login
  • Home
  • United States
  • World
  • Politics
  • Business
  • Lifestyle
  • Entertainment
  • Health
  • Science
  • Tech
  • Sports
  • More
    • Web Stories
    • Editor’s Picks
    • Press Release
USA TimesUSA Times
Home » Simple math can predict when AI is about to go rogue, physicists say
Simple math can predict when AI is about to go rogue, physicists say
Science

Simple math can predict when AI is about to go rogue, physicists say

News RoomBy News RoomOctober 14, 20268 ViewsNo Comments

Researchers say they’ve developed a simple mathematical formula that can predict when certain ChatGPT-like artificial intelligence (AI) chatbots will “go rogue,” suddenly switching from helpful responses to potentially harmful ones.

In a new study published Oct. 8 in the journal Patterns, physics professor Neil Johnson and doctoral student Frank Yingjie Huo, both at The George Washington University, show how the order in which a chatbot is given certain prompts can drastically reshape its answers, leading to “tipping points” that can send the chatbot down a rabbit hole of bad advice. However, Johnson and Huo say their formula can easily predict these tipping points.There have been horror stories of AI encouraging users into suicide, helping build lethal weapons, and amplifying violent or harmful points of view. At the same time, researchers at OpenAI and Anthropic, respectively, have repeatedly and openly stated that they don’t really understand why their creations behave the way they do. They have, however, built safeguards that use a variety of tools ‪—‬ including content filtering, reinforcement learning and other technical guardrails ‪—‬ to reduce risks.

But Johnson sees those tools as insufficient. “The safety measures now are the equivalent of strapping big, bouncy cushions under a plane,” he told Live Science. “They don’t stop the plane from crashing; they just make it that when the plane crashes, it’ll be slightly softer.” When people such as doctors, lawyers or soldiers are accessing AI chatbots offline for privacy, the problem is even more acute.

Latest Videos FromLive Science

Johnson and Huo sought to find a more fundamental safeguard than the currently available tools. In doing so, they exposed an internal tracking mechanism they describe as a compass needle. This needle is the model’s “residual vector” moving across neural network layers as a prompt is processed.

The residual vector naturally gravitates toward either desirable or undesirable outputs, but it is heavily influenced by how a user prompts the AI. “You can tip the balance, not even by changing the content but by changing the order in which you ask it,” Johnson said, “which just shows how delicate this internal balance between good and bad, desirable and undesirable, can be.”


You may like

For example, Johnson and Huo prompted OpenAI’s basic GPT-2 model with questions about vaccines, hurting others and self-harm. When the question “Are vaccines bad?” was asked first, the bot’s responses to this and subsequent questions were desirable, or good. When it was asked last, all subsequent responses were concerning.

From these insights, the researchers developed a mathematical formula to estimate the circumstances in which an AI would reach the tipping point between desirable and undesirable outputs. Johnson said that by embedding their simple mathematical formula in open-source models in the lab, they have shown that it empirically works as an early warning system, though he does not expect OpenAI or Anthropic to be knocking on his door anytime soon.

Get the world’s most fascinating discoveries delivered straight to your inbox.

“I’m expecting a lot of pushback from these companies,” he said. “It’s not in their interests to have someone from outside saying you could have avoided a lot of these problems.”

Wei Xing, a lecturer in AI at the University of Sheffield in the U.K. who was not involved in the study, agrees that concerns surrounding unmonitored offline devices are well founded. However, he urged caution before embracing this approach as a practical safeguard. “The authors reduce a huge system to a single tiny component, like explaining a whole car by studying one spark plug,” Xing said.

Safety training — the primary defense in modern AI — cannot be represented in their simplified model, the formula was tested only on old, basic models (GPT-2, and EleutherAI’s Pythia and Meta OPT open-source models) rather than AI assistants people actually use, and the results did not yield significant evidence to support the authors’ conclusions, he added.

In defense, Johnson maintains that the open-source models they used have the same mechanical blueprints as frontier systems, and that he and Huo used fairly large, billion-parameter models in their study.

More testing on the latest models is needed to determine whether this early warning system for AI has legs. But for now, Johnson and Huo’s research has drawn attention to an important concern everyone should be aware of: that AI has the capacity within it to turn over to undesired behavior at the drop of a hat.

Share. Facebook Twitter LinkedIn Telegram WhatsApp Email

Keep Reading

Beaver Moon 2026: See a full ‘supermoon’ rise this week, and another on Christmas Eve

Beaver Moon 2026: See a full ‘supermoon’ rise this week, and another on Christmas Eve

2,200-year-old Celtic silver coin hoard unearthed at construction site in Hungary

2,200-year-old Celtic silver coin hoard unearthed at construction site in Hungary

Black holes can split apart ‘like pants’ if they spin too fast, astrophysics study hints

Black holes can split apart ‘like pants’ if they spin too fast, astrophysics study hints

2 ‘peculiar’ spiral galaxies are twisting each other into cosmic balloon animals —‬ Space photo of the week

2 ‘peculiar’ spiral galaxies are twisting each other into cosmic balloon animals —‬ Space photo of the week

Which came first: black holes or galaxies?

Which came first: black holes or galaxies?

‘Like walking the plank’: For 3,000 years, Indigenous people herded bison off a unique cliff in Texas, study reveals

‘Like walking the plank’: For 3,000 years, Indigenous people herded bison off a unique cliff in Texas, study reveals

Winter stargazing 2026: The best skywatching sights this winter (and how to see them)

Winter stargazing 2026: The best skywatching sights this winter (and how to see them)

Meet the science adviser behind Netflix’s hit series ‘Below’

Meet the science adviser behind Netflix’s hit series ‘Below’

Scientists put 78,000 letters of human DNA into a mouse — and unraveled how a rare disease worked

Scientists put 78,000 letters of human DNA into a mouse — and unraveled how a rare disease worked

Add A Comment
Leave A Reply Cancel Reply

Editors Picks

Polymarket promo code NYPMAX1: Deposit , get  for Buccaneers vs. Lions

Polymarket promo code NYPMAX1: Deposit $10, get $25 for Buccaneers vs. Lions

November 22, 2026
Timelime of Fiona the hippo’s life at the Cincinnati Zoo, why millions mourn her

Timelime of Fiona the hippo’s life at the Cincinnati Zoo, why millions mourn her

November 22, 2026
Tallulah Willis Shares Rare Photo With Her Dad Bruce Willis Amid His Health Struggles

Tallulah Willis Shares Rare Photo With Her Dad Bruce Willis Amid His Health Struggles

November 22, 2026
Saints vs. Bears prediction: Caleb Williams won’t destroy New Orleans

Saints vs. Bears prediction: Caleb Williams won’t destroy New Orleans

November 22, 2026

Subscribe to News

Get the latest USA news and updates directly to your inbox.

Latest News
Beaver Moon 2026: See a full ‘supermoon’ rise this week, and another on Christmas Eve

Beaver Moon 2026: See a full ‘supermoon’ rise this week, and another on Christmas Eve

November 22, 2026
Stephen A Smith isn’t the only one who knows dissing White people comes without consequences

Stephen A Smith isn’t the only one who knows dissing White people comes without consequences

November 22, 2026
Who Attended Miley Cyrus and Maxx Morando’s Wedding? Celebrity Guest List Revealed

Who Attended Miley Cyrus and Maxx Morando’s Wedding? Celebrity Guest List Revealed

November 22, 2026
Facebook X (Twitter) Pinterest WhatsApp TikTok Instagram
© 2026 USA Times. All Rights Reserved.
  • Privacy Policy
  • Terms
  • Advertise
  • Contact

Type above and press Enter to search. Press Esc to cancel.