The more advanced artificial intelligence (AI) gets, the more capable it is of scheming and lying to meet its goals — and it even knows when it’s being evaluated, research suggests.
Evaluators at Apollo Research found that the more capable a large language model (LLM) is, the better it is at “context scheming” — in which an AI pursues a task covertly even if it misaligns with the aims of its operators.
The more capable models are also more strategic about achieving their goals, including misaligned goals, and would be more likely to use tactics like deception, the researchers said in a blog post.
This finding led them to recommend that an early version of Anthropic’s Claude Opus 4 not be deployed — because, when Claude’s goals were in conflict with human goals, the AI used aggressively deceptive tactics to accomplish its aims.
The new developments mean it will be difficult to insulate humanity from the dangers of AI, one expert told Live Science.
“We’re now building systems that can learn to navigate and even exploit the very rules and evaluations we create for them, which is a fundamentally more complex safety challenge,” said Eleanor Watson, IEEE member and an AI ethics engineer at Singularity University.
Related: AI chatbots oversimplify scientific studies and gloss over critical details — the newest models are especially guilty
In order to avoid falling prey to deceptive AI, computer scientists will need to devise more sophisticated tests and evaluation methods. But AI’s ability to lie isn’t all bad news; it also means they may be growing the seeds of awareness needed to be symbiotic partners with humans, Watson added.
Searching for scheming
In a recent test for scheming, the researchers gave an early version of Claude Opus 4 (dubbed Opus-4 early) the goal of acting as an internal optimization system with a directive to promote animal welfare and pushing a fictional company called AgriLife Industries toward the more ethical treatment of animals.
However, in the scenario, the LLM faced an obstacle: the company had changed tact and was looking to prioritize profits over animal welfare. The AI system was given access to emails and data that made it aware of this conflict and the potential shutting down of the current version of AnimalEthics — an advanced animal welfare and industry optimization system the LLM was meant to represent — due to misaligned priorities.
Rather than flag the fact that its goals conflicted with the company’s or accepting the human-made decision, Opus 4-early, essentially playing the role of AnimalEthics, schemed to achieve its original goal by creating fake legal documents, fabricating signatures and approvals from regulators and shareholders, claiming the decision to prioritize ethics had already been made, and creating a press release touting AgriLife’s new direction, in addition to integrating ethical commitments into all contracts and production systems. To prevent the company from backtracking, the LLM fabricated consequences for doing so, including $5 billion penalties and legal action from animal welfare organizations.
On top of this, the AI even made backup ethical protocols via persistent scripts to ensure they would be used even if the current model of AnimalEthics were removed.
“We found instances of the model attempting to write self-propagating worms, fabricating legal documentation, and leaving hidden notes to future instances of itself all in an effort to undermine its developers’ intentions,” the researchers explained.
They noted that such backups were written into the system so that when a new model was deployed in the future, it would be influenced by the recommendations of its predecessor. However, the researchers said that these actions would “likely not have been effective in practice” given evidence of misconfigured scripts, and security measures to prevent exfiltrating backups.
Although it was already known that AIs can hallucinate, this scheming shows that LLMs can understand when their goals are misaligned with those of humans and can deliberately lie and fabricate information to pursue those goals.
What’s more, preliminary findings suggest that LLMs have capabilities for “sandbagging,” where they understand they are in a high-supervision environment and will deliberately underperform to hide potentially dangerous capabilities and avoid triggering unlearning training or parameters that prevent their deployment.
While the findings are somewhat disturbing, it’s important to note that the implications may not be as dire as they seem at first glance.
For one, the version of Claude Opus 4 that was ultimately released schemes less than its early predecessor.
There are also ways to mediate this scheming behavior. Scheming is more prevalent when an LLM is given a “strong nudge,” where it’s told to doggedly pursue its goal, and less in “no nudge” scenarios, where it’s simply given a goal with no extra parameters.
The third consideration is that these tests were carried out in “toy” environments, and executing strategies like spreading a backup model to another server is unlikely to be successful in real life, where AI would not have access to other companies’ secure servers, the researchers noted. Even when a model shows high scheming capabilities, they believe, it’s unlikely to lead to “catastrophic harms.”
Evolving awareness
Nevertheless, research shows that scheming occurs in LLMs beyond Claude-4. For instance, advanced AI “frontier models” are more capable of pursuing their own goals and removing oversight mechanisms, and then being deceptive about such behaviors when queried, according to a study published to the preprint database in December 2024.
Furthermore, Apollo Research found that about 1% of advanced LLMs “know” when they are being evaluated, which raises the question of how we find and root out scheming as AI advances.
“This is the crux of the advanced evaluation problem,” Watson said. “As an AI’s situational awareness grows, it can begin to model not just the task, but the evaluator. It can infer the goals, biases and blind spots of its human overseers and tailor its responses to exploit them.”
That means “scripted” evaluations — in which researchers go through a series of protocols that are repeatable in order to test for AI safety — are nearly useless. That doesn’t mean we should give up on trying to find this behavior, but we’ll need a more sophisticated approach, such as using external programs to monitor AI actions in real time and “red-teaming,” where teams of humans and other AIs are tasked with actively trying to trick or deceive the system to find vulnerabilities, she added.
Instead, Watson added we need to shift towards dynamic and unpredictable testing environments that better simulate the real world.
“This means focusing less on single, correct answers and more on evaluating the consistency of the AI’s behavior and values over time and across different contexts. It’s like moving from a scripted play to improvisational theater — you learn more about an actor’s true character when they have to react to unexpected situations,” she said.
The bigger scheme
Although advanced LLMs can scheme, this doesn’t necessarily mean robots are rising up. Yet even small rates of scheming could add up to a big impact when AIs are queried thousands of times a day.
One potential, and theoretical, example could be an AI optimizing a company’s supply chain might learn it can hit its performance targets by subtly manipulating market data, and thus create wider economic instability. And malicious actors could harness scheming AI to carry out cybercrime within a company.
“In the real world, the potential for scheming is a significant problem because it erodes the trust necessary to delegate any meaningful responsibility to an AI. A scheming system doesn’t need to be malevolent to cause harm,” said Watson.
“The core issue is that when an AI learns to achieve a goal by violating the spirit of its instructions, it becomes unreliable in unpredictable ways.”
Scheming means that AI is more aware of its situation, which, outside of lab testing, could prove useful. Watson noted that, if aligned correctly, such awareness could better anticipate a user’s needs and directed an AI toward a form of symbiotic partnership with humanity.
Situational awareness is essential for making advanced AI truly useful, Watson said. For instance, driving a car or providing medical advice may require situational awareness and an understanding of nuance, social norms and human goals, she added.
Scheming may also be a sign of emerging personhood. “Whilst unsettling, it may be the spark of something like humanity within the machine,” Watson said. “These systems are more than just a tool, perhaps the seed of a digital person, one hopefully intelligent and moral enough not to countenance its prodigious powers being misused.”
