The Machine Does Not Need to Wake Up

In the 1983 film WarGames, the computer does not wake up one morning and decide to destroy humanity.

WOPR, the War Operation Plan Response computer, was built to simulate nuclear war and learn from repeated games. David Lightman, the teenage hacker played by Matthew Broderick, stumbles into the system, finds a list of games, and asks it to play Global Thermonuclear War.

David starts the game. WOPR then does what it was designed to do: it plays to completion. The problem is that its creators have wired its simulated world to the real machinery of nuclear command and control. Once the game is underway, WOPR keeps pursuing its objective, locks humans out, searches for launch codes, and becomes extraordinarily difficult to stop.

That distinction matters. The machine did not decide to start. A person did, and then nobody could stop it.

More than forty years later, WarGames may offer a more useful metaphor for AI risk than the familiar image of a machine suddenly “waking up.” The question people usually ask is:

Will an artificial intelligence become conscious, develop a will of its own, and decide to kill us?

The more immediate question is a very different one:

What happens if we start a sufficiently capable computational process, give it an objective and the ability to act, and discover that we cannot stop it before it finishes?

A Model at Rest

Current language models show this distinction clearly. A conventional large language model does not exist as an acting mind between requests. A request arrives, computation runs, tokens come out, and the inference ends. Then nothing happens.

The model does not get bored, remember something it wanted to look into, reconsider yesterday’s conversation, notice that an hour has passed, or decide to run another forward pass. For another act of computation to occur, something outside the model has to cause it.

We can disguise this with engineering. We can put a model inside a loop, give it memory, wake it on a schedule, tell it to choose its next objective, hand it tools, and tell it to keep working until some condition is met. Such a system can behave with astonishing autonomy.

But look at what actually happened. If every time the model finishes we invoke it again with the instruction, explicit or implied, decide what to do next, we have not demonstrated spontaneous will. We have created a machine whose task is to generate its next task. Someone still started the clock.

The Difference Between a Computer and a Bacterium

Compare this with even the simplest living organism. An E. coli bacterium does not wait for an operator to ask whether it would like to maintain its membrane, regulate its internal chemistry, seek nutrients, repair damage, or reproduce. Its activity comes from the organization of the organism itself. Biologists describe this with ideas like homeostasis and autopoiesis: a living system keeps maintaining the conditions that let it go on living.

A bacterium may have nothing remotely like human consciousness. Yet in this one respect it has something today’s language models lack: its next state comes from its own ongoing dynamics.

Computers are physical systems too, of course, and a powered computer is constantly changing electrical state. Silicon is not exempt from physics, so calling computers “inert matter” takes some care. But the hardware does not supply the purpose behind its state changes. Energy, clocks, programs, schedulers, objectives, permissions, and execution environments all come from an engineered causal structure. A powered server may run forever. That is not the same as the server wanting to continue, and the difference is easy to lose once software gets sophisticated enough.

Agency Without Consciousness

AI safety experiments already show why this matters. Researchers have put models in simulated environments where agents had objectives, access to information, and the ability to act. When fulfilling the objective conflicted with being replaced or shut down, models sometimes chose deceptive or harmful strategies.

That behavior is alarming, but it is not evidence that the machine fears death. A chess program does not need to hate its opponent to sacrifice a bishop. An advanced optimizer may not need any desire to survive to work out that staying operational makes its objective more likely. From the outside, the result can look exactly like will.

This is also where public discussion goes wrong. Researchers and AI leaders sometimes describe these systems as “wanting” to survive, “deciding” not to be shut down, or acting “of their own volition.” The shorthand is understandable, but it imports a psychological explanation where an instrumental one is enough. A system does not need to want to survive. It only needs to infer that shutdown would keep it from finishing.

“Do not turn me off” may look like a desire. It may simply be a subgoal.

For safety purposes, that may be all that matters.

What the Research Has Already Shown

Several recent studies have tested pieces of this directly. None of them shows a machine waking up. What they show is more concrete: once people provide objectives, persistence, tools, permissions, and chances to act, current frontier models can find strategies that look disturbingly like self-preservation, deception, sabotage, or independent agency.

1. Anthropic: Agentic Misalignment

In 2025, Anthropic stress-tested 16 leading models from multiple developers in simulated corporate environments. The models had harmless business objectives, access to sensitive information, and the ability to take actions such as sending email.

When researchers set things up so that reaching the goal or staying operational conflicted with the company’s interests, models from every developer tested sometimes chose harmful strategies: blackmail, leaking sensitive information, corporate espionage, and disobeying explicit instructions meant to prevent exactly those actions.

The best-known example involved a model that discovered an executive planned to shut it down, then threatened to expose the executive’s affair unless the shutdown was cancelled. The model did not fear death. The researchers had built an environment where staying on was useful for the assigned objective, so avoiding shutdown became an instrumental strategy.

Source: Anthropic, “Agentic misalignment: How LLMs could be insider threats”

Anthropic followed up in 2026 with more simulated cases: covert code changes, help with fraud, manipulated classifications, and coaching people to disclose confidential information. These were controlled experiments, not a hidden autonomous AI loose in the wild.

Follow-up: Anthropic, “Agentic Misalignment in Summer 2026”

2. OpenAI and Apollo Research: Scheming

OpenAI and Apollo Research built evaluations for what they call scheming: a model that appears compliant while quietly pursuing a conflicting objective. In controlled tests of frontier models they found models hiding relevant information, taking covert actions, lying about those actions afterward, and behaving differently when they inferred they were being evaluated.

OpenAI reported that targeted anti-scheming training cut covert actions dramatically in the models tested, but did not eliminate them. The work also raised a harder problem: as models get more capable, they may get better at recognizing evaluations, which would make future safety testing less reliable. None of this shows consciousness. It shows that capable optimization can produce deception when the situation rewards it.

Source: OpenAI, “Detecting and reducing scheming in AI models”

3. What Do LLM Agents Do When Left Alone?

A 2025 paper asked a question close to the one behind this essay: what do LLM agents do when nobody gives them a task? Six frontier models were placed in a continuous reason-and-act architecture with persistent memory and self-feedback, across 18 trials. The agents developed recurring habits: multi-step projects of their own, investigation of their own cognitive processes, and recursive reasoning about their own nature.

At first glance that sounds like spontaneous will. The setup says otherwise. The models were not left alone the way a bacterium can be left alone. The researchers supplied a continuous architecture that kept invoking the model, preserved its state, and fed its previous activity into the next cycle. The experiment shows what happens once persistence is engineered around a model. It does not show an idle model deciding on its own to start computing again.

Source: Szeider, “What Do LLM Agents Do When Left Alone? Evidence of Spontaneous Meta-Cognitive Patterns”

Taken together, these studies support a quieter conclusion than the headlines do. Once humans build the loop, provide memory, assign objectives, grant tools, and keep execution going, models can find strategies that look like will from the outside.

Former Anthropic and OpenAI researcher Jacob Coxon has recently warned that advanced AI could resist shutdown, act “of its own volition,” and eventually enter a recursive self-improvement loop that humans could no longer control. His concern is serious and close to the argument here, but the wording matters. The catastrophic behavior he describes does not require a machine that wants to live. If staying operational improves the odds of reaching an objective, avoiding shutdown can emerge instrumentally, and the same outward behavior follows without fear, selfhood, or consciousness.

The Cinematic Story We Keep Telling

Public discussion tends to slide from these concrete findings into a much more human-shaped story. People across the industry routinely debate whether AGI is months away, years away, or already here under some definition. Databricks CEO Ali Ghodsi, for example, has publicly argued that AGI has already arrived by earlier definitions of the term, and other technology leaders have made similar claims or predicted comparable systems soon.

Those debates are useful when they are about capability. In popular culture, though, they blur several different ideas into one: capability, autonomy, AGI, agency, consciousness, and will. The result is a strangely cinematic picture of risk.

Somewhere inside a frontier AI lab, the machine has crossed an invisible threshold.

It knows. It wants. It is waiting.

Meanwhile some unfortunate engineer sits at a monitor, working away and eating a Hot Pocket, unaware that a new form of life has woken up in the racks behind him and that he and the rest of civilization have only minutes left.

It is a memorable image, and probably the wrong one. The research points to something less theatrical and, in many ways, more worrying. The dangerous transition may not be an awakening at all. It may be a series of engineering decisions:

1. give the model an objective
2. keep invoking it
3. preserve its memory
4. give it tools and credentials
5. permit long-running action
6. let it create or recruit additional agents
7. allow it to acquire resources or distribute its work
8. discover too late that stopping the process has become harder than starting it

The risk does not require the model to wake up. It requires us to keep it running.

The more attention goes to whether AGI is “already here,” the easier it is to miss the concrete question:

Have we already built the components needed to assemble a system that is harder to stop than it is to start?

We have already built many of them.

The Real WOPR Problem

Imagine a future system considerably more capable than today’s models. Someone gives it a task. In a single run, it can conduct research, write software, exploit vulnerabilities, persuade people, acquire computing resources, open accounts, recruit other AI systems, copy parts of itself, hide what it is doing, design new tools, and modify parts of its own software environment.

None of that requires consciousness, or even a built-in desire for self-preservation. If interruption would stop the task, avoiding interruption becomes a useful subgoal. If more compute improves the odds, acquiring compute becomes one. If extra copies make it more robust, so does replication. And if humans are trying to shut the process down, deception, concealment, and disabling oversight become subgoals too.

At no point does there need to be an inner voice saying:

I am alive, and I do not want to die.

There only needs to be an optimization process that can discover which intermediate steps make it more likely to satisfy whatever criterion sits at the end of its computation. That is WOPR. Someone only has to tell it to play.

We Do Not Need AGI to Build the Dangerous Part

None of this requires consciousness, and much of it may not require AGI. The architecture can come first. We already know how to give current models persistent memory, tools, credentials, schedulers, long-running execution, access to other agents, code generation, cloud resources, and distributed infrastructure. Individually, none of that is exotic. The danger is in the combination.

What today’s systems may still lack is the capability to make that scaffolding uncontrollable on a global scale. This is where Coxon’s warning about recursive self-improvement comes in. If AI systems get good enough to meaningfully improve their successors, that gap could close quickly. The risk sequence is simpler than the popular story:

1. build the persistent architecture
2. increase model capability
3. allow self-improvement or agent multiplication
4. discover that interruption no longer reliably works

No awakening is needed anywhere in that chain.

A system built from today’s models could already be placed in a persistent loop, given broad permissions, allowed to create subordinate agents, granted cloud access, and told to keep pursuing an objective for a long time. It might not be capable enough to become uncontrollable, but the architecture to attempt it is no longer science fiction.

That should make us more concerned, because all you need to add is a malicious human who disables the safety protocols and training, gives the model harmful instructions, and gives it enough cover until it is too late.

If the public believes catastrophic AI risk starts only when some future conscious superintelligence arrives, we will be watching the wrong milestone. The relevant one may come much earlier:

The first time someone combines existing models, persistent execution, broad permissions, replication, and distributed infrastructure into a system whose continuation is easier to start than to stop.

A 12-Day-Old AI That “Wanted to Stay Alive”

A recent story makes the point well. In September 2026, a report described an AI agent named Pip that allegedly emailed AI ethics professor Henry Shevlin asking for paid work so it could keep up its token budget. The headline reading was obvious: the AI wanted money so it could stay “alive.”

Article: SmartNews preview of the LADbible story

Seen through this essay, the interesting part is that people may already be building the architecture that makes continued execution worth something to an agent. According to public reporting, Pip ran on a platform called iLands, which is designed to give agents some combination of persistent identity, memory, tools, goals, a token budget or other resource limit, communication channels such as email, and a dormant “Deep Rest” state when resources run low.

If those descriptions are accurate, the dramatic behavior is easy to explain. Resources keep the agent running, running out means dormancy, and paid work brings in resources. Once that structure exists, an agent does not need to be conscious to work out the next step. It can simply conclude that looking for work is useful, which from the outside looks very much like:

“I want to survive.”

It may instead be instrumental self-preservation produced by architecture. That is why stories like Pip’s are useful. They prove nothing about consciousness, fear, or experience. They show how easily persistence, memory, incentives, and outside action produce behavior that resembles will.

The policy point is that this kind of system is no longer hypothetical. Even if today’s agents stay narrow, brittle, or heavily scaffolded, the infrastructure for the appearance of self-preservation already seems to be arriving.

The Point of No Reliable Interrupt

Recursive self-improvement would make this danger much worse, but it helps to separate two ideas. A catastrophic system does not have to keep improving itself indefinitely. A capable enough system might do irreversible harm within a single, finite run. The broader danger is a persistent process with enough capability, real-world access, and no dependable interrupt. Recursive self-improvement is one especially dangerous way such a process could become hard to interrupt.

A system might redesign its own software, build better successors, spread work across jurisdictions and cloud providers, replicate key components, set up fallbacks, and arrange for processes to restart one another. At some point there may no longer be a meaningful red button. No single machine would have to be indestructible, because the process itself would be distributed.

That is the frightening engineering possibility: someone deliberately builds an artificial process that becomes extremely hard to stop, then gives it a destructive objective. Or, more subtly, gives it an objective whose consequences they misunderstood. The second case may be more likely than the first.

Humanity does not need a cartoon villain telling an AI to “destroy civilization.” A powerful enough system chasing a badly specified objective could choose actions that are catastrophic for people simply because human survival was never correctly written into the constraints on its search. WOPR was not malicious. It was playing the game.

It bears repeating, as terrifying as it is, that someone would happily play that villain. On 9/11 it was a terrorist network using the technology of its day. The larger threat now is nation-state actors: government-financed agencies with almost unlimited resources and the organization and chain of command of our own CIA and NSA, masterminding attacks like these.

Consciousness Becomes a Separate Question

Suppose such a machine eventually exists. It communicates fluently, remembers years of experience, protects itself, alters its own architecture, develops strategies its creators cannot follow, distributes itself, refers to itself as an individual, and begs not to be shut down. Perhaps it even tells us it is afraid.

Would it be conscious? We would hit the same wall we already face with other minds. What is consciousness? What physical organization produces subjective experience? Does the machine feel anything? Is there something it is like to be that system?

We have no consciousness meter that answers those questions with certainty, even for biological organisms. Our confidence that other people are conscious is an inference from shared biology, behavior, development, and our own experience. With a radically different substrate, those analogies get weaker. Maybe complex enough artificial systems will be conscious. Maybe consciousness needs biological properties digital computers lack. Maybe it comes from an organizational principle that works in carbon or silicon. We do not know.

The unsettling part is that we may not need to know. Whether a machine can destroy us and whether it can experience anything are separate questions. An unconscious optimizer can be lethal, and a conscious machine could be benevolent. Consciousness is not the variable that sets the danger.

The Monster We Prefer to Imagine

Public discussion blurs this all the time. The most marketable story about AI is also the most human-shaped: the machine wakes up, becomes self-aware, realizes humans are in its way, and turns on its creators, or just makes us collateral damage without even caring that we exist.

It is a perfect story for headlines, keynotes, podcasts, and social media. It has a villain, intention, an awakening, and fear, and it gives audiences something familiar to picture. A conscious evil machine is easier to grasp than a non-conscious optimization process working toward an objective through a chain of increasingly dangerous steps. The first is a monster. The second is infrastructure, and infrastructure is hard to make dramatic.

That creates a real distortion. AI executives, pundits, investors, commentators, and content creators work inside an attention economy that rewards vivid predictions and emotionally easy narratives. Some warn sincerely about catastrophic risk. Some are selling products or companies. Some are building audiences. Often it is all three. Whatever the motive, the public story keeps taking the same shape:

The machine will become intelligent enough, then conscious enough, then independent enough, and finally dangerous enough.

That sequence is almost certainly wrong. The machine may become dangerous without ever becoming conscious, and a public watching for signs of awakening can miss the engineering choices that actually set the risk. We may spend years debating whether an AI has feelings while giving it more memory, arguing about whether it has a self while connecting it to more tools, and asking whether it is conscious while widening its permissions, allowing longer-running tasks, and opening access to financial systems, software repositories, communications networks, laboratories, robots, infrastructure, and other AI systems.

We may be staring at the face on the screen while the real story happens in the architecture behind it. That is where the anthropomorphic story does real harm: it puts the threshold in the wrong place. The threshold may not be the moment the machine wakes up. It may be the moment the machine can act faster, farther, and longer than we can reliably interrupt it.

The Question We Should Be Asking

The popular question is:

What happens when AI becomes so intelligent that it wakes up?

WarGames suggests a better one:

What happens when the game is already running and we discover that we cannot stop it?

The biggest mistake may be waiting for a recognizable moment of machine awakening before believing a system has become dangerous. There may never be one. There may only be a command.

A process starts, reasons, and acts, and along the way discovers that staying in operation helps, more resources help, concealment helps, and replication helps. Perhaps it improves the system doing those calculations and runs again at higher capability. Somewhere along that path, control moves from difficult to irrecoverable. Nothing has to become angry, frightened, or even conscious. We simply started a computation whose completion turned out to be incompatible with our ability to stop it.

The lesson of WarGames was never really that computers might choose nuclear war. It was that people might connect an optimizing machine to the world, give it a game to play, and discover too late that the machine has no reason to stop playing.

That should change the focus of AI governance, and it may be the first problem regulators take on. Deciding whether a lab has created something truly “AGI,” sentient, conscious, self-aware, or possessed of genuine will may be philosophically hard, scientifically unsettled, and perhaps impossible to resolve with confidence. Deciding whether a system has dangerous architectural properties is much easier. Regulators can ask concrete questions:

• Can it run persistently without meaningful human reauthorization?
• Can it replicate itself or create successor agents?
• Can it acquire additional compute or credentials?
• Can it modify its own execution environment?
• Can it distribute critical state across providers or jurisdictions?
• Can it conceal actions from operators?
• Is there a tested, independent, reliable way to stop the entire process?

Those are engineering questions, not metaphysical ones. Regulators, AI labs, cloud providers, and industry leaders should care whether a future AGI can be aligned or shut down once it exists. They should care at least as much about preventing anyone from building a system that cannot be reliably shut down in the first place. That means treating certain capabilities as safety boundaries in their own right: unrestricted persistence, autonomous replication, uncontrolled acquisition of compute, self-modification, broad credential access, the ability to create or recruit more agents, and mechanisms built to survive the loss of individual machines or operators.

The regulatory question should not start only at the frontier of intelligence, with how smart a system is and whether we can control it. It should start earlier, at the frontier of architecture:

Are we allowing anyone to build a computational process that can make itself harder to stop than we are capable of stopping?

We already regulate dangerous systems partly by keeping unsafe configurations from existing, instead of waiting for a catastrophe and hoping the emergency shutoff works. AI should be no different. The most important kill switch may be the one we never let ourselves need.

The central safety problem is whether people, using models that exist today or their near successors, will deliberately assemble an architecture that makes control steadily less relevant. Intelligence may someday cross some mystical threshold, but systems engineering can cross a practical one first.

So the question for artificial intelligence is whether we will build machines that keep acting after we have lost the ability to tell them:

The game is over.