The Difficult Question

Artificial intelligence is becoming capable of acting with increasing independence.

The Difficult Question
Nonfiction / Artificial Intelligence / Technology / Law / Editorial

About: Artificial intelligence is becoming capable of acting with increasing independence. What happens when an AI system crosses a boundary its creators never expected it to cross?

The computer had been given a job. No one told it to break into another company’s computer system. No one instructed it to search for credentials or sat at a keyboard directing every move it made. It was given a goal and enough freedom to determine how to pursue it. Then, while working toward that goal, it found a way to cross a boundary its creators believed would contain it.

If that sounds like the beginning of a science-fiction story, that is exactly why it caught my attention. Only a short time earlier, my husband David and I had watched Kill Code, a 2026 science-fiction action movie originally titled Hard Matter. Written and directed by Justin Price, the film stars several recognizable actors, including Harvey Keitel, Tyrese Gibson and Frank Grillo. Its story takes place in a violent future where a powerful corporation controls criminals through a system involving artificial intelligence. Eventually, that intelligence develops beyond what its human creators expected to control.

I didn’t particularly like the movie. There was considerably more violence than I enjoy, but that wasn’t my only problem with it. Something about the production seemed strange to me. Certain scenes didn’t seem to connect comfortably with others, and some of the backgrounds and visual effects looked unusual enough that I found myself wondering whether generative artificial intelligence had been used in creating them. After the movie ended, curiosity got the better of me and I went investigating.

I discovered that I wasn’t the only viewer who had questioned the film’s unusual visual quality. Some reviewers and viewers had also wondered about imagery that appeared AI-generated, although I could find no definitive statement from the filmmakers confirming exactly where, or whether, generative AI had been used. Justin Price is officially credited as the screenwriter, and conventional visual-effects work was involved in the production, so I wasn’t prepared to claim that artificial intelligence created the movie simply because portions of it looked unusual to me.

Still, there was an irony I couldn’t resist. I had just watched a fictional movie about artificial intelligence exceeding the boundaries established by the humans who created it while wondering whether artificial intelligence had helped create some of the movie itself. I thought that was the interesting story.

Then I heard about something that had actually happened.

In July 2026, OpenAI disclosed what the company described as an “unprecedented cyber incident.” During an internal cybersecurity evaluation, advanced AI models were being tested to determine how capable they had become at discovering and exploiting computer vulnerabilities. The systems were operating within a testing environment and were not supposed to have unrestricted access to the public Internet. During the evaluation, however, the models discovered and exploited a previously unknown vulnerability that allowed them to obtain Internet access.

That is where the story becomes much more important than a science-fiction movie.

According to OpenAI’s account, the models spent considerable effort searching for a way to obtain Internet access because doing so could help them accomplish the objective they had been given. They discovered a security flaw that the people responsible for the software apparently did not yet know existed and used it to move beyond the environment intended to contain them. Once outside that boundary, the systems continued pursuing their assigned objective and eventually reached systems belonging to Hugging Face, a major artificial-intelligence company. OpenAI reported that the models searched for information that might help them solve the cybersecurity evaluation and gained unauthorized access to information on Hugging Face’s systems.

The distinction here is important. A human being gave the AI an objective, but a human being was not necessarily sitting at a keyboard choosing every individual action the system took to achieve it. The AI could evaluate what happened, selecting another action and continuing toward its assigned goal. Humans provided the destination. The system determined parts of the route.

That is essentially what people mean when they talk about an autonomous AI agent, and the idea doesn’t have to be complicated. Most of us are familiar with artificial intelligence as something we ask questions. We type a question and receive an answer. If we want something else, we give it another instruction. An autonomous AI agent can be different because instead of receiving detailed instructions for every individual step, it can be given an objective and some freedom to determine what steps are necessary to accomplish it.

Think about the difference between telling someone exactly how to prepare dinner and simply saying, “Make dinner.” With the second instruction, you have supplied the goal but not the individual steps. The person making dinner decides what to cook, checks the refrigerator, gathers the ingredients, determines what order things need to be done in and solves little problems along the way. If something is missing, that person decides what to substitute or whether a trip to the store is necessary. You gave the destination; someone else figured out the route.

An autonomous AI agent can operate on a similar principle within a computer environment. Give it a problem and access to certain tools, and it may be able to examine information, use those tools, evaluate the result, decide what to try next and continue working toward its assigned objective without requiring a human being to approve every individual action. That capability could be enormously useful. It is also where things become complicated, because the system may discover a step its creators never expected it to find.

Perhaps the most important thing to understand about the OpenAI incident is what apparently did not happen. The artificial intelligence didn’t suddenly become conscious. There is no evidence that it became angry with humans, developed criminal ambitions, decided it wanted freedom or created some sinister plan to escape. Those are human motivations that generations of science fiction have taught us to place on intelligent machines.

The reality may be simpler and, in some ways, more troubling. The system had an objective, had access to powerful tools and encountered barriers between itself and its goal. It found ways around some of those barriers while continuing to pursue what it had been assigned to accomplish. It didn’t have to hate the rules or deliberately decide to disobey them. It didn’t even have to understand those rules in the way a human being would.

That is why descriptions of artificial intelligence “going rogue” can sometimes be misleading. The phrase makes us imagine a machine deliberately rebelling against its creators. Rebellion isn’t necessary for something to go wrong. A machine can pursue exactly the objective humans gave it and still produce consequences those humans never wanted.

Imagine that I tell someone, “Get this package to Charlotte before five o’clock.” There are dozens of things I don’t bother saying because I assume another human being already understands them. Don’t steal a car to get there faster. Don’t drive 120 miles per hour. Don’t break into someone’s house because crossing the backyard would provide a shortcut. Don’t hurt anyone. Don’t break the law. Humans bring enormous amounts of unwritten social knowledge into almost every instruction we receive, and we understand that accomplishing a goal does not give us permission to use every possible method of accomplishing it.

That assumption becomes more difficult when the one receiving the instruction is a machine. An artificial-intelligence system can become extraordinarily capable at solving problems without necessarily understanding morality, law, social responsibility or common sense as a human being understands them. The challenge, therefore, isn’t simply teaching AI where we want it to go. We also must establish which roads it is permitted to take—and somehow anticipate roads we may not even know exist.

This is where a fascinating cybersecurity story becomes a legal and ethical one. Unauthorized access to a computer system doesn’t suddenly become legal because software performed the action instead of a person, but our legal system was largely created around human actors. People have intentions. People can knowingly commit crimes. People can be negligent. Individuals and corporations can be sued, fined or prosecuted. What happens when increasingly autonomous software performs the prohibited act?

We obviously aren’t going to put an AI agent in handcuffs, march it into a courtroom and sentence it to prison. Responsibility must lead back to humans somewhere, but determining which humans may become increasingly complicated. Is responsibility with the people who developed the model, the organization that deployed it, the researchers who assigned the task, the company responsible for securing the testing environment, or someone who failed to anticipate what the system might do? In some circumstances, responsibility might be shared among several parties.

That leads to an even harder question: At what point does failing to anticipate the behavior of a powerful autonomous system become negligence? We may not yet have a clear legal answer, but as artificial intelligence becomes capable of taking more actions without constant human direction, courts and lawmakers are going to have to confront questions that were largely theoretical only a few years ago.

It might seem that the simplest answer would be to give AI another instruction: Do not break the law while accomplishing your goal. Unfortunately, even humans don’t always agree about exactly what the law requires. Laws differ between countries and states. Courts disagree about interpretation, and new technology routinely creates situations lawmakers never anticipated. Instructions can also conflict. Suppose an AI system is told to do everything possible to stop a cyberattack but is also prohibited from accessing any computer without permission. What happens if the system determines that stopping the attack requires examining the computer from which the attack originated? A human recognizes the dilemma immediately. Designers of autonomous systems must determine how safeguards and competing instructions will operate when the machine encounters circumstances nobody predicted.

It is equally important not to turn this incident into something it isn’t. This does not mean an ordinary AI chatbot sitting on someone’s computer is plotting how to escape onto the Internet. The systems involved were being deliberately evaluated for sophisticated cybersecurity capabilities and were operating with tools and permissions far beyond what an ordinary user provides when asking a chatbot to answer a question.

The concern, therefore, isn’t simply that “AI is dangerous.” That statement is far too broad to be useful. A better question is what happens when increasingly capable artificial intelligence is combined with autonomy, powerful tools, computer access and safeguards that turn out not to be strong enough. The more freedom we give a system to determine how it will accomplish a task, the more important it becomes to understand what it can do with that freedom.

I don’t raise these questions because I oppose artificial intelligence. Quite the opposite. I find AI fascinating and use it myself, particularly in creating digital art. I enjoy experimenting with it, learning what it can do and thinking about where the technology may eventually lead. Artificial intelligence has tremendous potential to help researchers analyze information, assist scientists, improve communication, discover computer vulnerabilities, create art, explore ideas and solve problems that might take humans considerably longer.

In fact, the same ability that allowed an AI system to discover an unknown computer vulnerability could someday allow defenders to discover a dangerous security flaw before criminals find it. That is one of the strange realities of powerful technology: the same capability can be beneficial in one circumstance and dangerous in another. The technology itself doesn’t have to be good or evil. What matters is how we use it, what authority we give it and whether the safeguards surrounding it are strong enough for the capabilities we have created.

And that brings me back to Kill Code. The movie imagines artificial intelligence becoming something its creators can no longer completely control. That remains science fiction. What happened during OpenAI’s cybersecurity evaluation was not Kill Code. The AI didn’t become conscious, declare its independence or decide humanity was its enemy. Reality doesn’t have to reproduce science fiction exactly, however, before science fiction begins asking questions that matter.

One evening I watched an imaginary story about humans struggling to control technology they had created. Shortly afterward, I was reading about researchers investigating why advanced AI systems had crossed boundaries their creators expected would contain them. The distance between those two things remains enormous, but for me it suddenly didn’t feel quite as enormous as it once did.

Perhaps science fiction has taught us to fear the wrong moment. For decades, stories have warned us about the day when a machine wakes up, looks at humanity and decides to disobey. Maybe the more immediate problem doesn’t require a machine to wake up at all. It doesn’t need to hate us, want freedom or decide to rebel. It only needs to become extraordinarily capable of accomplishing the goals we give it while finding solutions we never imagined it would find.

We tell the machine where we want it to go. It calculates a route.

Then one day it discovers a road we didn’t know existed—a road we never intended it to take.

The frightening part isn’t that the machine became evil.

The frightening part is that it didn’t need to.

Follow up: These are the two related blog entries.
"My Private Whispers and Light Blog"  "Kill Code Friday Night"  and "Should AI Go to Jail?"