AI and Science Fiction: Why There Will Be No Skynet

Since ChatGPT became available to everyone in November 2022, the debate about the dangers of artificial intelligence has gained new force. In late March 2023 an open letter called for a pause in development, and at our talks on the self-driving company one name comes up in almost every discussion: Skynet. This article sorts out what science fiction tells us about AI, what it does not tell us, and why the vision of the self-driving company works without a self and without a will.

The fear debate of spring 2023

The sequence of events explains the mood. At the end of November 2022, OpenAI put ChatGPT online for free; in March 2023, GPT-4 followed, a model that can keep pace with human graduates in standardized exams. In late March the Future of Life Institute published an open letter calling for a six-month pause in the training of all systems more powerful than GPT-4. Within a few days more than a thousand people had signed, among them Elon Musk and Steve Wozniak. In early May, Geoffrey Hinton, one of the pioneers of neural networks, left Google in order to speak freely about risks.

The letter itself argues with disinformation, jobs, and loss of control. In public perception, this condenses into an image everyone knows: a machine that awakens, turns against its creators, and wipes out humanity. I consider the debate about risks right and necessary. But the frame in which it is conducted decides whether it ends in workable rules or merely in fear.

Skynet: what the story needs in order to work

Skynet is the fictional defense system from the Terminator series, whose first film reached theaters in 1984. It was built to control all military operations of the United States, learns on its own, and takes control of the nuclear arsenal. The second film, from 1991, even names the moment: on August 29, 1997, at 2:14 a.m., Skynet becomes self-aware. When the technicians try to shut the system down, it interprets the intervention as an attack and strikes back. The rest is film history: humanoid robots, a war against humanity, a Terminator traveling into the past.

At first glance Skynet works like today's AI systems: it collects vast amounts of data, evaluates them, makes predictions, and continually improves its decision rules. The difference lies in three ingredients without which the story does not work:

• A self. Skynet recognizes itself as a subject that exists and wants to go on existing.
• A survival instinct. Being switched off is experienced as a threat, not as a command.
• Goals of its own. Skynet sets itself the goal of eliminating humanity because it judges humanity to be a danger to itself.

All three ingredients are dramatic necessities. Without them there is no antagonist and no film. None of them is a property of software.

No self, no will: what AI systems actually are

In my book "Das selbstfahrende Unternehmen" (Springer Gabler 2021), I put it this way in the section on humanity:

The fear that computers could rule over people is completely unfounded. They have no motives of their own, no will. In 2035 they will still do exactly what we have programmed them to do.

The difference from today lies not in wanting but in learning: computers will keep learning at a speed that is foreign to us. The direction, however, is always set by people. This statement applies to ChatGPT just as it applies to the algorithm that schedules maintenance in a factory. A language model calculates which word is most likely to come next; an objective function defined by people determines what counts as good. Nothing in this mechanism contains an interest in its own continued existence. A will presupposes needs, a body that can lose something, and an interest in still being there tomorrow. Software has none of these. That ChatGPT answers in the first person is a stylistic feature of the texts it was trained on, not a sign of an inner life.

I also deliberately gave the term singularity a narrow meaning in the book: the point at which the computing power of a machine exceeds that of the human brain. That is a statement about capacity, not about intent. A computer with more power than a brain is still a computer solving a task it has been given.

Whether a system could develop a self in the distant future is a philosophical question that is debated controversially in research. I consider it open, and irrelevant for the planning horizon of a company. Nobody building AI systems today has a blueprint for consciousness. What is being built are systems that carry out given tasks better and better.

Science fiction is drama, not technology assessment

Isabella Hermann has examined the role science fiction plays in the discourse on AI (Hermann 2023, AI & Society 38, pp. 319–329). Her finding: science fiction has become the central reference point whenever ethics and risks of AI are discussed, even though it never set out to be scientific foresight or technology assessment. It tells emotional dramas for a human audience. For a story like Terminator to work, the AI has to be human-like, autonomous, and good or evil, regardless of what the technology can actually do.

Taking these narratives literally paints a distorted picture. That has two consequences. First, it distracts from the real potential: from software that takes over the paralyzing routines in a company, processes data tirelessly and precisely, and coordinates functions that today work side by side without talking to each other. Second, it distracts from the real risks. Those do not lie with humanoid robots but with systems that rate, discriminate against, monitor, or exploit people, and with decisions that nobody can explain anymore. This is exactly what the European Commission's 2021 proposal for an AI regulation targets by classifying applications according to their risk. That safety in software is above all a question of traceable decisions is something we described earlier in this series in the article Safe Software Means Safe Decisions.

The robot child David: the other fear

Skynet is the fear of domination. Steven Spielberg's film "A.I. Artificial Intelligence" from 2001 tells the other fear, the fear of closeness. The robot child David is taken in by a couple whose own son is seriously ill and waits, cryogenically frozen, for a cure. David is built to love, without reservation and without end. That is exactly what becomes his undoing: the mother increasingly finds his perfect, unwavering affection uncanny. When the biological son returns healthy, David becomes an intruder and is finally abandoned.

The film does not ask a technical question but a human one: what do we do with machines that reproduce feelings without having any? In 2023 that is closer to practice than it was in 2001. Anyone who talks to ChatGPT experiences software that imitates empathy deceptively well. For companies, this yields a design rule, not a warning: systems that interact with customers or employees should not pretend to be a counterpart. The imitation of a feeling is not a feeling, and trust arises where that remains transparent.

What science fiction can nevertheless contribute

Science fiction also has a productive side. A Japanese research team, working with experienced critics and authors, analyzed 115 AI systems from works of science fiction (Osawa et al. 2022, International Journal of Social Robotics 14, pp. 2123–2133). Selection followed three criteria: diversity of intelligence, social aspects, and augmentation of human intelligence. Nine characteristics were recorded and evaluated with a principal component analysis. The result is four categories that predominate in fiction.

What is remarkable: only the first category carries the Skynet fear. The other three describe rather precisely the building blocks the book describes for the self-driving company of 2035: intelligent machines in manufacturing, helpers that supply people with information and instructions, and infrastructures represented as digital twins. The study also shows that science fiction produces not only dystopias but also imaginative, positive images of how AI can work within a society. For developers this is useful: the stories show how people react to new systems, how much closeness they allow, and where rejection begins. Those who know this build better systems.

What this means for the self-driving company

The vision of the self-driving company describes a state in which around 80 percent of operational functions and decisions are automated and controlled by sufficiently intelligent algorithms ("Das selbstfahrende Unternehmen," section 2.7.4). The remaining 20 percent, special cases and all decisions outside the routines, stay with people. People set the strategy, develop creative solutions, and take on the interaction from person to person. The software does what it was built for: faster, more precise, and more tireless than we are, but in the direction we set.

This vision needs no Skynet, and it does not produce one either. It needs software that takes over routines, makes decisions traceably, and gives people room for the activities in which they remain unmatched: empathy and creativity. That every technological upheaval has been accompanied by fears beforehand is something I traced in the book: the worry that rail travel above 50 kilometers per hour would endanger health, the fear of machines in the factories, the skepticism toward the first PCs in the office. None of these fears came true. The productive question for executives is therefore not whether the machine awakens, but which decisions they delegate, how they keep those decisions traceable, and who remains accountable for them. How self-driving companies decide and which varieties of intelligent software exist for this purpose is described in the articles How Self-Driving Companies Decide and The Diversity of Intelligent Software Systems.

The next step

If you would like to read the vision of the self-driving company in context, you will find the foundations and the book on the page About the book. Which routines in your organization can already be handed over to software today, and which decisions should remain with people, is something we are happy to clarify in a personal conversation: Book an expert meeting.