Safe Software Means Safe Decisions

The vision of the self-driving company stands or falls with one number: around 80 percent of all operational decisions are made by software, while people decide in the remaining exceptional cases and set the framework. Many executives hear a loss of control in that first. I see the opposite, though under one condition: the software must be built so that its decisions are traceable, verifiable, and capable of learning. This article explains how software algorithms decide, why neural networks and deep learning keep raising the quality of those decisions, and how you recognize a safe software decision.

Why the decision is the touchstone

In "Das selbstfahrende Unternehmen" (Springer Gabler 2021), I defined the threshold this way: when about 80 percent of operational functions and decisions are automated and controlled by sufficiently intelligent algorithms, the company is self-driving. The number is deliberately an order of magnitude, because companies do not pass through the stages of evolution as a whole but in individual areas, departments, and teams. I described the stages themselves in the summary of the book.

What matters is what happens with the remaining 20 percent. People decide where software reaches its limits: in genuine exceptional cases and at the meta level, meaning the strategic guidelines and the ethical, ecological, and social conditions within which the software is allowed to decide. Management thus shifts from making individual operational decisions to setting the framework. Because every system state can be inspected at any time, this framework is not only prescribed but continuously checked. And because the state of the overall system is known, decisions can be simulated in advance instead of being tried out in daily operations.

Those who reject the vision in a single sentence usually argue on grounds of safety: software can make mistakes, its logic is opaque, nobody bears responsibility. All three objections are legitimate, and all three can be addressed once you understand how software arrives at decisions.

How software algorithms make decisions

Entrepreneurs know that a large part of their working day consists of decisions. Exactly that becomes the core task of software algorithms. Their advantage lies in two properties: they handle volumes of data no human can survey, and their decision behavior can be logged completely. An algorithmic decision emerges from computing relationships in the data; which data went in and which parameters tipped the balance can be traced step by step afterwards.

People decide differently. They almost always have incomplete information, weigh things intuitively, and, under time pressure, stress, and fear, in other words in ordinary office life, are subject to systematic biases that behavioral economics has described for decades. That is not an accusation, but it does suggest using the strict logic and data breadth of software for the bulk of operational and tactical decisions and deploying people where they are irreplaceable: in the meta decision that creates a coherent framework for all individual decisions.

This does not mean that software decides without error. Especially at the beginning of a learning process, an algorithm frequently makes wrong decisions. The difference from a human organization lies in how they are handled: the error is detected by people or by other algorithms and reported back to the algorithm, which corrects its decision parameters so that the changed configuration already takes effect at the next decision. In this model, errors are not an operational disruption but the systematic basis for goal-directed learning. In a department where errors are covered up to secure relative advantages, this feedback loop does not exist.

Deep learning: why decision quality rises

The fact that the quality of algorithmic decisions increases over time rests on the principle of neural networks. At first these are simply dumb. They consist of nodes connected to one another, and the network decides solely on the basis of these connections. Each node has a single task: it continuously takes inventory of the signals it receives and passes a result on to its neighbors. This process follows a mathematically simple construct that has been described for decades (Gasteiger and Zupan 1999). Deep learning refers to the interconnection of many such layers in order to reach a higher level of intelligence beyond simple tasks.

The principle becomes tangible in image recognition. A digital image is a bitmap of points with different color and gray values. The first layer of the network checks only whether each point is light or dark and passes the result on. The next layer determines whether there is a contrast between neighboring points and recognizes lines from that. Further layers distinguish hard from soft edges and straight from curved lines, until at some point a layer recognizes a nose or an ear, counts them, and infers a face. If the system receives many images of faces, it learns which features a bitmap must have for a face to be present. With a database, these faces can finally be assigned to individuals, as unlocking smartphones and managing photo libraries have long shown in everyday life.

In this way an algorithm learns at high speed and soon clearly surpasses its human colleagues in clearly defined tasks. Where a person reaches the end of their learning curve, the software keeps learning with every additional data set. For the first self-driving Tesla, the number of images the system learned from was around 70,000 in 2017; today it is several million, and the error rate falls with the volume, as I described in the book. Since the end of November 2022, moreover, anyone can try out with ChatGPT what a deep neural network does with language. I consider that an important moment, because the technology becomes tangible for executives. For business decisions, however, a clear distinction applies: a language model produces plausible text; it does not decide on the basis of company data and does not disclose its reasons. The decision networks of the self-driving company are trained for the specific use case, work on verified data, and log every decision.

I expect that over the coming ten years many applications in companies will rest on such self-learning algorithms and that an ever denser network of software decisions will emerge from them, permeating the entire company on the basis of shared data.

What makes a software decision safe

Safety is not a property of the algorithm but of the system in which it is embedded.

From our projects of recent years, four conditions have crystallized that make a software decision safe:

• Traceability. Every decision is logged with its input data, the rules applied, and the result. What is not logged can be neither checked nor corrected.
• Transparency. All system states can be inspected at any time. Deviations from the targets are detected in real time, not only in the quarterly report.
• Feedback. Detected errors flow back to the algorithm as a learning signal. A system without this loop repeats its errors with high precision.
• Framework. People define at the meta level which goals are pursued and which limits are observed. The software optimizes within this framework, never beyond it.

One caveat belongs here, and I did not conceal it in the book: with self-learning algorithms, decision behavior changes with every error that is reported back. After some time of active learning, the individual decision is no longer readily traceable for a human. That is why logging and verifiability belong in the specification of decision-capable software from the outset, not in a later add-on. The European Commission's proposal for an AI regulation, under negotiation since April 2021, points in exactly this direction with its requirements for transparency and human oversight.

The comparison shows where software is superior to people and where people remain:

Overall optimum and scenarios instead of departmental logic and gut feeling

Software-driven companies reach a new dimension of task handling: they decide more accurately and faster, solve more varied problems, do not tire, and work day and night, without weekends or vacations, at very low long-term cost. The essential reason for the new quality, however, lies elsewhere: the software can be strictly aligned with the overall optimum of the company.

Anyone who has worked for a longer time in a larger, hierarchically structured organization knows the opposite. Over time, every department develops its own system logic, looks after its own interests, withholds information, or passes on embellished figures. At best, this behavior is uncovered once a year in a workshop, and two weeks later everything is back to the way it was. The complete transparency of the overall system ends this game, because every deviation is immediately visible. Overarching control remains with people, but they exercise it selectively, and the reporting system behind it delivers at the push of a button what controlling teams today spend weeks compiling.

The second argument is the time horizon. Because of their limited capacity to absorb and process information, people are barely able to grasp fully the far-reaching consequences of a decision. In the firm belief that they are thinking rationally, considerable gaps remain. Nobody can calculate in their head the number of scenarios that would have to be considered as the future unfolds. Software in the self-driving company accesses all company data in real time, combines it with external data, and computes any desired scenario with all knock-on effects for the entire company. As a result, a secured data basis already exists at the time of the strategic decision, from which the operational plans are derived.

The volumes of data show how big the task is: according to a forecast by the International Data Corporation (cited from Statista 2020), the global data volume will grow from 33 zettabytes in 2018 to 175 zettabytes in 2025, more than fivefold. The real problem for companies is not storing this data but actively using it. Anyone already failing at the flood of emails today will not cope with this volume by adding more employees. Software must take over this task so that the top decision-makers can remain human.

The organization follows the decision

Companies are under constant pressure: market changes, new technologies, competition from start-ups, individual customer wishes, staff shortages. Classic, hierarchically structured organizations with poorly connected departments often cannot keep up with this pace, and the potential that software decisions offer for rapid adaptation remains untapped.

That is why the organizational form changes along with the decisions. Products, services, and business functions can only be adapted quickly if decisions and tasks are shared and decentralized. The rigid boundaries between departments dissolve, and interactive interfaces to suppliers, authorities, and partners emerge through which decisions are coordinated automatically. The greatest advantage of this software-based organization is at the same time its simplest rule: everyone consistently pursues the same goals, because the system does not allow otherwise. I described what this structure looks like in the article The corporate structure for true agility, and why algorithms rather than processes form the core in the article The end of processes, long live the algorithms.

The next step

Safe software means safe decisions, but only if traceability, transparency, feedback, and framework are specified from the outset. You will find the complete vision with the stages of evolution up to 2035 at About the book. Which decisions in your company can be handed over to software first, and which requirements belong in the specification for that, we will gladly clarify in a personal meeting: Book an expert meeting.