How Self-Driving Companies Decide

The vision of the self-driving company rests on the premise that around 80 percent of all operational functions and decisions are handled by software (Schnitzhofer, "Das selbstfahrende Unternehmen", Springer Gabler 2021). Anyone who takes this premise seriously has to clarify what a decision actually is, because the term means something different in psychology, in economics, and in computer science. This post continues the line of thought from January's post Safe Software Means Safe Decisions and shows how people decide, how algorithms calculate, and where the boundary between the two runs.

Three meanings of one word

Originally, a decision denotes the process by which a person selects an action, an intention, or an opinion from several possible alternatives. A decision is therefore, first of all, a deeply human phenomenon. Psychology describes it as a cognitive activity: a person chooses among several options while taking preferences, values, goals, available information, and subjective assessments into account. Emotions and the unconscious play a part in this to a degree that behavioral economics has been documenting for decades.

Economics likewise understands a decision as the choice of an action, but shifts the focus to goals and constraints: costs, efficiency, scarce resources. In companies, moreover, it is rarely individuals who decide, but teams, committees, and hierarchies, each with their own preferences.

For the self-driving company, a third meaning is added, that of information technology: a decision is a function that a computer or an algorithmic system executes in order to derive a specific action or output from a set of inputs. A loan application is approved or rejected, an order is triggered, a ticket is prioritized. This meaning is the only one that can be programmed, and it rests on mathematics.

The mathematical dilemma: calculations produce results

This is where it gets interesting, because mathematics does not know the decision as a concept in its own right. A correctly executed calculation does not lead to a decision but to a result. What computer science calls "decision-making" is a calculation based on supplied information, fixed criteria, and defined rules. Mathematical decision theory formalizes more complex cases with probabilities and utility functions: it asks how an actor who is as rational as possible chooses, from a set of actions, the one that maximizes expected utility under uncertain states of the world. That, too, remains a calculation. The goal, the utility, and the admissible actions come from outside.

In the book I described this as a meta-decision: algorithms do not make independent decisions; programmers have defined the decision space beforehand, technically through thresholds and defined result parameters. The meta-decision sets the frame for all individual decisions; the software fills it in, millions of times and without fatigue. This also applies to the language models that have been discussed so intensely since the launch of ChatGPT in November 2022: they calculate the probable continuation of a text. The result is often astonishing; a decision in the human sense it is not.

A calculation produces a result. It becomes a decision only through the goal that a human has set.

The limit of calculation shows as soon as information is incomplete. Whether a machinery manufacturer builds a new production facility depends on demand, interest rates, supply chains, staff, and politics over the next twenty years. For none of these variables is complete data available. Decision theory can attach probabilities to scenarios, but the probabilities themselves are assumptions.

From problem to decision: who masters which step

Unlike a calculation, there are problems without a unique solution. A problem exists when there is a gap between the desired state and the current situation and no known path closes it. The path from problem to decision runs through five steps, and for each step it is fairly clear whether software or people master it better today.

Software with artificial intelligence already handles the first step considerably better than any person, because it can process many times more data tirelessly and purposefully. In the second step, especially for weighty decisions, human creativity is called for: idea generation, problem-solving techniques, intuition, and unconscious judgment. Only once the alternatives are on the table can it be rationally evaluated which one solves the problem best, taking risks, resources, and goals into account, as far as the information reaches. After the decision, software comes into play again in the self-driving company: it defines the concrete steps, assigns responsibilities, and reports when implementation deviates from plan.

The human dilemma: cognitive errors and irrational behavior

People have a genuine advantage: they can decide under uncertainty, with too little or with too much information. In doing so they rely on unconsciously accumulated experience that is far more extensive than what they consciously process. Precisely this advantage is at the same time the source of many wrong decisions.

Behavioral economics has been studying this for decades. In 1974, Amos Tversky and Daniel Kahneman described in the journal Science how people judge under uncertainty using heuristics and are systematically biased in doing so; prospect theory followed in 1979; for this body of work Kahneman received the Nobel Prize in Economics in 2002. In "Thinking, Fast and Slow" (2011; German edition "Schnelles Denken, langsames Denken", 2012) he sums up the picture in two systems: System 1 works fast, automatically, and intuitively; System 2 slowly, effortfully, and deliberately. Most of the working day runs on System 1, and all the more so the more information, time pressure, and anxiety are involved. The effects are familiar from any office:

• Information overload. With too much information and limited cognitive resources, we often draw on the irrelevant pieces and crowd out the important ones that happen not to be available.
• Stereotyping. New information is assigned to familiar typologies even when it does not fit the pattern. Prejudice rests on this principle, and because the process runs unconsciously, it is hard to reflect on.
• Anchoring effect. The first assessment of a new situation shapes all later judgments, even if it turns out to be wrong (Tversky and Kahneman 1974).
• Halo effect. A single aspect shines so brightly that it distorts all subsequent evaluations.
• Group pressure and risky shift. Teams strive for consensus and harmony, avoid conflict, and thereby systematically blank out risks. Group decisions turn out differently from individual decisions even though the same data is available.
• Mental accounting. Someone who buys a concert ticket for 100 euros and loses it on the way usually does not buy a new one. Someone who loses 100 euros in cash on the way buys the ticket anyway. The loss is identical; the entry on the mental account "ticket" versus "total wealth" is not. The example goes back to Tversky and Kahneman (1981).

Then there are the motives. The American psychologist Steven Reiss (2004) identified 16 basic desires that influence our everyday actions, among them power, acceptance, order, status, vengeance, and tranquility. In hierarchical organizations they generate psychological tensions and conflicts from which countless wrong decisions arise that have nothing to do with the actual problem.

In fairness, intuition is not bad per se. In 2009, Kahneman and Gary Klein worked out when it is reliable: in environments with stable regularities and fast, unambiguous feedback. Firefighters and chess professionals meet this condition; a board that decides on a production facility once every ten years does not. The self-driving company creates exactly this environment: it delivers data in real time and feedback in short cycles, and with that, human decisions improve as well.

What software does better and what it will never be able to do

Against these biases, the advantage of software becomes apparent. It has no prejudices, does not strive for power and status, and bears no grudges. It listens to everyone attentively, does not tire, needs no vacation, and works through nights and weekends. There are no conflicts within the team and none with the manager. It receives all the data it needs and calculates results in a strictly rational way. Intelligent software is therefore the perfect micromanager, pursuing strategic, tactical, and operational goals quickly, precisely, and without error, if we program it to do so.

What it does not have is a motive and a will. Its decision rules are the goals of people, cast into rules: compliance with legal requirements, maximized output, minimized resource use, minimized risk. For a large share of everyday decisions, human involvement can thus be dropped; that is the 80 percent target from the book. The remaining 20 percent are the meta-decisions: setting goals, designing rules, handling exceptions, resolving goal conflicts.

Three tasks follow from this that nobody in the self-driving company can delegate to software:

1. Design the decision system. Which decisions are made by which rules, with which thresholds, from which data? This meta-decision is the actual leadership task.
2. Review the decision system. Algorithms must be checked continuously to ensure that the company acts effectively, efficiently, lawfully, and fairly and protects the interests of all stakeholders, its environment, and the natural environment. Judgment, ethics, and morality remain human.
3. Evolve the decision system. Learning algorithms make mistakes at the beginning; every detected wrong decision is raw material for the next version of the rules.

I consider this division of labor the real relief. Managers no longer have to strain their intuition to check scenarios that a machine can calculate in full. They work on a decision system of a higher order, calmly and freed from the burdens, conflicts, and emotions of everyday business.

The next step

How a company's decision system can be transferred into software step by step, and what role transparency and learning play in that, is described in the book "Das selbstfahrende Unternehmen" (Springer Gabler 2021): About the book. If you would like to examine which decisions in your company could already be made on a rule basis today, we would be glad to talk about it: Book an expert meeting.