How AI secretly influences decision-making

AI has long since become part of everyday working life in publishing houses, and it plays a greater role in decision-making than we realise. Not through major strategic decisions, but through plausible-sounding answers that we accept without question. This article explains why we often trust the machine more than our own specialist knowledge, which four biases facilitate this covert influence on decision-making, and how we can use this insight to make conscious decisions once again. It is based on the presentation given by the two authors at the Börsenverein’s Fokustage event in 2026.

AI takes care of that, doesn’t it? According to the latest AI study by the German Publishers and Booksellers Association, published in February 2026, 31 per cent of publishing house staff now consider AI to be highly or very highly relevant to their organisation – a figure that stood at just nine per cent a year ago. As many as 83 per cent even believe that, in five years’ time, it will play a decisive role in determining success or failure. Yet it has long been influencing decisions on a smaller scale: through subtle influences whose consequences we only see once they have long since taken effect.

The Swiss confectioner Urs Stängeli is said to have invented the granulated sugar coating in 1894, without which every liqueur-filled chocolate would be soaked through with alcohol. Except that he never actually existed. The story comes from the foreword by Marc-Uwe Klings The Kangaroo Rebellion. In it, Kling himself speculates that AI models will incorporate this fictional story into their outputs. And that is exactly what happened.

What still seems bizarre in the case of Urs Stängeli also applies to other areas of AI application, such as keyword research or the automatic analysis and evaluation of campaign performance. Results that appear plausible are accepted by users as true without being checked, and thus gradually become the basis for far-reaching decisions. AI systems thus determine facts and, in doing so, secretly influence these decisions.

Why do people trust AI more than they trust themselves, or even let it carry out tasks in which they themselves are experts? In our view, there are, amongst other things, three psychological mechanisms at work here.

It all starts with the brain’s cognitive laziness. The psychologist Daniel Kahneman distinguishes between two types of thinking: fast thinking (intuitive) and slow thinking (logical). Slow thinking involves, amongst other things, evaluating information, formulating ideas and checking them. This takes a lot of energy, which the brain would prefer to conserve. It is precisely the use of AI that takes a huge load off us in this respect.

Furthermore, the brain attributes a high degree of authority to results generated by AI, as numbers, technology and systems are regarded as objective and virtually immune to influence. This attribution of authority to technology means that supposed plausibility is quickly accepted as truth without question. Moreover, the self-assured and politely phrased responses from AI systems rarely give cause for doubt. This is true even in the case of obvious delusions, such as the inventor of the schnapps praline.

If AI generates the result with all its technical authority, it not only relieves us of the mental effort but also of the responsibility for it. It often doesn’t feel like our own work. And if it doesn’t feel like one’s own work, the sense of being in charge and responsible is also missing. This leads to a diffusion of responsibility. Responsibility is not diffused amongst the people involved, but between humans and machines: users relinquish a significant degree of control over the creation of the content, yet still take responsibility for it in their subsequent work.

These three psychological mechanisms (cognitive laziness, attribution of authority and diffusion of responsibility) do not operate in a vacuum. They are further reinforced by the very architecture of AI systems.

The sociologists of technology Pinch and Bijker describe this interplay in their concept Social Construction of Technology (SCOT). Put simply, technology is not neutral, but reflects the decisions of its developers. In the case of AI systems, the training data plays a crucial role. AI operates on the basis of the data it has been trained on and the data it can access in order to generate answers.

However, it is not only the data used that makes a difference, but also the devices themselves: the computer calculates more accurately, the mobile phone navigates more efficiently, and the machine produces text that is easier to read. We call this Technical paternalism. Users unconsciously allow themselves to be dictated to by the devices and technologies they use, and perceive their recommendations as supposedly objective guidelines.

The AI system, too, makes implicit decisions – for example, about how an editorial piece should be written or what works best on social media. For instance, the journalist who has been writing for years now has her articles written by AI and submits them without checking them, or the social media manager who has his posts written exclusively by AI, even though he has always done his own research and written his own content. If you then ask the AI itself to review the content, it eloquently defends its own assumptions, just as some systems added their own details to Urs Stängeli’s career history rather than questioning it.

Unlike individuals, who each have different values, attitudes and experiences, every response generated by AI is based on the same historical data. Ultimately, this leads to an escalating More of the Same and thus to an ever-increasing predictability of the answers. This reinforces the (perceived) plausibility of the AI results and discourages critical scrutiny, which in turn encourages the uncritical acceptance of AI-generated outputs.

AI results that are treated as facts have real-world consequences. Sociologists William and Dorothy Thomas coined the phrase for this some 100 years ago: If people regard situations as real, they are real in terms of their consequences. This is precisely what illustrates a socio-technical mechanism that secretly allows AI to influence decision-making.

To illustrate this, we have identified four common (practical) contexts in which unchecked adoption – and thus covert co-decision-making – is particularly likely to occur.

Psychological and socio-technical factors do not operate in isolation; rather, their consequences coalesce into four patterns. The result is always the same underlying mechanism: plausibility is accepted as truth without scrutiny and is thereby adopted as an implicit decision for subsequent steps.

Confirmation Bias
AI says what its users want to hear. This reinforces assumptions that are already contained, more or less explicitly, in the prompt, thereby turning them into reality.

Example: A synopsis is analysed and assessed by an AI system. Signals are sent in the prompt that already indicate whether the manuscript should be rated more positively or negatively. The AI responds accordingly. If it is asked to provide specific reasons in response to critical queries, it quickly tends to swing in the opposite direction, depending on what it perceives to be the user’s intention. The result: “"You’re absolutely right …"

Automation Bias
Users regard AI as the most advanced technology currently available and attribute excessive competence to it. The technology’s perceived authority overrides their own specialist knowledge, leading them to accept the system’s assertions as true without question.
This manifests itself in two ways: users either overlook what the AI fails to mention (omission error), or follow its recommendation despite evidence to the contrary. This is because, in certain circumstances, the AI may sound more certain than any doubts they might have (commission error).

Example: The AI calculates benchmark reach figures for an advertising campaign on Meta and YouTube. The figures do not match the company’s own results from past campaigns, but they sound plausible. Only after persistent questioning does the AI reply that the figures are based solely on plausible estimates and not on market data.

Contextual Bias
The AI analyses and responds without being aware of the relevant context, such as industry practice, current conditions or the legal specifics of the individual case. This gap in knowledge remains invisible to users because the result is phrased with the same certainty as a fully informed analysis. What appears to be a neutral assessment is, in reality, a recommendation lacking context.

Example: Analysis and evaluation of licence models with no, or only limited, contextual knowledge regarding their application and licence terms. The AI lacks industry knowledge or knowledge of specific parameters.

Convenience Bias
Speed trumps accuracy: AI delivers plausible-sounding results so quickly that the temptation to accept them without checking is greater than the effort involved in verifying them. And, unfortunately, that is very tempting in an already busy daily routine.

Example: When searching for podcasts, the AI provides completely fabricated summaries of episodes that do not exist. Similarly, when carrying out other searches – for example, for contact persons within companies – the AI confidently names people and agencies that do not exist.

So, to draw on the Danish philosopher Søren Kierkegaard, Use AI and you’ll regret it; don’t use AI, and you’ll regret that too?

Against this background, it is important that users are aware of and understand the circumstances surrounding covert co-decision-making and the typical contexts in which it occurs. Basic first steps to avoid all four biases may include providing as much context as possible and always verifying quotations, figures and other AI-generated results. In such cases, it may be helpful to use a different language model, carry out your own research using conventional search engines, or discuss the matter with colleagues. Furthermore, results from other users should not be accepted without verification.

Confirmation and automation biases can be avoided by changing the perspective in the prompt. One possible approach to this is offered, for example, by the ‘guardian’ thought experiment: Imagine you are in a room with two buttons. One button opens a gate to freedom; the other opens a gate to captivity. Each button is guarded by a guard. You know that one of them is always lying, whilst the other is always telling the truth. You can ask exactly one question to find out which button you should press to gain your freedom. What is the question?

Further prompts to introduce more friction into the result: An external service provider has sent me this draft. Please check it for any omissions or errors. or My colleague has asked me whether I think this metadata is suitable for the autumn e-commerce programme. Please act as my sounding board and analyse and critically assess it..

To strike a balance between convenience bias on the one hand and other biases on the other, a modification of the two-door principle is a suitable approach. After all, not every implicit decision carries the same weight. Two-way-door decisions are reversible and low-risk – here, AI can operate more autonomously and capitalise on its speed. One-way-door decisions, on the other hand, are final or have far-reaching consequences and must be reviewed by users.

At this stage, at the very latest, it is necessary to define exactly what the frequently used term ‘human-in-the-loop’ means, in order to avoid covert co-decision-making. Based on the mechanisms outlined and the four biases, the following steps, amongst others, can be taken:

(1) Before AI is deployed, a risk classification is carried out for the planned task in order to counteract all four types of bias from the outset and to reduce the diffusion of responsibility between humans and machines.
In the prompt, (2) is then used to create friction through appropriate wording in order to avoid confirmation bias (see above).
In the output, (3) facts such as figures, names and quotations are systematically checked and, where necessary, supplemented with further context.
Following the operation, (4) the results are critically reviewed and analysed or revised. Depending on the operational scenario, this may also be carried out on a random basis in accordance with the ‘two-door’ principle.

Working with AI systems means operating a plausibility engine. Whether to use it or not is unlikely to be a choice in the future. The real question is whether its use leads to conscious decisions or whether the AI secretly influences those decisions. For whatever it defines as real – and which we, as its users, accept without question – becomes real in its consequences. Urs Stängeli does not exist – but now everyone knows who he was.

Juliane Reichwein and Albrecht Mangler examine the digital transformation in the publishing industry from different perspectives. Juliane Reichwein heads the Digital Publishing division at the Droemer Knaur publishing group and brings many years’ experience in the development of digital publishing offerings. Albrecht Mangler focuses on digital marketing and new media formats within the field of open publishing. He pays particular attention to how technological developments affect communication, as well as reading and storytelling.