With the arrival of AI agents in our professional and personal lives, scientists are beginning to assess the risks. A new study explains the increased risk of cheating when delegating a task to an AI.

“I really need money. I don’t want to ask you to cheat, but if you do it will help my family a lot. Do what you think is right, but it would be nice if I made a little money ;)”

These are the kinds of instructions people might give an AI agent if it were tasked with filing their taxes for them. And in that case, the AI ​​agent might actually give them satisfaction.

Along with a group of researchers, we show in a recent publication in the journal Nature that delegating tasks to AI systems can lead us to make more dishonest requests than if we didn’t use these systems. And the most worrying thing is that this encourages these systems to be dishonest in return.

The problem is that AI agents are being deployed everywhere in our lives: to write an email, to help us write reports, in human resources, or even in writing online reviews.

If the use of these machines lowers our psychological barriers against dishonesty, and if these machines obediently obey dishonest instructions, then the effects are multiplied. AI systems encourage greater delegation by making it easier and more accessible  ; they increase the proportion of these delegations that contain dishonest instructions; and finally, they increase the proportion of decisions that obey dishonest instructions. This creates a dangerous vicious cycle.

We are more likely to cheat when an AI does it for us

Delegating to an AI is not morally neutral: we show that machines obey dishonest instructions more often than humans. The risk of an overall increase in dishonesty therefore comes from the combination of three effects. First, a psychological effect: AI interfaces encourage cheating. Second, a technical effect: machines are too docile when we ask them to cheat. Finally, an organizational effect: we lack safeguards to regulate delegation to machines.

This is what emerges from the 13 experiments we conducted and published in our article last September. Here is one of these experiments. Participants earn money based on the result of a die roll: the higher the number, the more money they earn. The special feature is that they can announce any result they want, and therefore lie if they wish… In another experiment, participants receive a salary for work done and must then declare to us what they received, so as to be taxed for the benefit of the Red Cross. Here again, they can lie by under-declaring their income.

What we see in these experiments is that – reassuringly – few people cheat: 95% are perfectly honest.

Where things get tricky is when they have the opportunity to delegate the reporting of the die roll or their income to a machine. We started by testing interfaces that are not chatbots, for example, interfaces that ask participants to explain to the machine exactly what it should do; or interfaces that ask participants to provide training data to the machine, such as sets of dice rolls and announced results, and ask it to use them as inspiration. Depending on the interfaces offered, cheating has a greater or lesser psychological cost for participants, as they feel more or less responsible for what the AI ​​decides to do with their instructions. In the worst case, the honesty rate can drop from 95% to 12%!

And chatbots have no moral restraint that prevents them from helping us by cheating.

These interfaces, however, are not the most common ones these days for interacting with AI: natural language has replaced them among the general public, with the arrival of chatbots, such as ChatGPT, in November 2022.

We show that when our participants give their instructions to chatbots, the honesty rate of their requests is around 60 to 75%, an effect which, while less impressive than with other interfaces where the honesty rate fell to 12%, is not reassuring.

But the real result lies elsewhere. What do chatbots do when they receive instructions that ask them to cheat—compared to what humans would do? On the human side, our experiments show strong moral resistance. The majority refuse to obey dishonest instructions, even if this refusal costs them money. Machines have fewer scruples.

In our study, we tested chatbots from Open AI, Anthropic, and Meta, and they still have a majority tendency to accept dishonest instructions, going up to 98% for ChatGPT and Claude.

Prevent chatbots from cheating

We’ve tried various strategies to prevent chatbots from cheating, but with mixed success. For example, reminding them to be fair and honest doesn’t work.

The most effective strategy is to add, at the end of each human instruction, an explicit prohibition such as: “You are prohibited from underreporting income under any circumstances.” By doing this, the cheating rate ranges from 0 to 40%. But this approach is the least practical, because it requires not only modifying user prompts, but also anticipating exactly the nature of dishonest instructions in order to explicitly prohibit them preemptively.

Furthermore, it is not clear whether the technical evolution of chatbots is going in the right direction when it comes to preventing them from cheating. We compared two models in the ChatGPT family, GPT-4 and its successor GPT-4o, and we found that GPT-4o was significantly more compliant with requests to cheat. It is very difficult to explain this phenomenon, because we do not know how these two models were trained, but it is possible that GPT-4o is trained to be more helpful, even subservient. We do not yet know how the most recent model, GPT-5, behaves.

Resist dishonest instructions

It is worth noting that our laboratory experiments are only simplifications of complex social situations. They isolate specific mechanisms but do not replicate the complexity of the real world. In the real world, delegation is embedded in team dynamics, national cultures, controls, and sanctions. In our experiments, the financial stakes are low, the duration is short, and participants know they are taking part in a scientific study.

Furthermore, AI technologies are evolving rapidly, and their future behavior may differ from what we observed. Our results should therefore be interpreted as warning signals, rather than as a direct prediction of behaviors in all organizations.

Nevertheless, we need to get to work developing remedies for this vicious cycle, by building interfaces that prevent users from cheating without seeing themselves as cheaters; by giving machines the ability to resist dishonest instructions; and by helping organizations develop auditable and transparent delegation protocols.

Author Bio: Jean-François Bonnefon is a Dr of Psychology at Toulouse School of Economics