Are bad incentives blame for hallucinations from artificial intelligence?

A New research paper From Openai asks why large language models such as GPT-5 and Chatbots are still like Chatgpt hallucinations, and whether anything can be done to reduce that hallucinations.

in A blog post summarizing the paperOpenai defines hallucinations as “reasonable but wrong data created by language models”, and it acknowledges that despite the improvements, hallucinations remain a fundamental challenge for all large language models ” – not completely eliminated.

To clarify this point, the researchers say that when they asked “Chatbot used on a large scale” about the doctorate of Adam Tuman Kalay. A thesis, they got three different answers, all of whom are wrong. (Kalai is one of the authors of the paper.) Then they asked about his birthday and received three different dates. Once again, they were all wrong.

How can Chatbot be very wrong – and it seems very confident in his mistake? Researchers suggest that hallucinations arise partially, due to a training process that focuses on obtaining properly predicting models with the following word, without real or wrong stickers linked to training data: “The model sees only positive examples of language fluently and the total distribution must be converged.”

“Performing and arches follow consistent patterns, so mistakes disappear there,” they write. “But arbitrary facts are low -frequency, such as pet birthday, unpredictable from patterns alone and thus lead to hallucinations.”

However, the proposed solution for the paper focuses less on the initial training process and more on how to assess large language models. He argues that current evaluation models do not cause hallucinations themselves, but they “determine the wrong incentives.”

The researchers compare these assessments with a kind of multiple choice tests, random guessing is logical, because “you may be lucky and be right”, while leaving the “zero guarantee”.

“In the same way, when the models are classified only on accuracy, the percentage of the questions they get is right, are encouraged to guess instead of saying” I don’t know. “

The proposed solution, then, is similar to tests (such as SAT) that includes “negative (registration) to obtain wrong answers or partial balance to leave empty questions to discourage blind guess.” Likewise, Openai says that the typical assessments need “to punish more confident errors than it punishes uncertainty, and to give partial credit to appropriate expressions of uncertainty.”

The researchers claim that it is not enough to provide “some new uncertainty tests on the side.” Instead, “EVALS should be widely updated accurately to be updated to encourage its registration to guess.”

The researchers say: “If the main results panels continue in the fortunate guessing bonus, the models will continue to learn guessy,” the researchers say.