

Metaverse
AI models make stuff up. How can hallucinations be controlled? – Crypto News
There are kinder ways to put it. In its instructions to users, OpenAI warns that ChatGPT “can make mistakes”. Anthropic, an American AI company, says that its LLM Claude “may display incorrect or harmful information”; Google’s Gemini warns users to “double-check its responses”. The throughline is this: no matter how fluent and confident AI-generated text sounds, it still cannot be trusted.
Hallucinations make it hard to rely on AI systems in the real world. Mistakes in news-generating algorithms can spread misinformation. Image generators can produce art that infringes on copyright, even when told not to. Customer-service chatbots can promise refunds they shouldn’t. (In 2022 Air Canada’s chatbot concocted a bereavement policy, and this February a Canadian court has confirmed that the airline must foot the bill.) And hallucinations in AI systems that are used for diagnosis or prescription can kill.
All the leaves are brown
The trouble is that the same abilities that allow models to hallucinate are also what make them so useful. For one, LLMs are a form of “generative” AI, which, taken literally, means they make things up to solve new problems. They do this by producing probability distributions for chunks of characters, or tokens, laying out how likely it is for each possible token in its vocabulary to come next. The mathematics dictate that each token must have a non-zero chance of being chosen, giving the model flexibility to learn new patterns, as well as the capacity to generate statements that are incorrect. The fundamental problem is that language models are probabilistic, while truth is not.
This tension manifests itself in a number of ways. One is that LLMs are not built to have perfect recall in the way a search engine or encyclopedia might. Instead, because the size of a model is much smaller than the size of its training data, it learns by compressing. The model becomes a blurry picture of its training data, retaining key features but at much lower resolution. Some facts resist blurring—“Paris”, for example, may always be the highest-probability token following the words “The capital of France is”. But many more facts that are less statistically obvious may be smudged away.
Further distortions are possible when a pretrained LLM is “fine-tuned”. This is a later stage of training in which the model’s weights, which encode statistical relationships between the words and phrases in the training data, are updated for a specific task. Hallucinations can increase if the LLM is fine-tuned, for example, on transcripts of conversations, because the model might make things up to try to be interesting, just as a chatty human might. (Simply including fine-tuning examples where the model says “I don’t know” seems to keep hallucination levels down.)
Tinkering with a model’s weights can reduce hallucinations. One method involves creating a deliberately flawed model trained on data that contradict the prompt or contain information it lacks. Researchers can then subtract the weights of the flawed model, which are in part responsible for its output, from those of the original to create a model which hallucinates less.
It is also possible to change a model’s “temperature”. Lower temperatures make a model more conservative, encouraging it to sample the most likely word. Higher temperatures make it more creative, by increasing the randomness of this selection. If the goal is to reduce hallucinations, the temperature should be set to zero. Another trick is to limit the choice to the top-ranked tokens alone. This reduces the likelihood of poor responses, while also allowing for some randomness and, therefore, variety.
Clever prompting can also reduce hallucinations. Researchers at Google DeepMind found that telling an LLM to “take a deep breath and work on this problem step-by-step” reduced hallucinations and improved problem solving, especially of maths problems. One theory for why this works is that AI models learn patterns. By breaking a problem down into smaller ones, it is more likely that the model will be able to recognise and apply the right one. But, says Edoardo Ponti at the University of Edinburgh, such prompt engineering amounts to treating a symptom, rather than curing the disease.
Perhaps, then, the problem is that accuracy is too much to ask of LLMs alone. Instead, they should be part of a larger system—an engine, rather than the whole car. One solution is retrieval augmented generation (RAG), which splits the job of the AI model into two parts: retrieval and generation. Once a prompt is received, a retriever model bustles around an external source of information, like a newspaper archive, to extract relevant contextual information. This is fed to the generator model alongside the original prompt, prefaced with instructions not to rely on prior knowledge. The generator then acts like a normal LLM and answers. This reduces hallucinations by letting the LLM play to its strengths—summarising and paraphrasing rather than researching. Other external tools, from calculators to search engines, can also be bolted onto an LLM in this way, effectively building it a support system to enhance those skills it lacks.
Even with the best algorithmic and architectural antipsychotics available, however, LLMs still hallucinate. One leaderboard, run by Vectara, an American software company, tracks how often such errors arise. Its data shows that GPT-4 still hallucinates in 3% of its summaries, Claude 2 in 8.5% and Gemini Pro in 4.8%. This has prompted programmers to try detecting, rather than preventing, hallucinations. One clue that a hallucination is under way lies in how an LLM picks words. If the probability distribution of the words is flat, ie many words have similar likelihoods of being chosen, this means that there is less certainty as to which is most likely. That is a clue that it might be guessing, rather than using information it has been prompted with and therefore “knows” to be true.
Another way to detect hallucination is to train a second LLM to fact-check the first. The fact-checker can be given the “ground truth” along with the LLM’s response, and asked whether or not they agree. Alternatively, the fact-checker can be given several versions of the LLM’s answer to the same question, and asked whether they are all consistent. If not, it is more likely to be a hallucination. NVIDIA, a chipmaker, has developed an open-source framework for building guardrails that sit around an LLM to make it more reliable. One of these aims to prevent hallucinations by deploying this fact-checking when needed.
Although such approaches can decrease the hallucination rate, says Ece Kamar, head of the AI frontiers lab at Microsoft, “it is unclear whether any of these techniques is going to completely get rid of hallucinations.” In many cases, that would be akin to self-sabotage. If an LLM is asked to generate ideas for a fantasy novel, for example, its output would be disappointing if limited to the world as it is. Consequently, says Dr Kamar, her research aims not to get rid of all hallucinations, but rather to stop the model from hallucinating when it would be unhelpful.
Safe and warm
The hallucination problem is one facet of the larger “alignment” problem in the field of AI: how do you get AI systems to reliably do what their human users intend and nothing else? Many researchers believe the answer will come in training bigger LLMs on more and better data. Others believe that LLMs, as generative and probabilistic models, will never be completely rid of unwanted hallucinations.
Or, the real problem might be not with the models but with its human users. Producing language used to be a uniquely human capability. LLMs’ convincing textual outputs make it all too easy to anthropomorphise them, to assume that LLMs also operate, reason and understand like humans do. There is still no conclusive evidence that this is the case. LLMs do not learn self-consistent models of the world. And even as models improve and the outputs become more aligned with what humans produce and expect, it is not clear that the insides will become any more human. Any successful real-world deployment of these models will probably require training humans how to use and view AI models as much as it will require training the models themselves.
Curious about the world? To enjoy our mind-expanding science coverage, sign up to Simply Science, our weekly subscriber-only newsletter.
© 2023, The Economist Newspaper Limited. All rights reserved. From The Economist, published under licence. The original content can be found on www.economist.com
-
others1 week ago
Gold retreats while Fed Powell and President Trump clash over interest rates – Crypto News
-
Technology1 week ago
Top 10 air coolers for monsoon: Handpicked products for effective cooling from trusted brands – Crypto News
-
others1 week ago
EUR/JPY steadies near 169.00 as traders await the next catalyst – Crypto News
-
Cryptocurrency1 week ago
Friday charts: Retail is one-upping Wall Street – Crypto News
-
Technology1 week ago
Too many messages to read? WhatsApp launches AI-enabled summary feature for unread chats. Here’s how it works – Crypto News
-
Blockchain1 week ago
Breakout To $2,800 Or Crash To $2,000? – Crypto News
-
Cryptocurrency1 week ago
SHIB Price Prediction for June 26 – Crypto News
-
Cryptocurrency1 week ago
Wormhole price jumps 12% amid Ripple’s XRPL integration – Crypto News
-
Cryptocurrency1 week ago
Permissionless IV, Day 3 takeaways – Crypto News
-
others1 week ago
USD/INR drops to two-week low as Rupee gains on weak US Dollar – Crypto News
-
others6 days ago
Japan CFTC JPY NC Net Positions increased to ¥132.3K from previous ¥130.9K – Crypto News
-
others1 week ago
AI-Focused Layer-1 Blockchain Altcoin SAHARA Flames Out Following New Binance Listing – Crypto News
-
Blockchain1 week ago
Deaton Says Ripple IPO Could Trigger $100B Valuation, How High Will The XRP Price Be? – Crypto News
-
Cryptocurrency1 week ago
Vodafone Share Price Tests 78p Ahead of July Earnings, Is a Breakout Imminent? – Crypto News
-
Cryptocurrency1 week ago
US Housing Chief Orders Fannie Mae, Freddie Mac to Prepare for Crypto Assessment in Mortgages – Crypto News
-
De-fi1 week ago
Barclays to Ban Crypto Purchases via Credit Card – Crypto News
-
Technology1 week ago
Microsoft launches Mu AI model for smart local tasks on Windows PCs – Crypto News
-
Cryptocurrency1 week ago
Is LTC ready for a breakout after testing the $75 low? – Crypto News
-
Cryptocurrency1 week ago
TRON price forecast as USDT supply surpasses $80 billion – Crypto News
-
De-fi1 week ago
Sei Soars 70% as Wallet Growth and On-Chain Activity Hit New Highs – Crypto News
-
Cryptocurrency1 week ago
Komodo tanks 25% after Binance announces delisting – Crypto News
-
De-fi1 week ago
Russia’s Central Bank Pushes CBDC Launch to 2026 – Crypto News
-
Blockchain1 week ago
Bitcoin Crash To $100K Likely Due To Tariffs, War And Weather – Crypto News
-
Technology1 week ago
It isn’t a dream. Tech stocks are breaking out—and the gains can keep coming. – Crypto News
-
De-fi1 week ago
Aptos DEX Activity Hits Record, Nears $200 Million in Daily Volume – Crypto News
-
De-fi1 week ago
U.S Judge Denies Ripple-SEC Request to Lift Injunction and Reduce $125 Million Fine – Crypto News
-
Technology7 days ago
Best 5G phone under ₹10,000 in June 2025: Lava Storm Play, Samsung M06 and more – Crypto News
-
Metaverse1 week ago
AI valuations are verging on the unhinged – Crypto News
-
Cryptocurrency1 week ago
FUNToken’s Telegram Bot Gets a Major Upgrade With High-Stakes Spins – Crypto News
-
De-fi1 week ago
Mastercard, Chainlink Integration Expands Web3 Access as Firms Push TradFi and DeFi Convergence – Crypto News
-
Blockchain1 week ago
The Smarter Web Company Raises $56M After 196 Bitcoin Buy – Crypto News
-
Technology1 week ago
Is Google preparing to bring Pixel Call Screening to India after 7 years? Know what report suggests – Crypto News
-
others1 week ago
USD/INR drops to two-week low as Rupee gains on weak US Dollar – Crypto News
-
others1 week ago
Sweden Trade Balance (MoM) fell from previous 6.6B to 3.9B in May – Crypto News
-
Technology1 week ago
Xiaomi Mix Flip 2, a clamshell foldable smartphone, launched: Price, features and more – Crypto News
-
Cryptocurrency1 week ago
Is LTC ready for a breakout after testing the $75 low? – Crypto News
-
others1 week ago
United States 4-Week Bill Auction down to 4% from previous 4.06% – Crypto News
-
Cryptocurrency1 week ago
Will Huge $15 Billion Bitcoin Options Expiry Impact Crypto Markets? – Crypto News
-
Technology1 week ago
Google to stop Chrome updates for older Android versions: Should you be worried? – Crypto News
-
Cryptocurrency1 week ago
Epic $100 Trillion Prediction Issued by Jim Cramer: What’s Going On? – Crypto News
-
others1 week ago
Asian Crime Syndicates Tap Chase, Bank of America, Wells Fargo and Other Lenders To Launder Billions Siphoned in Pig Butchering Scams: Report – Crypto News
-
De-fi1 week ago
Bitcoin Consolidates as U.S. Inflation Ticks Higher – Crypto News
-
Technology1 week ago
Facebook wants your photos to train its AI and create edits, even if you don’t upload them – Crypto News
-
Technology1 week ago
Poco F7 vs iQOO Neo 10: Which phone is a better buy under ₹35,000? – Crypto News
-
others1 week ago
Monetary policy well-positioned, time for patience and care – Crypto News
-
Cryptocurrency1 week ago
Reclaims $2.20 Support But Upside Traction Weak – Crypto News
-
Blockchain1 week ago
Is The Bitcoin Top In? Bitcoin MVRV-Score Has The Answer – Crypto News
-
Cryptocurrency1 week ago
BTC recovers to $107K after weekly volatility; focus shifts to US economic data – Crypto News
-
others1 week ago
Mass Bank Fraud Scheme Taken Down After Woman Found With Suitcase Filled With Fake Driver’s Licenses, Credit Cards and IDs: DOJ – Crypto News
-
Blockchain1 week ago
Ethereum Fakes Out Bears – Altcoin Rally Depends On Key Level Breakout – Crypto News