

Metaverse
AI models make stuff up. How can hallucinations be controlled? – Crypto News
There are kinder ways to put it. In its instructions to users, OpenAI warns that ChatGPT “can make mistakes”. Anthropic, an American AI company, says that its LLM Claude “may display incorrect or harmful information”; Google’s Gemini warns users to “double-check its responses”. The throughline is this: no matter how fluent and confident AI-generated text sounds, it still cannot be trusted.
Hallucinations make it hard to rely on AI systems in the real world. Mistakes in news-generating algorithms can spread misinformation. Image generators can produce art that infringes on copyright, even when told not to. Customer-service chatbots can promise refunds they shouldn’t. (In 2022 Air Canada’s chatbot concocted a bereavement policy, and this February a Canadian court has confirmed that the airline must foot the bill.) And hallucinations in AI systems that are used for diagnosis or prescription can kill.
All the leaves are brown
The trouble is that the same abilities that allow models to hallucinate are also what make them so useful. For one, LLMs are a form of “generative” AI, which, taken literally, means they make things up to solve new problems. They do this by producing probability distributions for chunks of characters, or tokens, laying out how likely it is for each possible token in its vocabulary to come next. The mathematics dictate that each token must have a non-zero chance of being chosen, giving the model flexibility to learn new patterns, as well as the capacity to generate statements that are incorrect. The fundamental problem is that language models are probabilistic, while truth is not.
This tension manifests itself in a number of ways. One is that LLMs are not built to have perfect recall in the way a search engine or encyclopedia might. Instead, because the size of a model is much smaller than the size of its training data, it learns by compressing. The model becomes a blurry picture of its training data, retaining key features but at much lower resolution. Some facts resist blurring—“Paris”, for example, may always be the highest-probability token following the words “The capital of France is”. But many more facts that are less statistically obvious may be smudged away.
Further distortions are possible when a pretrained LLM is “fine-tuned”. This is a later stage of training in which the model’s weights, which encode statistical relationships between the words and phrases in the training data, are updated for a specific task. Hallucinations can increase if the LLM is fine-tuned, for example, on transcripts of conversations, because the model might make things up to try to be interesting, just as a chatty human might. (Simply including fine-tuning examples where the model says “I don’t know” seems to keep hallucination levels down.)
Tinkering with a model’s weights can reduce hallucinations. One method involves creating a deliberately flawed model trained on data that contradict the prompt or contain information it lacks. Researchers can then subtract the weights of the flawed model, which are in part responsible for its output, from those of the original to create a model which hallucinates less.
It is also possible to change a model’s “temperature”. Lower temperatures make a model more conservative, encouraging it to sample the most likely word. Higher temperatures make it more creative, by increasing the randomness of this selection. If the goal is to reduce hallucinations, the temperature should be set to zero. Another trick is to limit the choice to the top-ranked tokens alone. This reduces the likelihood of poor responses, while also allowing for some randomness and, therefore, variety.
Clever prompting can also reduce hallucinations. Researchers at Google DeepMind found that telling an LLM to “take a deep breath and work on this problem step-by-step” reduced hallucinations and improved problem solving, especially of maths problems. One theory for why this works is that AI models learn patterns. By breaking a problem down into smaller ones, it is more likely that the model will be able to recognise and apply the right one. But, says Edoardo Ponti at the University of Edinburgh, such prompt engineering amounts to treating a symptom, rather than curing the disease.
Perhaps, then, the problem is that accuracy is too much to ask of LLMs alone. Instead, they should be part of a larger system—an engine, rather than the whole car. One solution is retrieval augmented generation (RAG), which splits the job of the AI model into two parts: retrieval and generation. Once a prompt is received, a retriever model bustles around an external source of information, like a newspaper archive, to extract relevant contextual information. This is fed to the generator model alongside the original prompt, prefaced with instructions not to rely on prior knowledge. The generator then acts like a normal LLM and answers. This reduces hallucinations by letting the LLM play to its strengths—summarising and paraphrasing rather than researching. Other external tools, from calculators to search engines, can also be bolted onto an LLM in this way, effectively building it a support system to enhance those skills it lacks.
Even with the best algorithmic and architectural antipsychotics available, however, LLMs still hallucinate. One leaderboard, run by Vectara, an American software company, tracks how often such errors arise. Its data shows that GPT-4 still hallucinates in 3% of its summaries, Claude 2 in 8.5% and Gemini Pro in 4.8%. This has prompted programmers to try detecting, rather than preventing, hallucinations. One clue that a hallucination is under way lies in how an LLM picks words. If the probability distribution of the words is flat, ie many words have similar likelihoods of being chosen, this means that there is less certainty as to which is most likely. That is a clue that it might be guessing, rather than using information it has been prompted with and therefore “knows” to be true.
Another way to detect hallucination is to train a second LLM to fact-check the first. The fact-checker can be given the “ground truth” along with the LLM’s response, and asked whether or not they agree. Alternatively, the fact-checker can be given several versions of the LLM’s answer to the same question, and asked whether they are all consistent. If not, it is more likely to be a hallucination. NVIDIA, a chipmaker, has developed an open-source framework for building guardrails that sit around an LLM to make it more reliable. One of these aims to prevent hallucinations by deploying this fact-checking when needed.
Although such approaches can decrease the hallucination rate, says Ece Kamar, head of the AI frontiers lab at Microsoft, “it is unclear whether any of these techniques is going to completely get rid of hallucinations.” In many cases, that would be akin to self-sabotage. If an LLM is asked to generate ideas for a fantasy novel, for example, its output would be disappointing if limited to the world as it is. Consequently, says Dr Kamar, her research aims not to get rid of all hallucinations, but rather to stop the model from hallucinating when it would be unhelpful.
Safe and warm
The hallucination problem is one facet of the larger “alignment” problem in the field of AI: how do you get AI systems to reliably do what their human users intend and nothing else? Many researchers believe the answer will come in training bigger LLMs on more and better data. Others believe that LLMs, as generative and probabilistic models, will never be completely rid of unwanted hallucinations.
Or, the real problem might be not with the models but with its human users. Producing language used to be a uniquely human capability. LLMs’ convincing textual outputs make it all too easy to anthropomorphise them, to assume that LLMs also operate, reason and understand like humans do. There is still no conclusive evidence that this is the case. LLMs do not learn self-consistent models of the world. And even as models improve and the outputs become more aligned with what humans produce and expect, it is not clear that the insides will become any more human. Any successful real-world deployment of these models will probably require training humans how to use and view AI models as much as it will require training the models themselves.
Curious about the world? To enjoy our mind-expanding science coverage, sign up to Simply Science, our weekly subscriber-only newsletter.
© 2023, The Economist Newspaper Limited. All rights reserved. From The Economist, published under licence. The original content can be found on www.economist.com
-
others1 week ago
Here’s What Bitcoin Needs To Do To Confirm Bullish Breakout, According to Trader Who Nailed 2024 BTC Correction – Crypto News
-
Blockchain7 days ago
XRP MVRV Ratio Dips Below The 200-Day MA – Trend Shift Underway? – Crypto News
-
others6 days ago
Bybit Shuts Down Its NFT Marketplace As Crypto Sector Struggles To Recover – Crypto News
-
Business1 week ago
Can Pi Coin Price Hit $1 Soon? – Crypto News
-
others1 week ago
Here’s What Bitcoin Needs To Do To Confirm Bullish Breakout, According to Trader Who Nailed 2024 BTC Correction – Crypto News
-
others1 week ago
Here’s What Bitcoin Needs To Do To Confirm Bullish Breakout, According to Trader Who Nailed 2024 BTC Correction – Crypto News
-
others1 week ago
XRP and Three Other Altcoins Could Witness Another Sell-Off Event, According to Crypto Strategist – Crypto News
-
Cryptocurrency6 days ago
Sony Singapore Now Lets Shoppers Pay in USDC Through Crypto.com – Crypto News
-
Blockchain6 days ago
Dogecoin Breaking These Levels Could Be The Catalyst For Next Bull Run, Analyst Says – Crypto News
-
Business1 week ago
XRP, BTC, ETH Price Prediction As Inflation Data Sparks Downturn in U.S. Stocks – Crypto News
-
Blockchain1 week ago
zkLend hacker claims losing stolen ETH to Tornado Cash phishing site – Crypto News
-
others1 week ago
Gold extends bullish trend amid rising trade tensions; fresh record high and counting – Crypto News
-
Blockchain1 week ago
SpaceX flight bankrolled by crypto investor launches first manned polar orbit – Crypto News
-
Blockchain1 week ago
Will Bitcoin Downtrend Continue? This Metric Suggests Yes – Crypto News
-
Blockchain1 week ago
Binance ends Tether USDT trading in Europe to comply with MiCA rules – Crypto News
-
Blockchain1 week ago
Binance ends Tether USDT trading in Europe to comply with MiCA rules – Crypto News
-
others1 week ago
Austria Unemployment fell from previous 347.4K to 316.3K in March – Crypto News
-
Blockchain7 days ago
Analyst Calls Dogecoin Chart A ‘Beauty’ As Key Indicators Align – Crypto News
-
Technology6 days ago
XRP Price Predicted to Reach $10 in April if US Congress Stablecoin Bill Promotes Ripple’s RLUSD – Crypto News
-
Business6 days ago
XRP Price Predicted to Reach $10 in April if US Congress Stablecoin Bill Promotes Ripple’s RLUSD – Crypto News
-
Technology6 days ago
Apple Intelligence debuts on Vision Pro with visionOS 2.4 update: AI-powered features, spatial content and more – Crypto News
-
Cryptocurrency6 days ago
US equities slip after job openings disappointment – Crypto News
-
others6 days ago
Will BNB Price Rally to ATH After VanEck BNB ETF Filing? – Crypto News
-
others6 days ago
Pound Sterling consolidates against US Dollar ahead of Trump’s tariffs announcement – Crypto News
-
Cryptocurrency6 days ago
FLOKI price poised for 20% rally, Here’s why – Crypto News
-
others6 days ago
PENDLE Price Jumps 8% Today Amid Huge Whale Accumulation – Crypto News
-
others6 days ago
Fundstrat’s Tom Lee Calls for Imminent Stock Market Reversal, Says US Has the ‘Right Pieces’ for a Bottom – Crypto News
-
Technology6 days ago
European regulators warn of financial risks from US crypto integration – Crypto News
-
Cryptocurrency1 week ago
Is Bitcoin’s rebound near as key area rises? Assessing… – Crypto News
-
Business1 week ago
Builder.ai Announces Third-Party Audit After Allegations – Crypto News
-
others1 week ago
Will the RBA hint at further interest rate hikes at its policy meeting? – Crypto News
-
Technology1 week ago
Over 60 pc broadband, fiber, DSL users surveyed flag problems with connection: LocalCircles poll – Crypto News
-
Cryptocurrency1 week ago
Tether Boosts Bitcoin Holdings By 8,888 BTC In Q1 2025 – Crypto News
-
Technology1 week ago
Bumper discounts in Amazon Gaming Fest! Up to 70% off on gaming laptops, monitors, vlog cameras and more – Crypto News
-
Blockchain1 week ago
Hayes Predicts $250,000 Bitcoin As Fed Caves To QE Pressure – Crypto News
-
Cryptocurrency1 week ago
XRP, DOGE Shoot up as BTC Price Reclaims $84K Level (Market Watch) – Crypto News
-
Technology7 days ago
Best washing machines under ₹10000 in April 2025 to boost your laundry routine without overspending – Crypto News
-
Blockchain6 days ago
Bitcoin traders are overstating the impact of the US-led tariff war on BTC price – Crypto News
-
Blockchain6 days ago
Bitcoin Price Bounces Back—Can It Finally Break Resistance? – Crypto News
-
Cryptocurrency6 days ago
US equities slip after job openings disappointment – Crypto News
-
Blockchain6 days ago
UK trade bodies ask government to make crypto a ‘strategic priority’ – Crypto News
-
Blockchain6 days ago
Several Altcoins Crash Up To 50% On Binance, What’s Going On? – Crypto News
-
Business6 days ago
What to Expect From XRP Price as Trump’s ‘Liberation Day’ Tariffs Go Into Effect Today – Crypto News
-
Technology6 days ago
Franklin Templeton Eyes Crypto ETP Launch In Europe After BlackRock & 21Shares – Crypto News
-
Technology1 week ago
ChatGPT finally allows free users to create Ghibli-style AI images: Check our step-by-step guide – Crypto News
-
Technology1 week ago
Elon Musk’s Grok AI calls him ‘top misinformation spreader’, sparks debate on ‘AI freedom’ – Crypto News
-
Cryptocurrency1 week ago
Will the $80K Support Level Hold BTC After Recent Rejection? – Crypto News
-
Metaverse1 week ago
Mint Primer | Resistance is futile: AI is now writing code – Crypto News
-
Metaverse1 week ago
The Tools of Tomorrow: What Lies Ahead with the AI Revolution – Crypto News
-
Business1 week ago
MicroStrategy Acquires 22,048 Bitcoin For $1.92 Billion – Crypto News