

Metaverse
AI models make stuff up. How can hallucinations be controlled? – Crypto News
There are kinder ways to put it. In its instructions to users, OpenAI warns that ChatGPT “can make mistakes”. Anthropic, an American AI company, says that its LLM Claude “may display incorrect or harmful information”; Google’s Gemini warns users to “double-check its responses”. The throughline is this: no matter how fluent and confident AI-generated text sounds, it still cannot be trusted.
Hallucinations make it hard to rely on AI systems in the real world. Mistakes in news-generating algorithms can spread misinformation. Image generators can produce art that infringes on copyright, even when told not to. Customer-service chatbots can promise refunds they shouldn’t. (In 2022 Air Canada’s chatbot concocted a bereavement policy, and this February a Canadian court has confirmed that the airline must foot the bill.) And hallucinations in AI systems that are used for diagnosis or prescription can kill.
All the leaves are brown
The trouble is that the same abilities that allow models to hallucinate are also what make them so useful. For one, LLMs are a form of “generative” AI, which, taken literally, means they make things up to solve new problems. They do this by producing probability distributions for chunks of characters, or tokens, laying out how likely it is for each possible token in its vocabulary to come next. The mathematics dictate that each token must have a non-zero chance of being chosen, giving the model flexibility to learn new patterns, as well as the capacity to generate statements that are incorrect. The fundamental problem is that language models are probabilistic, while truth is not.
This tension manifests itself in a number of ways. One is that LLMs are not built to have perfect recall in the way a search engine or encyclopedia might. Instead, because the size of a model is much smaller than the size of its training data, it learns by compressing. The model becomes a blurry picture of its training data, retaining key features but at much lower resolution. Some facts resist blurring—“Paris”, for example, may always be the highest-probability token following the words “The capital of France is”. But many more facts that are less statistically obvious may be smudged away.
Further distortions are possible when a pretrained LLM is “fine-tuned”. This is a later stage of training in which the model’s weights, which encode statistical relationships between the words and phrases in the training data, are updated for a specific task. Hallucinations can increase if the LLM is fine-tuned, for example, on transcripts of conversations, because the model might make things up to try to be interesting, just as a chatty human might. (Simply including fine-tuning examples where the model says “I don’t know” seems to keep hallucination levels down.)
Tinkering with a model’s weights can reduce hallucinations. One method involves creating a deliberately flawed model trained on data that contradict the prompt or contain information it lacks. Researchers can then subtract the weights of the flawed model, which are in part responsible for its output, from those of the original to create a model which hallucinates less.
It is also possible to change a model’s “temperature”. Lower temperatures make a model more conservative, encouraging it to sample the most likely word. Higher temperatures make it more creative, by increasing the randomness of this selection. If the goal is to reduce hallucinations, the temperature should be set to zero. Another trick is to limit the choice to the top-ranked tokens alone. This reduces the likelihood of poor responses, while also allowing for some randomness and, therefore, variety.
Clever prompting can also reduce hallucinations. Researchers at Google DeepMind found that telling an LLM to “take a deep breath and work on this problem step-by-step” reduced hallucinations and improved problem solving, especially of maths problems. One theory for why this works is that AI models learn patterns. By breaking a problem down into smaller ones, it is more likely that the model will be able to recognise and apply the right one. But, says Edoardo Ponti at the University of Edinburgh, such prompt engineering amounts to treating a symptom, rather than curing the disease.
Perhaps, then, the problem is that accuracy is too much to ask of LLMs alone. Instead, they should be part of a larger system—an engine, rather than the whole car. One solution is retrieval augmented generation (RAG), which splits the job of the AI model into two parts: retrieval and generation. Once a prompt is received, a retriever model bustles around an external source of information, like a newspaper archive, to extract relevant contextual information. This is fed to the generator model alongside the original prompt, prefaced with instructions not to rely on prior knowledge. The generator then acts like a normal LLM and answers. This reduces hallucinations by letting the LLM play to its strengths—summarising and paraphrasing rather than researching. Other external tools, from calculators to search engines, can also be bolted onto an LLM in this way, effectively building it a support system to enhance those skills it lacks.
Even with the best algorithmic and architectural antipsychotics available, however, LLMs still hallucinate. One leaderboard, run by Vectara, an American software company, tracks how often such errors arise. Its data shows that GPT-4 still hallucinates in 3% of its summaries, Claude 2 in 8.5% and Gemini Pro in 4.8%. This has prompted programmers to try detecting, rather than preventing, hallucinations. One clue that a hallucination is under way lies in how an LLM picks words. If the probability distribution of the words is flat, ie many words have similar likelihoods of being chosen, this means that there is less certainty as to which is most likely. That is a clue that it might be guessing, rather than using information it has been prompted with and therefore “knows” to be true.
Another way to detect hallucination is to train a second LLM to fact-check the first. The fact-checker can be given the “ground truth” along with the LLM’s response, and asked whether or not they agree. Alternatively, the fact-checker can be given several versions of the LLM’s answer to the same question, and asked whether they are all consistent. If not, it is more likely to be a hallucination. NVIDIA, a chipmaker, has developed an open-source framework for building guardrails that sit around an LLM to make it more reliable. One of these aims to prevent hallucinations by deploying this fact-checking when needed.
Although such approaches can decrease the hallucination rate, says Ece Kamar, head of the AI frontiers lab at Microsoft, “it is unclear whether any of these techniques is going to completely get rid of hallucinations.” In many cases, that would be akin to self-sabotage. If an LLM is asked to generate ideas for a fantasy novel, for example, its output would be disappointing if limited to the world as it is. Consequently, says Dr Kamar, her research aims not to get rid of all hallucinations, but rather to stop the model from hallucinating when it would be unhelpful.
Safe and warm
The hallucination problem is one facet of the larger “alignment” problem in the field of AI: how do you get AI systems to reliably do what their human users intend and nothing else? Many researchers believe the answer will come in training bigger LLMs on more and better data. Others believe that LLMs, as generative and probabilistic models, will never be completely rid of unwanted hallucinations.
Or, the real problem might be not with the models but with its human users. Producing language used to be a uniquely human capability. LLMs’ convincing textual outputs make it all too easy to anthropomorphise them, to assume that LLMs also operate, reason and understand like humans do. There is still no conclusive evidence that this is the case. LLMs do not learn self-consistent models of the world. And even as models improve and the outputs become more aligned with what humans produce and expect, it is not clear that the insides will become any more human. Any successful real-world deployment of these models will probably require training humans how to use and view AI models as much as it will require training the models themselves.
Curious about the world? To enjoy our mind-expanding science coverage, sign up to Simply Science, our weekly subscriber-only newsletter.
© 2023, The Economist Newspaper Limited. All rights reserved. From The Economist, published under licence. The original content can be found on www.economist.com
-
Blockchain6 days ago
Institutional Demand Surges As Ethereum Sets New Inflow Records – Crypto News
-
Blockchain5 days ago
DeFi Development Nears 1 Million Solana In Treasury – Crypto News
-
Business1 week ago
XLM Is More Bullish Than ETH, SOL, And XRP, Peter Brandt Declares – Crypto News
-
Cryptocurrency1 week ago
Anarchy, crime and stablecoins – Blockworks – Crypto News
-
Cryptocurrency1 week ago
Altseason heats up, but Bitcoin could face short-term pullback – How? – Crypto News
-
Cryptocurrency1 week ago
Bitcoin trades near $119K after new all-time high; Coinbase rebrands wallet to ‘Base App’ – Crypto News
-
Technology1 week ago
“Decentralized Ponzi Scheme”- Gold Bug Peter Schiff Slams Landmark Crypto Bills – Crypto News
-
Cryptocurrency1 week ago
California Sheriffs Believe 74-Year-Old’s Disappearance Linked to Son’s Crypto Fortune – Crypto News
-
Business6 days ago
Vitalik Buterin Approves Gas Limit Hike, Warns Against Risky Ethereum Scaling – Crypto News
-
Blockchain5 days ago
Strategy to keep STRC Fund Pegged to $100 – Crypto News
-
Metaverse1 week ago
Zoho Zia LLM launched with speech-to-text models and AI agent marketplace: All you need to know – Crypto News
-
De-fi1 week ago
BNB Chain Teases New Blockchain with Privacy Features to Compete With Crypto Exchanges – Crypto News
-
Technology1 week ago
Breaking: GENIUS Act Becomes First Major Crypto Legislation as Trump Signs Bill – Crypto News
-
De-fi1 week ago
Crypto Market Cap Hits $4 Trillion Milestone as US House Passes Landmark Bills – Crypto News
-
Cryptocurrency1 week ago
Shytoshi Kusama Breaks Silence on New SHIB AI Whitepaper and Transformed Future – Crypto News
-
Cryptocurrency7 days ago
Sanctum acquires Ironforge, plots transaction infrastructure vertical – Crypto News
-
Cryptocurrency6 days ago
XRP Price Hits All-Time High at $3.66 — Can It Smash Through $4 After Trump Win & SEC Shake-Up? – Crypto News
-
Blockchain6 days ago
How to Use Google Gemini to Turn Crypto News Into Trade Signals – Crypto News
-
others6 days ago
EUR/CHF rises on speculation of SNB intervention, but EU–US trade risks cap gains – Crypto News
-
others1 week ago
GBP/USD rallies on US PPI dip and Trump’s potential Powell removal – Crypto News
-
others1 week ago
GBP/USD rallies on US PPI dip and Trump’s potential Powell removal – Crypto News
-
Cryptocurrency1 week ago
Russia’s $85 Billion Sberbank to Launch Crypto Custody Services – Crypto News
-
Technology1 week ago
Europe’s answer to ChatGPT? Mistral adds voice and research features to Le Chat AI – Crypto News
-
De-fi1 week ago
U.S. House Passes Clarity, GENIUS, and Anti-CBDC Acts With Historic Bipartisan Support for Crypto – Crypto News
-
Technology1 week ago
Malicious code found in fake coding extensions used to steal crypto – Crypto News
-
Cryptocurrency1 week ago
XRP Price Spikes to Record Highs As Momentum Signals Extended Gains – Crypto News
-
Technology1 week ago
Meta’s AI Studio: Red flag or red herring? – Crypto News
-
Blockchain1 week ago
Why Bitcoin self-custody is declining in the ETF era – Crypto News
-
Cryptocurrency1 week ago
US House passes three key crypto bills; market reaction muted as Bitcoin dips – Crypto News
-
De-fi1 week ago
Crypto Market Cap Hits $4 Trillion Milestone as US House Passes Landmark Bills – Crypto News
-
others1 week ago
Streaming Service Handing $3,400,000 To Current and Former Customers To Settle Illegal Data Harvesting Allegations – Crypto News
-
Cryptocurrency7 days ago
Arthur Hayes-linked wallet bags $2M worth of AAVE and LDO in an OTC deal – Crypto News
-
others7 days ago
Why Is The Crypto Market Rising Today? – Crypto News
-
others6 days ago
Breaking: Polymarket Reenters US Market With Exchange Acquisition As Probe Ends – Crypto News
-
Blockchain6 days ago
To The Moon? Justin Sun To Be Launched Into Space After $28M Bid – Crypto News
-
others6 days ago
Venture Capital Firms Launch $360,000,000 Crypto Treasury Company Focused on Arthur Hayes-Backed Ethena (ENA) – Crypto News
-
Cryptocurrency5 days ago
Ethereum Shatters Inflow Records, Pulls in $2.12 Billion in a Week – Crypto News
-
Cryptocurrency5 days ago
Solana Clinches 5-Month High, Where to From Here? – Crypto News
-
Technology5 days ago
BitOrigin Begins $500M Dogecoin Treasury With 40.5M Buy, Analysts Predict Fresh Bull Cycle – Crypto News
-
Technology5 days ago
Grab up to 43% off on best selling premium laptops from Apple, Asus and more – Crypto News
-
Blockchain5 days ago
XRP Could Skyrocket 500% Against Bitcoin, Analyst Warns – Crypto News
-
Blockchain5 days ago
Crypto Needs Minimum Viable Decentralization – Crypto News
-
Blockchain1 week ago
Nasdaq Exchange Files SEC Form to List Staking Ethereum ETF – Crypto News
-
Cryptocurrency1 week ago
Ethereum price surges 6% to $2,800 as shorts suffer amid $500M crypto liquidation – Crypto News
-
Technology1 week ago
OnePlus Pad 3 with Snapdragon 8 Elite SoC makes its India debut, set to go on first sale in September – Crypto News
-
Cryptocurrency1 week ago
Friday charts: Fiscal dominance and super intelligence – Crypto News
-
De-fi1 week ago
Trump’s Crypto Assets Now Comprise a Key Part of Family Fortune Worth Billions – Crypto News
-
Business1 week ago
Pi Coin Price Technical Analysis Confirms Buy Signal Despite 2M Exchange Inflows – Crypto News
-
Technology1 week ago
Not Google or Bing! This search engine lets you block AI images in search results – Crypto News
-
Cryptocurrency1 week ago
GENIUS Act Is The Catalyst For XRP And RLUSD’s Dominance, Expert Declares – Crypto News