<\/p>\n
WHEN SAM ALTMAN, boss of OpenAI, posted a gnomic tweet this month saying \u201cThere is no wall,\u201d his followers on X, a social-media site, had a blast. \u201cTrump will build it,\u201d said one. \u201cNo paywall for ChatGPT?\u201d quipped another. It has since morphed from an in-joke among nerds into a serious business matter.<\/p>\n<\/div>\n
The wall in question refers to the view that the forces underlying improvements in generative artificial intelligence (AI) over the past 15 years have reached a limit. Those forces are known as scaling laws. \u201cThere\u2019s a lot of debate: have we hit the wall with scaling laws?\u201d Satya Nadella, Microsoft\u2019s boss, asked at his firm\u2019s annual conference on November 19th. A day later Jensen Huang, boss of Nvidia, the world\u2019s most valuable company, said no.<\/p>\n<\/div>\n
Scaling laws are not physical laws. Like Moore\u2019s law, the observation that processing performance for semiconductors doubles roughly every two years, they reflect the perception that AI performance in recent years has doubled every six months or so. The main reason for that progress has been the increase in the computing power that is used to train large language models (LLMs). No company\u2019s fortunes are more intertwined with scaling laws than Nvidia, whose graphics processing units (GPUs) provide almost all of that computational oomph.<\/p>\n<\/div>\n
On November 20th, during Nvidia\u2019s results presentation, Mr Huang defended scaling laws. He also told The Economist that the first task of Nvidia\u2019s newest class of GPUs, known as Blackwells, would be to train a new, more powerful generation of models. \u201cIt\u2019s so urgent for all these foundation-model-makers to race to the next level,\u201d he says.<\/p>\n<\/div>\n
The results for Nvidia\u2019s quarter ending in October reinforced the sense of upward momentum. Although the pace of growth has slowed somewhat, its revenue exceeded $35bn, up by a still-blistering 94%, year on year (see chart). And Nvidia projected another $37.5bn in revenues for this quarter, above Wall Street\u2019s expectations. It said the upward revision was partly because it expected demand for Blackwell GPUs to be higher than it had previously thought. Mr Huang predicted 100,000 Blackwells would be swiftly put to work training and running the next generation of LLMs.<\/p>\n<\/div>\n Not everyone shares his optimism. Scaling-law sceptics note that OpenAI has not yet produced a new general-purpose model to replace GPT-4, which has underpinned ChatGPT since March 2023. They say Google\u2019s Gemini is underwhelming given the money it has spent on it.<\/p>\n<\/div>\n But, as Mr Huang notes, scaling laws not only apply to the initial training of LLMs, but also to the use of the model, or inference, especially when complex reasoning tasks are involved. To explain why, he points to OpenAI\u2019s latest model, o1, which has stronger reasoning capabilities than GPT-4. It can do advanced maths and other complex tasks by taking a step-by-step approach that its maker calls \u201cthinking\u201d. This enhanced inference process uses far more computing power than a typical ChatGPT response, Mr Huang says. \u201cWe know that we need more compute whatever the approach is,\u201d he says.<\/p>\n<\/div>\n The more AI is adopted,the more important inference will become. Mr Huang says that Nvidia\u2019s previous generations of GPUs can be used for inference, but that Blackwells will make performance dozens of times better. Already at least half of Nvidia\u2019s infrastructure is used for inference.<\/p>\n<\/div>\n Mr Huang clearly has an interest in portraying scaling in the best possible light. Some sceptics question how meaningful the advances in reasoning are. Although a handful of business models are being disrupted, many firms are struggling to adopt AI at scale, which could eventually weigh on demand for the technology. These are early days, though. Tech giants continue to spend big on GPUs, and Mr Huang points out that new technologies take time to digest. Nvidia\u2019s back is not against the wall yet.<\/p>\n<\/div>\n \u00a9 2025, The Economist Newspaper Ltd. All rights reserved. From The Economist, published under licence. The original content can be found on www.economist.com<\/i><\/p>\n<\/div>\n<\/div>\n Catch all the Business News<\/a>, Market News<\/a>, Breaking News<\/a> Events and Latest News<\/a> Updates on Live Mint. Download The Mint News App<\/a> to get Daily Market Updates.<\/p>\n More<\/span>Less<\/span><\/div>\n","protected":false},"excerpt":{"rendered":" WHEN SAM ALTMAN, boss of OpenAI, posted a gnomic tweet this month saying \u201cThere is no wall,\u201d his followers on X, a social-media site, had a blast. \u201cTrump will build it,\u201d said one. \u201cNo paywall for ChatGPT?\u201d quipped another. It has since morphed from an in-joke among nerds into a serious business matter. The wall […]<\/p>\n","protected":false},"author":2,"featured_media":360610,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[27774,27775,284,263,262,27776,260,259,258,11938,27773,265,27777,202,3906,12705,5792,8204,261,27778,264],"class_list":["post-360604","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-metaverse","tag-ai-has-hit-the-wall","tag-ai-scaling-laws","tag-artificial-intelligence","tag-axie-infinity","tag-axs","tag-blackwell-gpus","tag-decentraland","tag-facebook","tag-game","tag-generative-artificial-intelligence","tag-jensen-huang-of-nvidia","tag-mark-zuckerberg","tag-next-generation-of-llms","tag-nft","tag-nvidia","tag-nvidia-gpus","tag-openai","tag-sam-altman","tag-sandbox","tag-training-of-llms","tag-vr"],"_links":{"self":[{"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/posts\/360604","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/comments?post=360604"}],"version-history":[{"count":1,"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/posts\/360604\/revisions"}],"predecessor-version":[{"id":360608,"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/posts\/360604\/revisions\/360608"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/media\/360610"}],"wp:attachment":[{"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/media?parent=360604"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/categories?post=360604"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dripp.zone\/news\/wp-json\/wp\/v2\/tags?post=360604"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}