Companies are scrambling to curtail soaring AI costs – Crypto News – Crypto News
Connect with us
Companies are scrambling to curtail soaring AI costs Companies are scrambling to curtail soaring AI costs

Metaverse

Companies are scrambling to curtail soaring AI costs – Crypto News

Published

1 month ago

on

“It’s going to be an absolute nightmare,” says an executive at a big American tech company. He is talking about an emerging problem for businesses that use artificial intelligence. AI agents—bots that can read, interpret and act—use masses of processing power and have started to run up huge bills. As they proliferate, the problem will grow. Big companies, the executive points out, typically use hundreds of software programs. If each of those offer agents (as they probably will), AI costs could easily spiral out of control.

Budget management is a new worry for AI adopters. Not long ago employees were encouraged to binge on the technology, as bosses and investors saw spending as a sign of innovation. Burning through vast numbers of tokens—the chunks of text that models process, which are often used as a unit of pricing—became a badge of honour; techies dubbed it “tokenmaxxing”. Companies showed off staff’s AI use on internal leaderboards. Meta’s display awarded top users titles like “Token Legend”.

Such incentives partly explain the boom in AI spending. Another contributing factor is a change in the way enterprises use the technology. Token-heavy applications, such as reasoning models and agents, are growing more popular. In some cases agents build their own agents, sending costs higher still. Ramp, a corporate-credit-card provider, analyses its clients’ transaction data to shed light on how they use AI. It reckons their overall spending has risen 13-fold in the past year. In April Uber said that it had already spent its annual AI budget in four months. Other firms are experiencing similar problems. One reportedly spent $500m on AI tokens in a month. Sam Altman, the boss of OpenAI, has described mounting customer costs as “a huge issue”.

For now, the problem is concentrated. The top spenders tend to be tech firms, because they are early adopters of the technology and because AI is particularly good at writing software. Ramp reckons that the 1% of clients that spend the most on AI per employee are racking up bills of about $7,450 per person per month on average. That compares with just $11 for the median Ramp customer. And although the big spenders’ AI bills are low compared with the cost of hiring a developer in San Francisco, they are high compared with employing one in Delhi.

Companies are responding in various ways. Although AI laggards are still racing to adopt the technology, for the heaviest users tokenmaxxing is out. Some, including Meta and Amazon, have scrapped their leaderboards. Many are thinking more carefully about model choice. Plenty of tasks don’t require pricey, cutting-edge models. Aran Khanna of Archera.ai, which helps companies reduce their cloud costs, points out that in some cases Sonnet, a lagging-edge model from Anthropic, can cost a twentieth of what Opus, a leading-edge one, does. And Kimi, an open-source model from Moonshot AI, a Chinese startup, can cost a twentieth of that.

Another approach is spending caps. Uber has limited its employees to $1,500-worth of tokens each month per coding tool. How firms choose to allocate tokens depends on where they can generate the most value from AI, says Rachel Laycock of Thoughtworks, a consultancy. That often means companies apportion the most tokens to their core business function, such as engineers in a tech firm, she adds.

Some software vendors that incorporate AI into their products are experimenting with new pricing plans, such as outcome-based ones, in order to ease concern among clients over costs. Intercom, a software provider, offers a service whereby the customer pays only for queries that are resolved by its IT-support agent. The big cloud providers have launched cost-management services, too, including budgeting tools and ways to triage queries to the most appropriate model.

For the model-makers themselves, the balancing act is a trickier one. They want their biggest customers to use as many tokens as possible without balking at the cost. Today the cost of providing AI services is subsidised by the labs. Indeed, OpenAI’s plan for winning customers from Anthropic reportedly involves drastic price cuts. But eventually the model-makers will need to turn a profit, which will mean raising prices. Pressure to do so will only intensify later this year if, as expected, Anthropic and OpenAI go public. Their customers, then, can look forward to more jaw-dropping AI bills.

Related Topics:
Continue Reading

Trending