
Metaverse
‘We want to fix the language gap in AI language models’, says Two Platforms’ Pranav Mistry – Crypto News
Backed by billionaire Mukesh Ambani’s Jio Platforms and South Korea’s Naver Corp, the artificial reality startup also plans to soon release an artificial intelligence (AI)-powered messaging and social app called Zappy in India, according to Two Platform’s founder and CEO, Pranav Mistry.
“Sutra is our mission to fix the language gap in AI language models. We are committed to pioneering AI solutions for non-English markets. We believe our Sutra models will unlock AI growth opportunities in large economies such as India, Korea, Japan, and the MEA (Middle East and Africa) region,” Mistry said in an interview with Mint.
But there are some basic differences “in our approach to building these models”, he insisted. First, unlike most other startups and companies that are building ‘local’ or ‘Indic’ LLMs for India by fine-tuning global LLMs, “we have built a foundational, and not a fine-tuned model,” he said.
General-purpose foundational models such as Google’s BERT and Gemini, OpenAI’s generative pre-trained transformer (GPT) variants, and Meta’s LlaMA series, have been pre-trained on humungous amounts of data from the internet, books, media articles, and other sources. But most of this training data is in English.
A Transformational Approach
Most companies in India are building their Indic LLMs atop these foundational models (hence they’re called ‘wrappers’) by fine tuning these general-purpose LLMs on a smaller, task-specific dataset (such as regional languages like Hindi, Marathi, Gujarati, Tamil, Telegu, Malayalam, etc., and their dialects), which allows the models to learn the nuances of the language and improves its performance.
Sutra, instead, uses two different transformer architectures. Developed by Google, transformers predict the next word in a sequence of text based on large, complex data sets. Since they process words in a single sequence while understanding their relationships with each other, transformers are very effective for tasks like translating languages.
The multilingual LLM Sutra, according to Mistry, has combined an LLM architecture with a Neural Machine Translation (NMT) one. The reason: while LLMs may struggle due to the lack of specialized training data while translating specific pairs of language, NMT systems are typically better equipped to translate idiomatic expressions and colloquial language.
Second, while “GPT-4 is great in Korean or Hindi, too, its size and cost make it more expensive for a country like India”, argued Mistry. The Sutra architecture “decouples concept learning (we learn concepts by associating new information with existing knowledge, such as learning that both apples and oranges are fruits) from language learning. So, when you use Sutra, the number of the tokens used are similar to using English tokens. This saves almost five to eight times in costs,” he explained.
Third, “our specialized NMT models are significantly smaller in parameter size, requiring much less data for training”, Mistry said. When you add more data, say Korean or some Indian language, you also increase the tokens (loosely, pieces of words and sub-words that an LLM can understand. For example, banana is a word, while homework can be split into two words, home and work). This makes the model bigger, but also slows it down. It increases the costs, too, since similar information content in English, when expressed in a language such as Hindi would need three to four times more tokens.
“Besides, in this approach, the quality of, say Hindi, can never surpass that of English in the original,” Mistry added. For instance, about 80% of a general-purpose foundational model pre-training would typically be from sources such as the internet, books, and media articles, which are mostly in English.
Innovation, Not Fine-tuning
However, if you’re fine-tuning this model with data in Hindi from India, for instance, “most of the data would be about cricket, data found on Twitter, or from people discussing news articles, etc., in Hindi. Hence, a Hindi language model built atop a foundational model that has pre-trained mostly on English will not be able to do full justice to the output in Hindi”.
“As an example, if you want to translate Gujarati to Tamil, most models first translate from Gujarati to English and then from English to Tamil, because that’s the data they have trained on. Our model does not do that, so we also require fewer tokens, which also lowers the cost of running the model,” he explained. Mistry adds that Two Platforms’ model is also aligned to human values, a process technically known as ‘AI alignment’.
Sutra, which is currently available in three versions—Light (56 billion parameters), Online (internet-connected multilingual model with 56 billion parameters), and Pro (150 billion parameters)—supports more than 50 languages, “of which 31 are fully tested”, according to Mistry. He emphasized that Sutra’s architecture and use of “synthetically translated data” not only lowers the computing costs of running these models, but also makes the model more efficient.
“Sutra maintains an impressive performance in English of 77% on the MMLU (massive multitask language understanding) benchmark. It also demonstrates superior and consistent performance in the range of 65-75% across languages. In contrast, many leading language models score closer to 25% on non-English MMLU tasks,” Mistry said.
Two Platforms uses its “in-house GPU (graphics processing unit) cluster and rents top-tier cloud GPUs when needed”. “As we expand, the rising costs of training will require us to create specialized models for different areas like images and video,” Mistry added. His company is also in the process of raising a Series A round “to accelerate the development of Sutra into a model-as-a-service (MaaS)” platform. In February 2022, Jio Platforms had invested $15 million in Two Platforms for a 25% equity stake, while a Naver Corp unit, Snow Corp., had invested $5 million.
Other than Sutra, India has Sarvam AI—a generative AI (GenAI) startup that has launched the Open Hathi series; Tech Mahindra’s Indus Project; the ‘Hanooman’ model that was jointly released this month by SML India and 3AI Holding, an Abu Dhabi-based investment firm; CoRover’s BharatGPT LLM-based chatbot; and Ola Cabs and Ola Electric co-founder Bhavish Aggarwal’s Krutrim AI. Meanwhile, the ‘Nilekani Center at AI4Bharat’ at IIT Madras, too, released ‘Airavata’ an open-source LLM for Indian languages.
In a wider context, the LLM market is projected to grow from $6.4 billion in 2024 to $36.1 billion by 2030, according to a research report released by MarketsandMarkets in March. Moreover, India-specific LLMs are certainly the need of the hour but “we need faster, more affordable, multilingual, and energy-efficient LLMs that can bridge the existing market gaps”, concluded Mistry, who hopes Sutra will be one of those companies that “fills this gap”.
-
Blockchain1 week ago
Ripple and Ctrl Alt Team to Support Real Estate Tokenization – Crypto News
-
others1 week ago
EUR/USD recovers with trade talks and Fed independence in focus – Crypto News
-
Technology1 week ago
Fed Rate Cut Odds Surge As Powell’s Future Hangs In The Balance – Crypto News
-
Cryptocurrency1 week ago
It’s a Statement, Says Bitfinex Alpha – Crypto News
-
Metaverse1 week ago
Why voice is emerging as India’s next frontier for AI interaction – Crypto News
-
Technology1 week ago
Fed Rate Cut Odds Surge As Powell’s Future Hangs In The Balance – Crypto News
-
Business1 week ago
XLM Is More Bullish Than ETH, SOL, And XRP, Peter Brandt Declares – Crypto News
-
Cryptocurrency1 week ago
Anarchy, crime and stablecoins – Blockworks – Crypto News
-
others1 week ago
Top Crypto Exchange by Trading Volume Binance Announces Airdrop for New Ethereum (ETH) Ecosystem Altcoin – Crypto News
-
Cryptocurrency1 week ago
Bitcoin trades near $119K after new all-time high; Coinbase rebrands wallet to ‘Base App’ – Crypto News
-
Technology6 days ago
“Decentralized Ponzi Scheme”- Gold Bug Peter Schiff Slams Landmark Crypto Bills – Crypto News
-
Cryptocurrency5 days ago
Altseason heats up, but Bitcoin could face short-term pullback – How? – Crypto News
-
Blockchain4 days ago
Institutional Demand Surges As Ethereum Sets New Inflow Records – Crypto News
-
Cryptocurrency1 week ago
1inch price forecast: 1INCH hits 7-month high after double digit gains – Crypto News
-
Cryptocurrency1 week ago
1inch price forecast: 1INCH hits 7-month high after double digit gains – Crypto News
-
others1 week ago
VanEck Details Key Drivers Boosting Bitcoin Price, Including Corporate Treasury Demand, ETF Flows and More – Crypto News
-
Business1 week ago
XRP Lawsuit Update: Ripple Paid $125M in Cash, Settlement Hinges on Appeal – Crypto News
-
De-fi7 days ago
BNB Chain Teases New Blockchain with Privacy Features to Compete With Crypto Exchanges – Crypto News
-
Technology6 days ago
Breaking: GENIUS Act Becomes First Major Crypto Legislation as Trump Signs Bill – Crypto News
-
Cryptocurrency6 days ago
California Sheriffs Believe 74-Year-Old’s Disappearance Linked to Son’s Crypto Fortune – Crypto News
-
Cryptocurrency5 days ago
Shytoshi Kusama Breaks Silence on New SHIB AI Whitepaper and Transformed Future – Crypto News
-
Cryptocurrency4 days ago
XRP Price Hits All-Time High at $3.66 — Can It Smash Through $4 After Trump Win & SEC Shake-Up? – Crypto News
-
Technology1 week ago
XLM Price Forecast: Why Stellar Lumens May Crash After 80% Rally in Last 7 Days – Crypto News
-
Cryptocurrency1 week ago
Hypercharged Exposure to XRP and Solana Now Available With These Two ETFs – Crypto News
-
Blockchain1 week ago
BitMine Shares Rallied After Peter Thiel Investment. – Crypto News
-
others1 week ago
Scammer Drains $10,000,000 From IRS in International Tax Fraud and Identity Theft Scheme: DOJ – Crypto News
-
Business1 week ago
XRP ETF Still in Play- SEC Commissioner Debunks Rejection Speculations – Crypto News
-
Business1 week ago
Ethereum Price Prediction- Bulls Target $3,700 As ETH Treasury Accumulation Soars – Crypto News
-
others1 week ago
GBP/USD rallies on US PPI dip and Trump’s potential Powell removal – Crypto News
-
others1 week ago
GBP/USD rallies on US PPI dip and Trump’s potential Powell removal – Crypto News
-
Cryptocurrency1 week ago
Russia’s $85 Billion Sberbank to Launch Crypto Custody Services – Crypto News
-
De-fi1 week ago
U.S. House Passes Clarity, GENIUS, and Anti-CBDC Acts With Historic Bipartisan Support for Crypto – Crypto News
-
Cryptocurrency7 days ago
XRP Price Spikes to Record Highs As Momentum Signals Extended Gains – Crypto News
-
Blockchain7 days ago
Why Bitcoin self-custody is declining in the ETF era – Crypto News
-
Cryptocurrency6 days ago
US House passes three key crypto bills; market reaction muted as Bitcoin dips – Crypto News
-
De-fi6 days ago
Crypto Market Cap Hits $4 Trillion Milestone as US House Passes Landmark Bills – Crypto News
-
De-fi6 days ago
Crypto Market Cap Hits $4 Trillion Milestone as US House Passes Landmark Bills – Crypto News
-
others5 days ago
Why Is The Crypto Market Rising Today? – Crypto News
-
Cryptocurrency5 days ago
Sanctum acquires Ironforge, plots transaction infrastructure vertical – Crypto News
-
Cryptocurrency1 week ago
Exclusive: Bitwise to roll out daily attestations for bitcoin ETFs – Crypto News
-
Metaverse1 week ago
Broadcom challenges Nvidia’s AI dominance with ultra-connected Tomahawk networking chip launch – Crypto News
-
De-fi1 week ago
Solana RWA Growth Outpaces Ethereum in 2025 – Crypto News
-
Cryptocurrency1 week ago
Nothing Burger or Crypto Catalyst? – Crypto News
-
Metaverse1 week ago
Nvidia’s Jensen Huang says AI ‘fundamental like electricity’, praises Chinese models as ‘catalyst for global progress’ – Crypto News
-
Business1 week ago
$800 Billion JPMorgan To Rival Tether, Circle, and Ripple In Stablecoin Race – Crypto News
-
Cryptocurrency1 week ago
300% Bitcoin (BTC) Skyrocketing, Ethereum (ETH) Below $3,000 Again, Was Shiba Inu (SHIB) Dump Controlled? – Crypto News
-
De-fi1 week ago
TAC Token Rallies 50% as TON Application Chain Launches to Bring DeFi Back to Telegram – Crypto News
-
others1 week ago
Crypto Exchange Hack: BigONE Users Lose A Massive $27 Million In Recent Exploit – Crypto News
-
Cryptocurrency1 week ago
Pi Network Activity Soars as Mystery Wallet Acquires Millions in PI Coins – Crypto News
-
Blockchain1 week ago
Citigroup CEO Confirms Interest In Issuing A Proprietary Stablecoin—Reuters – Crypto News