<\/p>\n
Preethi, who goes by a single name, as is common in the region, is among the 70 workers hired in Agara and neighboring villages by a startup called Karya to gather text, voice and image data in India\u2019s vernacular languages. She is part of a vast, unseen global workforce \u2014\u00a0operating in countries like India, Kenya and the Philippines \u2014\u00a0who collect and label the data that AI chatbots and virtual assistants rely on to generate relevant responses. Unlike many other data contractors, however, Preethi gets paid well for her efforts, at least by local standards.<\/p>\n
After three days of working with Karya, Preethi earned 4,500 rupees ($54), more than four times the amount the 22-year-old high school graduate usually makes as a tailor in an entire month. The money is enough, she said, to pay off that month\u2019s installment on a loan taken to partly repair the crumbling mud walls of her home that have been carefully patched up with colorful saris. \u201cAll I need is a phone and the internet.”<\/p>\n
Karya was founded in 2021, before the rise of ChatGPT, but this year\u2019s frenzy around generative AI has only added to tech companies\u2019 insatiable demand for data. India alone is expected to have nearly one million data annotation workers by 2030, according to Nasscom, the country\u2019s tech industry trade body. Karya differentiates itself from other data vendors by offering its contractors \u2013 mostly women, and mostly in rural communities \u2013 as much as 20 times the prevailing minimum wage, with the promise of producing better quality Indian-language data that tech companies will pay more to obtain.<\/p>\n
\u201cEvery year, big tech companies spend billions of dollars collecting training data for their AI” and machine learning\u00a0models,\u00a0said Manu Chopra, the 27-year-old Stanford-educated computer engineer behind the startup, told Bloomberg in an interview. \u201cPoor pay for such work is an industry failure.”\u00a0<\/p>\n
If meager wages are\u00a0an industry failure, it\u2019s one that Silicon Valley bears some responsibility for creating. For years, tech companies have outsourced tasks like data labeling and content moderation to cheaper contractors overseas. But now, some of Silicon Valley\u2019s most prominent names are turning to Karya to address one of the biggest challenges for their AI products: finding high-quality data to build tools that can better serve billions of potential non-English speaking users. These partnerships could represent a powerful shift in the economics of the data industry and Silicon Valley\u2019s relationship with data providers.<\/p>\n
Microsoft Corp. has used Karya to source local speech data for its AI products. The Bill & Melinda Gates Foundation is working with Karya to reduce gender biases in data that feeds into large language models, the technology underpinning AI chatbots. And Alphabet Inc.\u2019s Google is leaning on\u00a0Karya and other local partners to gather speech data in 85 Indian districts. Google plans to expand to every district to include the majority language or dialect spoken and build a\u00a0 generative AI model for 125 Indian languages.<\/p>\n
Many AI services have been disproportionately developed with English-language internet data, such as articles, books and social media posts. As a result, these AI models poorly represent the diversity of languages for internet users in other countries who are accessing AI-powered smartphones and apps faster than they\u2019re learning English. Nearly one billion such potential users live in India alone, as the government pushes for a rollout of AI tools in every sphere from healthcare to education to financial services.\u00a0<\/p>\n
\u201cIndia is the first non-Western country we are doing this in, and we are testing Bard in nine Indian languages,”\u00a0said Manish Gupta, head of Google Research in India, referring to the company\u2019s AI chatbot. \u201cOver 70 Indian languages spoken by over a million people each had zero digital corpus. The problem is so stark.”<\/p>\n
Gupta ticked off a list of\u00a0issues that AI firms need to address in order to serve India\u2019s internet users: Non-English datasets are dismally low quality; hardly any conversational data exists in Hindi and other Indian languages; and digitized content from books and newspapers in Indian languages is very limited.<\/p>\n
When used for South Asian languages, some large language models have been found to make up words and struggle with basic grammar. There are also concerns these AI services may reflect a more skewed view of other cultures. It\u2019s critical to have broad representation of training data, including non-English data, so AI systems \u201cdon\u2019t perpetuate harmful stereotypes, produce hate speech, nor yield misinformation,” said Mehran Sahami, a professor in the computer science department at Stanford University.<\/p>\n
Karya, a social impact startup headquartered in Bangalore and supported by grants, is able to broaden the pool of languages represented in part by specifically targeting workers in rural areas who might not otherwise\u00a0be contracted for such\u00a0tasks. Karya\u2019s app can work without internet access and it provides voice support for those with limited literacy.\u00a0 In India, over 32,000 crowdsourced workers have logged into the app, completing 40 million paid digital tasks such as image recognition, contour alignments, video annotation and speech annotation.\u00a0<\/p>\n
For Chopra, the goal isn\u2019t just to improve the supply of data but to fight poverty. Karya\u2019s founder grew up in an impoverished neighborhood called Shakur Basti in West Delhi. He won a scholarship to study in an elite school where he was bullied because his classmates said he \u201csmelled poor.” Chopra landed at Stanford to study computer science but realized he hated the \u201chow you make a billion dollars” mindset he encountered there.\u00a0<\/p>\n
After graduating in 2017, he began working on his long-held interest: using technology to tackle poverty. \u201cIt takes a mere $1,500 in savings to make an Indian eligible to enter the middle class,” Chopra said. \u201cBut the impoverished can take 200 years to reach that level of savings.”<\/p>\n
Microsoft, he learned, had been paying a hefty amount for collecting speech data, albeit of poor quality, to feed its AI systems and research. In 2017, for instance, although 1 million hours of digitized spoken data was available in Marathi, a language spoken in Mumbai and its Western India region, only 165 hours was available for purchase. His startup has since put together 10,000 hours of Marathi speech data for Microsoft\u2019s AI services, read by men and women from five different\u00a0regions.\u00a0<\/p>\n
\u201cTech companies want the data, accent and all,” Chopra said. \u201cYou cough, they want that in the speech \u2013 it represents natural language.”Saikat Guha, a researcher at Microsoft Research India who focuses on the ethics of data collection, said he has also used Karya\u2019s content for a project to aid those with visual disabilities in finding jobs. \u201cThe quality of data is far better than any other source I\u2019ve used,” said Guha. \u201cIf you pay workers fairly, they\u2019re more invested in their work, and the end result is better data.”<\/p>\n
Meanwhile, over 30,000 young, school-educated women are working with Karya to help collect \u201cgender intentional” datasets \u2013 such as that the doctor or boss isn\u2019t always a he \u2013 in six Indian languages for the Bill & Melinda Gates Foundation. It\u2019s the biggest such effort in Indian languages and will serve as a corpus to build datasets to reduce gender-related biases in LLMs.Karya isn\u2019t stopping with India. The company said it\u2019s in talks to sell its platform as a service to organizations in Africa and South America who will do similar work.<\/p>\n
For now, women in Yelandur, another village southwest of Bangalore, eagerly await Karya\u2019s next project: transcribing from a Kannada audio recording. Among them is Shambhavi S., 25, who earned a few thousand rupees from a previous assignment while working in the quiet of her home after feeding her in-laws dinner and putting her children to bed.\u00a0<\/p>\n
\u201cI don\u2019t know what artificial intelligence is, I haven\u2019t heard of it,” said Shambhavi. \u201cI want to earn and educate my children, so they can learn how to use it.”<\/p>\n
Milestone Alert!<\/strong>Livemint tops charts as the fastest growing news website in the world \ud83c\udf0f<\/span> Click here<\/strong><\/a> to know more.<\/p>\n