Artificial intelligence is a familiar-looking monster, say Henry Farrell and Cosma Shalizi – Crypto News

Lovecraft’s shoggoths were artificial servants that rebelled against their creators. The shoggoth meme went viral because an influential community of Silicon Valley rationalists fears that humanity is on the cusp of a “Singularity”, creating an inhuman “artificial general intelligence” that will displace or even destroy us.

But what such worries fail to acknowledge is that we’ve lived among shoggoths for centuries, tending to them as though they were our masters. We call them “the market system”, “bureaucracy” and even “electoral democracy”. The true Singularity began at least two centuries ago with the industrial revolution, when human society was transformed by vast inhuman forces. Markets and bureaucracies seem familiar, but they are actually enormous, impersonal distributed systems of information-processing that transmute the seething chaos of our collective knowledge into useful simplifications.

As the economist Friedrich Hayek argued, any complex economy has to somehow make use of a terrifyingly large body of disorganised and informal “tacit knowledge” about supply and exchange relationships. No individual brain or government can possibly comprehend them, which is why Hayek thought that the planned economy was unworkable. But the price mechanism lets markets summarise this knowledge and make it actionable. A maker of car batteries doesn’t need to understand the particulars of lithium-processing. They just need to know how much lithium costs, and what they can do with it.

Likewise, the political anthropologist James Scott has explained how bureaucracies are monsters of information, devouring rich, informal bodies of tacitly held knowledge and excreting a thin slurry of abstract categories that rulers use to “see” the world. Democracies spin out their own abstractions. The “public” depicted by polls and election results is a drastically simplified sketch of the amorphous mass of opinions, beliefs and knowledge held by individual citizens.

Lovecraft’s monsters live in our imaginations because they are fantastical shadows of the unliving systems that run on human beings and determine their lives. Markets and states can have enormous collective benefits, but they surely seem inimical to individuals who lose their jobs to economic change or get entangled in the suckered coils of bureaucratic decisions. As Hayek proclaims, and as Scott deplores, these vast machineries are simply incapable of caring if they crush the powerless or devour the virtuous. Nor is their crushing weight distributed evenly.

It is in this sense that LLMs are shoggoths. Like markets and bureaucracies, they represent something vast and incomprehensible that would break our minds if we beheld its full immensity. That totality is the product of human minds and actions, the colossal corpuses of text that LLMs have ingested and turned into the statistical weights that they use to predict which word comes next.

As the psychologist Alison Gopnik has argued, LLMs are not nascent individual intelligences but “cultural technologies” which reorganise and noisily transmit human knowledge. Chatbots may wear more human-seeming masks than markets and bureaucracies, but they are no more or less beyond our control. We would be better off figuring out what will happen as LLMs compete and hybridise with their predecessors than weaving dark fantasies about how they will rise up against us.

For example, what if LLMs or other forms of machine learning better capture Hayek’s “tacit knowledge” than market prices can? We could see an economy in which artificial entities compete on the basis of non-price-based representations of complex underlying economic relationships. Half a century ago the economist Martin Weitzman suggested that planned economies might use mathematical objects called “separating hyperplanes” to adapt on the fly. Machine learning can find such hyperplanes, making planning more feasible than before. Alternatively, markets might mutate into a poisonous alien ecology where economic agents fight proxy wars using text-spewing and text-summarising LLMs, just as they use crude algorithms to manipulate Amazon Marketplace and search results today. Would such markets be fairer or more stable than today’s? It seems unlikely.

LLMs might give bureaucrats new tools for adjudicating complex situations. Already, algorithms are being used to help decide whether to grant parole or bail to accused criminals. It is not hard to imagine bureaucrats using LLMs to summarise complex regulations or provide recommendations about how to apply them to novel situations. It could prove impossible to evaluate how well they work, as LLMs don’t leave paper trails. But that might not stop their deployment.

Democratic politics, too, may be transformed. Already, researchers talk about substituting LLMs for opinion polls—they may be out of date, or inaccurate, but polls can be inaccurate, too, and you can interrogate LLMs more dynamically. Perhaps chatbots will help improve democratic debate, helping people clarify what they believe, or turn quarrels into agreement. Or, instead, they might degrade debate with their tendency to spin convincing factoids from thin air, and their capacity to flood online discussion with spurious opinions that purport to come from real people.

Repurposing the shoggoth might help us begin to answer these questions. Rather than speculate about the motives of intelligent AIs, we could ask how LLMs might interact with their older cousins. The modern world has been built by and within monsters, which crush individuals without remorse or hesitation, settling their bulk heavily on some groups, and feather-light on others. We eke out freedom by setting one against another, deploying bureaucracy to limit market excesses, democracy to hold bureaucrats accountable, and markets and bureaucracies to limit democracy’s monstrous tendencies. How will the newest shoggoth change the balance, and which politics might best direct it to the good? We need to start finding out.

Henry Farrell is a professor of international affairs and democracy at Johns Hopkins University, and co-author of “Underground Empire: How America Weaponized the World Economy”.

Cosma Shalizi is a professor of statistics and machine learning at Carnegie Mellon University and external faculty member at the Santa Fe Institute.

© 2023, The Economist Newspaper Limited. All rights reserved. From The Economist, published under licence. The original content can be found on www.economist.com

Updated: 21 Aug 2023, 02:57 PM IST