This piece originally appeared on Second Best.
This picks up from Part I but can also be read standalone.
AI safety means different things to different people, but whether the focus is job loss or the x-risk from an unaligned superintelligence, the concerns are always presented as relatively first-order. That is, AI safety is usually conceptualized in terms of what AI will do directly, rather than in terms of AI’s likely indirect, second-order effects on society and the shape of our institutions. This is an enormous blind spot.
Circa the early 2000s, “internet safety” discussions revolved around first-order issues like identity theft, cybercrime and child exploitation. But with the benefit of hindsight, these direct concerns were swamped by the internet’s second-order effects on our politics and culture. Indeed, between an information tsunami and new platforms for mass mobilization, the internet destabilized political systems worldwide, even leading to outright regime change in the case of the Arab Spring.
To the extent AI is simply the next stage in the digital revolution, I expect these trends to only intensify. The issue is not that AI and informational technology are inherently destabilizing. Rather, to put it in slightly Marxian terms, the issue is that society’s technological base is shifting faster than its institutional superstructure can keep up. Populist leaders who promise to root out corruption and reset the system are a symptom of governance structures that have in some sense lost their “direction of fit,” like clothes that shrank in the wash or a species outside its evolutionary niche.
China’s ruling class studied the internet-enabled revolts in Cairo and Tunisia with close concern. As good historical materialists, the CCP grokked how changes in the mode of production can lead to changes in the structure of social relations. They therefore redoubled their investment in internet surveillance and other societal controls, nipping their own Jasmine Revolution in the bud.
Democratized AI is a much greater regime change threat than the internet, and the CCP is treating it as such. Chinese regulators recently released draft rules to require user verification and security reviews for ChatGPT-like models. The rules will also prohibit “any content that subverts state power, advocates the overthrow of the socialist system, incites splitting the country or undermines national unity.”
America and the liberal democracies in the West are open societies and unlikely to ever adopt such strict controls over AI, if they could even be enforced. At the same time, we are only truly “open” at the meta-level. Private services and organizations impose any number of terms and conditions on their users, including things that would seem draconian if imposed by the state. From workplace policies and credit scores to DRM and data brokers, the liberal constitutional tradition doesn’t so much preclude invasive surveillance and social regulation as offload those functions into competing private hands.
For generations, it was an implicit assumption of U.S. foreign policy that economic liberalization and free trade would push autocratic regimes to democratize. Information technologies that give voice to the voiceless were key to this grand strategy; a kind of glasnost in a bottle. This is why the Defense Department invested so heavily in the early internet and why our international development agencies promote internet access abroad. As one USAID report puts it, “Connecting People. Transforming Nations.” The idea that technology can precipitate a regime change is thus not foreign to the U.S. political establishment; they just assumed our relative openness would keep those dynamics from playing out at home.
The theory that free trade and communications technology would promote democracy was always underspecified. Democracy is itself a hazy concept, a bit like clinical depression. We may know a depressed person when we see one, but brains are complex systems, so it’s quite likely that depression is actually a variety of different phenomena that get bundled according to their symptomatology. Saying “the internet promotes democracy” is thus like saying “SSRIs treat depression.” In reality, SSRIs block the reabsorption of serotonin into particular neurons, which causes a cascade of poorly understood effects that happen to include improved mood.
The effects of information technology on society are just as complex and liable to cut in different directions. While weaker states faced crises, China adapted their digital institutions to mitigate popular dissent and regulate society with unprecedented granularity. Similarly, the signal extraction unlocked by AI could point to a radically more transparent society; or, at second-order, society could react to that heightened transparency by building their fences higher. In stronger states, access to AI could even become monopolized by an all-seeing Leviathan, turning the democratic telos of information technology on its head.
Internalizing costs
Hobbes’ case for the Leviathan ultimately rests on a kind of negative externality. The more technology empowers and emboldens the sovereign individual, the more space there is for conflict at the boundary of our intersecting spheres of influence. Wherever the negative externality to coexisting is high, one thus tends to find either civil conflict or a draconian enforcement regime to compensate.
In medieval times, freedom simply meant the condition of not being a serf or slave, while the richer concept of liberty, to the extent it existed at all, referred to the inviolability of the great estate. It was an inherently corporate concept; the liberty to govern your fief — the walled garden of yore. The modern, individualist conception of liberty, in contrast, emerged with the consolidation of the modern nation-state and the adoption of impersonal legal institutions. These institutions were ultimately made possible by the printing revolution and the growing legibility of nature and society, but only after a bloody dialectic.
Individual rights, the rule of law, and negative liberties thus have a historical if somewhat paradoxical dependence on strong, centralized governments with high rates of fiscal and administrative capacity. This makes liberal democracy as we know it the byproduct of a contingent technological equilibrium; one that AI is almost certainly going to change.
One can think about the “industrial organization” of modern nation-states in terms of Ronald Coase’s Theory of the Firm. When trust is low and transaction costs are high, it often makes sense to produce things in-house rather than contract with a third party. After all, contractors are hard to monitor and don’t always do exactly what you want, much less on time. Corporations thus exist to economize on transaction costs by integrating production within a vertical hierarchy, enabling greater monitoring and command-and-control.
Governments exist for similar reasons. Some things, like national defense and basic legal institutions, are close to pure public goods. Other things, like roads and bridges, can be produced privately, but often involve so many competing interests as to make bottom-up negotiation impractical. The same goes for many kinds of market failure. Pollution externalities could, in principle, be solved by creating property rights over the air and letting rights-holders negotiate, but in practice it’s easier (read: the transaction costs are lower) to simply tax polluters according to an estimate of the social cost of their pollution. The size and scope of liberal democracies thus grew beyond its laissez faire phase in tandem with the complexity of industrial society.
Governments and corporations also economize on transaction costs by creating universal standards and procedures. Just as courts establish common precedents and companies require their employees to use the same software, governments coordinate shared time zones, units of measurement, and industry standards.
Decision makers in corporations and governments don’t necessarily understand Coase’s theory explicitly. Rather, organizations look the way they do because of some mix of intentional planning, competition and path dependency. In competitive markets, corporations that outsource excessively or insource the wrong things are less profitable than those that don’t. They thus go out of business or are reorganized from within. The competitive pressures on governments are weaker and more indirect but still exist. Democracies must continuously broker with internal stakeholders. Institutions that attract productive capital and labor tend to thrive, while grossly inefficient governments are punished by their bond holders and forced to adopt structural reforms. If a government expands too far beyond its optimal scope, whether in policy substance or territorially, it can face a popular revolt or even disintegrate, as occurred with the American Revolution, the break-up of the Soviet Union, and past waves of decolonization.
The modern nation-state is itself the product of institutional competition. Around the time of the printing revolution, the fragmented kingdoms and principalities of Europe began to vertically integrate into consolidated nation-states. They built armies, professionalized tax collection, and started threatening their neighbors with conquest and conversion, compelling those neighbors to build armies of their own. Old settlements broke down and war reigned for decades before stabilizing with the Peace of Westphalia and the mutual recognition of sovereign borders. The modern nation-state thus exists not in spite of anarchy but because of it.
I was an anarchist through my teenage years, but learned to love the state once I realized that, absent dramatic technological change, an anarchic world would quickly snap back to a world of centralized governments. While neoreactionaries and anarchocapitalists fantasize about a patchwork system of competing, opt-in network states, the presence of transaction costs makes those jurisdictions vulnerable to might-makes-right conquest. In such a system, competing jurisdictions would be driven by evolutionary selection pressures to merge with their neighbors, harmonize their laws, produce public goods, and forge treaties with their remaining foes. To the extent this exactly mirrors the transition from feudalism to centralized nation-states, we are already living under a kind of anarchy — a point Hobbes understood well. Between the realist anarchy of the international order and the polycentricity of our domestic institutions, our contemporary anarchy is simply one that trades-off unalloyed freedom for enormous efficiencies of scale.
Anarchy unbound
The smarter anarchists know all this, and so pair their political philosophy with a kind of techno-capitalist accelerationism. Blending ‘70s counterculture with Hayek’s work on information theory, monetary economics, and sensory cybernetics, the early techno-libertarians imagined a future in which the internet, AI, and digital money would converge to bring about a radical new form of decentralized self-government.
Timothy Leary was perhaps the most prophetic on this account. In a controversial speech at the 1977 Libertarian Party Convention titled “Terrestrial and Post-Terrestrial Freedom,” Leary extrapolated from the early ARPANET to “a network that would connect computers worldwide,” allowing real time communication and a revolution against the established order. While the audience dismissed him at the time, Leary’s prescience has since been vindicated by the growing tempo of internet-enabled revolts, from the Arab Spring and the protests in Iran, to America’s own slapdash peasant rebellion on January 6th.
From the late 1990s to the mid-2000s, techno-libertarians could be found on obscure internet forums debating superintelligence timelines, life extension technologies, and the ideal Martian system of government. They called themselves the Extropians. Among their ranks were many now-recognizable names, including AI theorists like Eliezer Yudkowsky and Nick Bostrom, cryptocurrency pioneers like Hal Finney and Nick Szabo, nanotechnologists like Eric Drexler, and would-be brain uploads like Max More and Robin Hanson.
The Extropians could both see where technology was headed and appreciated the interplay between technology and institutional evolution. Assuming our civilization survives, the AI historians of the future may look back on their (often pseudonymous) treatises as a kind of singularitarian Republic of Letters.
I was born in 1991 but have been online since 1999, the year I created my first Newgrounds account from the Windows 95 in my parents’ attic. As my interests shifted from stickman animation to science and philosophy, I became acquainted with libertarian anarchists like David Friedman and futurists like Ray Kurzweil, and saw the emergence of self-regulating services like E-bay and Wikipedia as demonstrating their productive fusion. I thus studied as much social science as I could and, in 2015, moved from rural Canada to Washington, DC, for a graduate fellowship within the Mercatus Center’s technology program; the reputed home for rationalist libertarians, “analytical anarchists,” and even a few OG Extropians.
The Narrow Corridor
This biographical detour is just to establish my bona fides with the e/acc world, as the heirs to the Extropians. I remain no less committed to the cause of human freedom; however, I have updated to the fragility of its institutional conditions. Political liberalism, I’ve come to realize, exists within what Acemoğlu and Robinson call “The Narrow Corridor,” a kind of harmonious saddle path for the co-evolution of state and society. As Acemoğlu puts it, “You need this conflict to be balanced… If society is too weak, that leads to despotism. But on the other side, if society is too strong, that results in weak states that are unable to protect their citizens.”
From: “Populism and the ‘Narrow Corridor’ of Liberty and Justice” (2022)
Compare Switzerland to Afghanistan. Switzerland is a federal republic composed of 26 cantons with significant autonomy, making it one of the most decentralized countries on earth. It also routinely ranks first on various indices of freedom and human development. Afghanistan, in contrast, suffers from many geographic and cultural barriers to state formation, from hills that insurgents can escape to, to clan-based social networks that inhibit inclusive institutions like the rule of law. This results in a weak state typified by the constant threat of organized violence, creating a market opportunity for the Taliban as a provider of social order of last resort, supported in part through repressive religious doctrines that filter out free-riders and help solve for the problem of credible commitment.
Switzerland has its share of mountains and clans, which is why it’s so federated in the first place. Nonetheless, the decentered nature of Swiss institutions betrays a hidden, extended order, akin to a crystalline material whose special properties depend on atoms arranged into an unlikely, low entropy configuration.
Weakening the nation-state is thus not synonymous with promoting liberty. Rather, the proper synthesis between despotism and anarchy is captured in the classical concept of “order liberty.” As Hayek notes in The Constitution of Liberty,
Not Locke, nor Hume, nor Smith, nor Burke, could ever have argued, as Bentham did, that ‘every law is an evil for every law is an infraction of liberty.’ Their argument was never a complete laissez faire argument, which, as the very words show, is also part of the French rationalist tradition, and in its literal sense was never defended by any of the English classical economists.
Instead, Hayek argues the British classical liberals located liberty in “the evolution of ‘well-constructed institutions’ where the ‘rules and principles of contending interests and compromised advantages’ would be reconciled,” thereby channeling “individual efforts to socially beneficial aims.”
The liberal tradition is thus divided between utopian rationalists and pragmatic fallibilists. The former are descendants of the first Enlightenment, self-conscious of their fundamental autonomy and eager to redesign society from first principles. The latter adhere to what Thomas Sowell called “the constrained vision.” They recognize that there are no ideal solutions, only second-best trade-offs, and have a Hayekian — if not Hegelian — appreciation for the ways in which Reason is socially mediated, rather than the wellspring of an individual ego.
The use of intelligence in society
The Hayekian case for a strong but limited government is ultimately more epistemic than normative. Central planners suffer from a knowledge problem. Government bureaucrats simply can’t match the parallel computation of the price system, even on their best day.
Yet as Tyler Cowen once pointed out, governments are spontaneous orders too, and will thus expand or contract as the epistemic conditions change. The economists Nicola Mastrorocco and Edoardo Teso recently tested this by looking at the role of monitoring technology in the evolution of U.S. federal bureaucracy from 1817 to 1905. By exploiting the staggered expansion of railroad and telegraph networks, they find that the growing ability of politicians to monitor state agents throughout the territory was an important driver of bureaucratic growth. As they put it,
“The results suggest that high monitoring costs are associated with small, personalistic state organizations based on networks of trust; technological shocks lowering monitoring costs facilitate the emergence of modern bureaucratic states.”
In other words, monitoring costs are a core transaction cost influencing the equilibrium size and scope of modern governments. This is what James C. Scott meant by Seeing Like a State. To return to the metaphor from Part I, “Seeing like an AI state” suggests our bureaucracies are about to get high resolution x-ray glasses.
Whether or not this leads to a bureaucratic expansion is another question. On the one hand, AI could enable regulators to devise rules with fractal specificity, micromanaging things that used to be illegible. On the other hand, the human and physical footprint of our bureaucracies could radically shrink, as even the finest-grained forms of compliance become automatic and thus invisible.
Nevertheless, the civil rights and liberties enshrined in our laws and Constitution will tend to shift the most extreme forms of monitoring into private entities. Suppose AI gives us 99.999% reliable lie detectors, thereby rendering “innocent until proven guilty” obsolete. Yet whether such evidence can be gathered in the first place will be up to the accused, as the Fifth Amendment protects against self-incrimination. Most private entities will be under no such obligation, however. And while it’s illegal for employers to compel a lie detector test under the Employee Polygraph Protection Act, this becomes unenforceable once a surreptitious recording of an employee’s face or body language suffices.
These inherent constraints on government, combined with AI’s much faster diffusion through the private sector, suggest a net weakening of liberal governments relative to the rest of society. The rapid degeneracy of our legacy institutions could thus make a kind of high-tech anarchy suddenly viable, bootstrapped off the latent demand for social order and other public goods.
The moment governments realize that AI is a threat to their sovereignty, they will be tempted to clamp down in a totalitarian fashion. It’s up to liberal democracies to demonstrate institutional co-evolution as a third-way between degenerate anarchy and an AI Leviathan. At a minimum, this will require embracing AI tooling within the machinery of government; painful concessions to the government functions that AI simply renders obsolete; and the dialectical construction of a new social contract — an AI ordered-liberty — that one hopes is far more Swiss than Pashtun.
Regardless of what path we take, one thing is certain: the U.S. government of 2040 will look as different to our contemporaries as the U.S. government of the 1940s must have looked to the men and women of the pre-industrial era.
Part III will dig deeper into how this could all play out.
