A Big Little Idea Called Legibility (2010)

James C. Scott’s fascinating and seminal book, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, examines how, across dozens of domains, ranging from agriculture and forestry, to urban planning and census-taking, a very predictable failure pattern keeps recurring.  The pictures below, from the book (used with permission from the author) graphically and literally illustrate the central concept in this failure pattern, an idea called “legibility.”

States and large organizations exhibit this pattern of behavior most dramatically, but individuals frequently exhibit it in their private lives as well.

Along with books like Gareth Morgan’s Images of Organization, Lakoff and Johnson’s Metaphors we Live By, William Whyte’s The Organization Man and Keith Johnstone’s Impro, this book is one of the anchor texts for this blog. If I ever teach a course on ‘Ribbonfarmesque Thinking,’ all these books would be required reading. Continuing my series on complex and dense books that I cite often, but are too difficult to review or summarize, here is a quick introduction to the main idea.

The Authoritarian High-Modernist Recipe for Failure

Scott calls the thinking style behind the failure mode “authoritarian high modernism,” but as we’ll see, the failure mode is not limited to the brief intellectual reign of high modernism (roughly, the first half of the twentieth century).

Here is the recipe:

• Look at a complex and confusing reality, such as the social dynamics of an old city
• Fail to understand all the subtleties of how the complex reality works
• Attribute that failure to the irrationality of what you are looking at, rather than your own limitations
• Come up with an idealized blank-slate vision of what that reality ought to look like
• Argue that the relative simplicity and platonic orderliness of the vision represents rationality
• Use authoritarian power to impose that vision, by demolishing the old reality if necessary
• Watch your rational Utopia fail horribly

The big mistake in this pattern of failure is projecting your subjective lack of comprehension onto the object you are looking at, as “irrationality.” We make this mistake because we are tempted by a desire for legibility.

Legibility and Control

Central to Scott’s thesis is the idea of legibility. He explains how he stumbled across the idea while researching efforts by nation states to settle or “sedentarize” nomads, pastoralists, gypsies and other peoples living non-mainstream lives:

The more I examined these efforts at sedentarization, the more I came to see them as a state’s attempt to make a society legible, to arrange the population in ways that simplified the classic state functions of taxation, conscription, and prevention of rebellion.  Having begun to think in these terms, I began to see legibility as a central problem in statecraft. The pre-modern state was, in many crucial respects, particularly blind; it knew precious little about its subjects, their wealth, their landholdings and yields, their location, their very identity. It lacked anything like a detailed “map” of its terrain and its people.

The book is about the 2-3 century long process by which modern states reorganized the societies they governed, to make them more legible to the apparatus of governance. The state is not actually interested in the rich functional structure and complex behavior of the very organic entities that it governs (and indeed, is part of, rather than “above”). It merely views them as resources that must be organized in order to yield optimal returns according to a centralized, narrow, and strictly utilitarian logic. The attempt to maximize returns need not arise from the grasping greed of a predatory state. In fact, the dynamic is most often driven by a genuine desire to improve the lot of the people, on the part of governments with a popular, left-of-center mandate. Hence the subtitle (don’t jump to the conclusion that this is a simplistic anti-big-government conservative/libertarian view though; this failure mode is ideology-neutral, since it arises from a flawed pattern of reasoning rather than values).

The book begins with an early example, “scientific” forestry (illustrated in the picture above). The early modern state, Germany in this case, was only interested in maximizing tax revenues from forestry. This meant that the acreage, yield and market value of a forest had to be measured, and only these obviously relevant variables were comprehended by the statist mental model. Traditional wild and unruly forests were literally illegible to the state surveyor’s eyes, and this gave birth to “scientific” forestry: the gradual transformation of forests with a rich diversity of species growing wildly and randomly into orderly stands of the highest-yielding varieties. The resulting catastrophes — better recognized these days as the problems of monoculture — were inevitable.

The picture is not an exception, and the word “legibility” is not a metaphor; the actual visual/textual sense of the word (as in “readability”) is what is meant. The book is full of thought-provoking pictures like this: farmland neatly divided up into squares versus farmland that is confusing to the eye, but conforms to the constraints of local topography, soil quality, and hydrological patterns; rational and unlivable grid-cities like Brasilia, versus chaotic and alive cities like Sao Paolo. This might explain, by the way, why I resonated so strongly with the book.  The name “ribbonfarm” is inspired by the history of the geography of Detroit and its roots in “ribbon farms” (see my About page and the historic picture of Detroit ribbon farms below).

High-modernist (think Bauhaus and Le Corbusier) aesthetics necessarily lead to simplification, since a reality that serves many purposes presents itself as illegible to a vision informed by a singular purpose. Any elements that are non-functional with respect to the singular purpose tend to confuse, and are therefore eliminated during the attempt to “rationalize.” The deep failure in thinking lies is the mistaken assumption that thriving, successful and functional realities must necessarily be legible. Or at least more legible to the all-seeing statist eye in the sky (many of the pictures in the book are literally aerial views) than to the local, embedded, eye on the ground.

Complex realities turn this logic on its head; it is easier to comprehend the whole by walking among the trees, absorbing the gestalt, and becoming a holographic/fractal part of the forest, than by hovering above it.

This  imposed simplification, in service of legibility to the state’s eye, makes the rich reality brittle, and failure  follows. The imagined improvements are not realized. The metaphors of killing the golden goose, and the Procrustean bed come to mind.

The Psychology of Legibility

I suspect that what tempts us into this failure is that legibility quells the anxieties evoked by apparent chaos. There is more than mere stupidity at work.

In Mind Wide Open, Steven Johnson’s entertaining story of his experiences subjecting himself to all sorts of medical scanning technologies, he describes his experience with getting an fMRI scan. Johnson tells the researcher that perhaps they should start by examining his brain’s baseline reaction to meaningless stimuli. He naively suggests a white-noise pattern as the right starter image. The researcher patiently informs him that subjects’ brains tend to go crazy when a white noise (high Shannon entropy) pattern is presented. The brain goes nuts trying to find order in the chaos. Instead, the researcher says, they usually start with something like a black-and-white checkerboard pattern.

If my conjecture is correct, then the High Modernist failure-through-legibility-seeking formula is a large scale effect of the rationalization of the fear of (apparent) chaos.

[Techie aside: Complex realities look like Shannon white noise, but in terms of deeper structure, their Kolmogorov-Chaitin complexity is low relative to their Shannon entropy; they are like pseudo-random numbers or π, rather than real random numbers; I wrote a two-part series on this long ago, that I meant to continue, but never did].

The Fertility of the Idea

The idea may seem simple (though it is surprisingly hard to find words to express it succinctly), but it is an extraordinarily fertile one, and helps explain all sorts of things. One of my favorite unexpected examples from the book is the “rationalization” of people names in the Philippines under Spanish rule (I won’t spoil it for you; read the book). In general, any aspect of a complex folkway, in the sense of David Hackett Fischer’s Albion’s Seed, can be made a victim of the high-modernist authoritarian failure formula.

The process doesn’t always lead to unmitigated disaster. In some of the more redeeming examples, there is merely a shift in a balance of power between more global and more local interests. For example, we owe to this high-modernist formula the creation of a systematic, global scheme for measuring time, with sensible time zones. The bewilderingly illegible geography of time in the 18th century, while it served a lot of local purposes very well (and much better than even the best atomic clocks of today), would have made modern global infrastructure, ranging from the railroads (the original driver for temporal discipline in the United States) to airlines and the Internet, impossible. The Napoleanic era saw the spread of the metric system; again an idea that is highly rational from a centralized bird’s eye view, but often stupid with respect to the subtle local adaptions of  the systems it displaced. Again this displaced a good deal of local power and value, and created many injustices and local irrationalities, but the shift brought with it the benefits of improved communication and wide-area commerce.

In all these cases, you could argue that the formula merely replaced a set of locally optimal modes of social organization with a globally optimal one. But that would be missing the point. The reason the formula is generally dangerous, and a  formula for failure, is that it does not operate by a thoughtful consideration of local/global tradeoffs, but through the imposition of a singular view as “best for all” in a pseudo-scientific sense. The high-modernist reformer does not acknowledge (and often genuinely does not understand) that he/she is engineering a shift in optima and power, with costs as well as benefits. Instead, the process is driven by a naive “best for everybody” paternalism, that genuinely intends to improve the lives of the people it affects. The high-modernist reformer is driven by a naive-scientific Utopian vision that does not tolerate dissent, because it believes it is dealing in scientific truths.

The failure pattern is perhaps most evident in urban planning, a domain which seems to attract the worst of these reformers. A generation of planners, inspired by the crazed visions of Le Corbusier, created unlivable urban infrastructure around the world, from Braslia to Chandigarh. These cities end up with deserted empty centers populated only by the government workers forced to live there in misery (there is even a condition known as “Brasilitis” apparently), with slums and shanty towns emerging on the periphery of the planned center; ad hoc, bottom-up, re-humanizing damage control as it were. The book summarizes a very elegant critique of this approach to urban planning, and the true richness of what it displaces, due to Jane Jacobs.

Applying the Idea

Going beyond the book’s own examples, the ideas shed a whole new light on other stories/ideas. Two examples from my own reading should suffice.

The first is a book I read several years back, by Nicholas Dirks, Castes of Mind: Colonialism and the Making of Modern India, which made the argument (originally proposed by the orientalist Bernard Cohn), that caste in the sense of the highly rigid and oppressive, 4-varna scheme was the result of the British failing to understand a complex social reality, and imposing on it their own simplistic understanding of it (the British Raj is sometimes called the “anthropological state” due to the obsessive care it took to document, codify and re-impose as a simplified, rigidified, Procrustean prescription, the social structure of pre-colonial India).  The argument of the book — obviously one that appeals to Indians (we like to blame the British or Islam when we can) — is that the original reality was a complex, functional social scheme, which the British turned into a rigid and oppressive machine by attempting to make it legible and governable. While I still don’t know whether the argument is justified, and whether the caste system before the British was as benevolent as the most ardent champions of this view make it out to be, the point here is that if it is true, Scott’s failure model would describe it perfectly.

The second example is Gibbon’s Decline and Fall of the Roman Empire, which I am slowly reading right now (I think it is going to be my personal Mount Everest; I expect to summit in 2013). Perhaps no other civilization, either in antiquity or today, was so fond of legible and governable social realities.  I haven’t yet made up my mind, but reading the history through the lens of Scott’s ideas, I think there is  strong case to be made that the fall of the Roman empire was a large-scale instance of the legibility-failure pattern. Like the British 1700 years later, the Romans did try to understand the illegible societies they encountered, but their failure in this effort ultimately led to the fall of the empire.

Aside: if you decide to attempt Mount Everest along with me, take some time to explore the different editions of Gibbon available; I am reading a $0.99 19th century edition on my Kindle — all six volumes with annotations and comments from a decidedly pious — and critical — Christian editor. Sometimes I don’t know why I commit these acts of large-scale intellectual masochism.  The link is to a modern, abridged Penguin edition.

Is the Model Relevant Today?

The phrase “high-modernist authoritarianism” might suggest that the views in this book only apply to those laughably optimistic, high-on-science-and-engineering high modernists of the 1930s. Surely we don’t fail in these dumb ways in our enlightened postmodern times?

Sadly, we do, for four reasons:

1. There is a decades-long time lag between the intellectual high-watermark of an ideology and the last of its effects
2. There are large parts of the world, China in particular, where authoritarian high-modernism gets a visa, but postmodernism does not
3. Perhaps most important: though this failure mode is easiest to describe in terms of high-modernist ideology, it is actually a basic failure mode for human thought that is time and ideology neutral. If it is true that the Romans and British managed to fail in these ways, so can the most postmodern Obama types. The language will be different, that’s all.
4. And no, the currently popular “pave the cowpaths” and behavioral-economic “choice architecture” design philosophies do not provide immunity against these failure modes. In fact paving the cowpaths in naive ways is an instance of this failure mode (the way to avoid it would be to choose to not pave certain cowpaths). Choice architecture (described as “Libertarian Paternalism” by its advocates) seems to merely dress up authoritarian high-modernism with a thin coat of caution and empirical experimentation. The basic and dangerous “I am more scientific/rational than thou” paternalism is still the central dogma.

[Another Techie aside: For the technologists among you, a quick (and very crude) calibration point should help: we are talking about the big brother of waterfall planning here. The psychology is very similar to the urge to throw legacy software away. In fact Joel Spolsky’s post on the subject Things You Should Never Do, Part I, reads like a narrower version of Scott’s arguments. But Scott’s model is much deeper, more robust, more subtly argued, and more broadly applicable.  I haven’t yet thought it through, but I don’t think lean/agile software development can actually mitigate this failure mode anymore than choice architecture can mitigate it in public policy]

So do yourself a favor and read the book, even if it takes you months to get through. You will elevate your thinking about big questions.

High-Modernist Authoritarianism in Corporate and Personal Life

The application of these ideas in the personal/corporate domains actually interests me the most. Though Scott’s book is set within the context of public policy and governance, you can find exactly the same pattern in individual and corporate behavior. Individuals lacking the capacity for rich introspection apply dumb 12-step formulas to their lives and fail. Corporations: well, read the Gervais Principle series and Images of Organization. As a point of historical interest, Scott notes that the Soviet planning model, responsible for many spectacular legibility-failures, was derived from corporate Taylorist precedents, which Lenin initially criticized, but later modified and embraced.

Final postscript: these ideas have strongly influenced my book project, and apparently, I’ve been thinking about them for a long time without realizing it. A very early post on this blog (I think only a handful of you were around when I posted it), on the Harry Potter series and its relation to my own work in robotics, contains some of these ideas. If I’d read this book before, that post would have been much better.

from Hacker News https://ift.tt/2msTvde