Whimsley

What Cascade Theories Don't Tell Us

Attention conservation notice: 3,000 words of amateur rumination on the problems with agent-based theories of uprisings. Part of a series about identity, institutions, and uprisings.

In the torrent of debate over the causes and dynamics of the "Arab Spring" uprisings, one of the strongest currents has its source in cascade models of uprisings. The starting points for these  models are Mark Granovetter's simple and abstract Threshold Models of Collective Behaviour and Thomas Schelling's similar ideas in "Macromotives and Microbehaviour". If a population of individuals is presented with a choice between A and B, and if each will choose A only if some number of other people (their "threshold") also choose A, then very small differences in the distribution of thresholds can lead to very different results. In some cases, everyone chooses A, in others everyone chooses B, in others, the population is split. One spark sputters; an identical spark starts a prairie fire.

Timur Kuran and Susanne Lohmann used these ideas to explain the sudden and surprising uprisings of 1989 in Eastern Europe. Both drew attention to the paucity of information in an authoritarian state, about what others believe and about the nature of the state itself. Highly motivated protesters with a low threshold engage in dissent or protest and thereby reveal information to other disgruntled people about the breadth of disenchantment, or about the nature of the state, or about the experience of other individuals in relation to the state. Others join in and by doing so reveal more. And so it goes. With the right distribution of thresholds, a single person's action can light a fire that sweeps across a continent. Sudden, dramatic uprisings in authoritarian states are a switch from one equilibrium to another as dissent draws back the veil of silence and people see each other, see the nature of the regime under which they live, and realize that everyone else sees it too.

Digital cascades

A new twist has been added to the cascade mix by the 2009 revolt in Iran and the "Arab Spring" uprisings that erupted in Tunisia two years ago. The highly-publicised role of social media posed new challenges for theories of contentious politics, and three related facets of the events of Spring 2011 made cascade theories appealing:

• The sudden, unexpected nature of the uprisings and their swift spread from country to country bring to mind the Eastern European regimes in 1989-1990.
• The lack of a strong, organized opposition during the early stages of the uprisings is a challenge for theories of social movements, and plays to the strengths of cascade models.
• The prominent role of digital technology, and particularly social media, lends itself to network-based models of society.

Cascade models suggest that contentious politics is an assurance game that can be solved so long as the transaction costs associated with information exchange are low enough. According to a popular narrative, social media networks lower those costs, rendering newly vulnerable those states that rely for their stability on information scarcity and on preference falsification. In an information-rich world, the low-information equilibrium is no longer tenable.

Cascade models have, well, cascaded out from academia and into the mainstream world as the primary way to understand the Arab Spring uprisings. For one influential statement, see Clay Shirky in Foreign Affairs, but a quick look through a bibliography of Arab Spring literature (here, here) will reveal many more.

As the current gained strength, it absorbed a number of tributaries. One was the Habermasian idea that a rich public sphere or a strong civil society is the fertile ground in which discontent can take root. Another stream focuses on the role of information and the media, bringing communication studies and media studies scholars into the cascade current. The end result is a current that has brought to the fore the role of social media in the Arab Spring, flowing into the sea of mainstream opinion. If you follow the River Nile of commentary about "Twitter Revolution" and "Facebook Revolution" and "Revolution 2.0" upstream far enough you reaach the theoretical source: Granovetter and Schelling, Kuran and Lohmann.

But it's a long way from Granovetter to the Arab Spring, and along the way important things have been lost, and Mistakes Have Been Made.

Caught in the Net

One of the lovely things about cascade theories is that they lend themselves to simulations. Take that idea of a threshold, for example. Doesn't it make sense that your threshold for action depends more on the actions of people in your social networks than people in some distant city that you've never met? So you can start doing computer simulations of cascades, and how they depend on network structure, and how they depend on the density of connections, and… well, the possibilities are almost endless. So network models have become a common way to extend the original simple ideas in more sophisticated ways. New hypotheses get expressed as network models, and the conclusions feed back into the theory.

But all the simulations in the world don't change the fact that adopting a network model is an input to the theory, not an output, and that the network society perspective brings with it a whole set of assumptions and priorities that need examination. For example, adopting a network model means relegating organizations and institutions to the periphery, and moving ideas of "self-organization", connectivity, and peer-to-peer communication to the centre -- not because of any factual conclusion but because some concepts can be expressed naturally within a network model and others can't. Concepts of symbolism, identity, institutions, and the difficulty of establishing trust are hard to express and so get pushed aside or ignored completely. The popularity of agent-based models doesn't disprove the importance of such concepts, it just makes us blind to them.

Information and symbolism

Just as the cascade point of view has taken on the "network society", so it has highlighted the role of information in authoritarian states, and hence the role of media and technology in fostering change. It's true that information in authoritarian states is limited, almost by definition, but it's a long way from that to the idea that the paucity of common knowledge is what is holding back uprisings. Again, it's easy to express ideas of information flow across networks within agent-based models, but that doesn't actually tell us that information flow in authoritarian societies is the barrier to social change. Yes, there is something of interest in the lens that information provides, but treating it as the only lens of interest is too narrow.

Once you start swimming in the cascade current, you find yourself surrounded by like-minded individuals. It's no surprise that agent-based theories have found fertile ground among those interested in technology. After all, technophiles are typically happy with a focus on the role of information, tend to have a mathematical cast of mind, but are less happy with sociological concepts or social-historical investigations of social movements. Economic concepts and techniques find a welcome home (transaction costs and utility functions) while talk of collective identity is more difficult to translate into agent models. To be blunt, adopting the agent-based outlook saves you a lot of work, because you don't have to read all those historical studies of movements and organizations and make up your mind about the issues they raise.

That there is something of a cultural divide between the agent-based modelers and other currents of the debate, such as those who study social movements, is not new to me of course: here is Andrew Walder:

The field of social movements and contentious politics has been a prolonged effort to establish a sociological alternative to the more parsimonious theories of economics. The increasing insistence on the subjective dimensions of mobilization—collective action frames, the formation of collective identities, the role of emotions—is essentially motivated by a feeling that the initial emphasis on organizations, networks, and political opportunity structures were not sufficiently different from rational social choice models to offer a fully sociological alternative.

Protests and other acts of dissent share many common features. For one, they are often deeply symbolic acts. Protesters choose symbolic days on which to make their point (the 40th anniversary of the GDR), they focus their actions on national symbols (the storming of the National Palace by the Sandinistas). Presenting protest as information-revelation ignores the symbolic nature of protests, the importance of demonstrating "worthiness, unity, numbers, and commitment", the practice of drawing from a shared repertoire of actions, which other currents of thought have shown to be important. At least, my reading of the literature suggests that there is a lot of verbiage around the basic theory, and sometimes in this verbiage there is reference to the symbolic nature of the protests, but there's little to actually tie the words to the theory.

If scholars of social movements have demonstrated that identity and its formation are important motivators for political action, and if these factors appear nowhere in the cascade models that seek to explain sudden political change, then there's a gap in the theory.

Preference falsification, or obedience?

In Timur Kuran's work, the central concept of "preference falsification" is grounded in the distinction between an internal, private viewpoint and an externally expressed opinion. There is, however, a difficulty with ascribing political function to an internal, psychological state of mind. A diagnosis of "preference falsification" is only possible after the fact of an uprising. Before the fact, from a political point of view there is no observable difference between a nation of preference falsifiers and a nation of contented citizens, and to the extent that there is any observable difference between the two, preference falsification loses its explanatory power. Similarly, what is the observable difference before the uprising between Lohmann's East German citizens, who each have bad experiences with their government but are unaware of its overall bad performance, and an East Germany of citizens who have had good experiences with their government?

The concept of preference falsification makes it dangerously easy to read one's own assumptions and views into the lives of others. The danger becomes apparent in Kuran's book-length exploration of preference falsification. He treats affirmative action in American society as a case of preference falsification, in which the "real" feelings of the silent majority were (are?) opposed to affirmative action, but were not voiced because "to voice misgivings is to invite censure. Conscious of the risks, Americans have tended to hide their reservations behind a veneer of public consent" (p 222), a claim that many would consider exaggerated, at least. He is quick to interpret an incorrect poll on the eve of Nicaragua's 1990 election as evidence of oppression by the Sandinistas over the previous decade. And while Kuran spends much time on preference falsification under Soviet Bloc communism, he has nothing to say about 1930s Fascist Europe. What does the concept of preference falsification have to say about lack of successful internal uprisings against the governments of Nazi Germany and Fascist Italy? Were these cases of falsified preferences that did not have a chance to be unfalsified, or did the populace "really believe" in the official ideology? And is the distinction between the two politically meaningful or did Hanna Arendt have it right when she wrote that "politics is not like the nursery; in politics obedience and support are the same".

The Arendt quotation comes from a persuasive essay by Xavier Marquez, "On the Meaning of Political Support". Marquez goes on to quote Robert F. Worth of the New York Times writing about the fall of Gaddafi: "Everyone in Tripoli, it seemed, had been with Qaddafi, at least for show; and now everyone was against him." Here is Marquez at length:

Were these people deceiving themselves or others? Did the soldiers really support Gaddafi in the past but now do not? Do some of these people support Gaddafi still? The question makes less sense to me than it once did. It is clear that they once obeyed Gaddafi and now do not… but to attempt to determine if, in their heart of hearts, these people supported Gaddafi then (net of all of these forces) and now do not seems slightly absurd. Their obedience and disobedience, support and lack of support are nothing but the vector product of all the forces (threats of coercion, positive incentives, beliefs about Gaddafi, idiosyncratic likes and dislikes, moral convictions, obscure and half-formed ideas about the future, etc.) operating through them. It may make sense to attempt to disentangle these forces if we are interested in legal or moral responsibility, or in the private tragedies of everyday life in Libya, but it does not make sense to me to attempt to figure out if Gaddafi enjoyed some "genuine" level of support (independent of coercion, money, etc.) as a separate explanatory factor.

It's not that there is nothing to preference falsification, but it has come to occupy a status of orthodoxy, and it has pushed other mechanisms into undeserved obscurity. In Steven Pfaff's excellent "Exit-Voice Dynamics and the Collapse of East Germany", which presents a rich and multifaceted account of the events of 1989, I could not help but see a tension: it seemed to me that he falls back on preference falsification as explanation more often than he would like to (although he does point to other mechanisms) because of the absence of alternative frameworks in which to think.

Safety in numbers?

Finally, somewhere in the middle of most cascade models there is a "safety in numbers" assumption. Lohmann (on p92 of her major article) writes that the cost of protest "is assumed to be decreasing in the turnout… This assumption can be motivated with the 'safety in numbers' characteristic of the technology of suppression: given the amout of resources a regime devotes to suppressing mass protest, it is plausible that a higher number of activists is associated with a lower likelihood that any one activist will experience injury, death, or imprisonment." Kuran writes "The external payoff to siding with the opposition… is apt to become increasingly favorable… with S (the size of public opposition). The larger S, the smaller the individual dissenter's chances of being persecuted for his identification with the opposition."

This assumption is weak. An action that the government could afford to ignore during a period of political calm cannot be ignored if it gets big enough. Tienanmen Square did not become a safer place the more people gathered, and neither does Tahrir Square: perhaps quite the opposite. Many protests are deliberately risk-seeking, actively looking to provoke a response from the government. As Gandhi said, "The function of a civil resistance is to provoke response and we will continue to provoke until they respond or change the law."

Do people join large protests because it is safer than joining small protests? Or do they join them because, for example, large protests simply matter more? When a country polarizes, the central issues become deeper dividing lines between supporters of the status quo and those in opposition. The question "which side are you on?" becomes one that has to be answered.

Wrapping Up

What I've tried to say is that there are underexplored weaknesses in the cascade theory description of uprisings which have continuing impacts on current debates, not only in the academic world, but in the wider world - even including the decisions and actions of dissidents in perilous circumstances. The weaknesses lie in the notions of preference falsification, in assumptions about safety in numbers, in assumptions built into the network models constructed on top of cascade models, and in blind spots regarding the forms of political action, the importance of symbols, the roles of institutions, the formation of identity, and other factors that shape the events around sudden uprisings. The effects of these weaknesses are amplified by the enthusiastic adoption of cascade models, supplemented by loose analogies and anecdotes as if they form part of the theory itself, and have a distorting effect on our understanding of these events, making it easy to see some patterns and difficult to see others. Of course, it would be easier to counter the influence of information-driven cascade theories if there were alternative approaches that also reproduced the dramatic "cascade" results. More on that next time.

 Org version 7.8.10 with Emacs version 23

When Theories Matter: Uprisings in Authoritarian States

(Second in a series of hopefully accessible posts about this hard-to-read paper).

From time to time, sitting in a comfortable chair with a cup of coffee reading disputes about Twitter Revolts and Facebook Revolutions, it is easy to think that The Argument is the Thing. But it isn't, of course. The public profile of these debates about how digital technologies intertwine with dissent in authoritarian states, sprawling from the pages of the New Yorker to Foreign Policy, from specialized academic journals to urgent pamphlets, means that the arguments may influence the choices of dissidents operating in perilous environments; may sway them one way or another as they make life-changing decisions. So the least we can do, even those of us on the fringes of the debates, is to try for the truth.

Particularly strange, perhaps, is that these disputes are unavoidably theoretical. Of course, it matters greatly to tell a coherent and accurate story of how events played out in each particular case, but the implications of the debates are most urgent for uprisings that have not yet happened and for protests that have not yet been organized. No matter how exhaustively one recounts the unfolding of events in Tunisia in 2011 and 2012, such a telling alone cannot provide guidance to dissidents in Azerbaijan, in Russia, or in Canada for that matter. We cannot avoid theory: we point to a set of mechanisms and say "this happened here because of these conditions and for these reasons. In other conditions in different places, here are the possible outcomes". Talk turns inevitably to what is primary and what is mere epiphenomenon. We tell stories that highlight what seems essential and draw attention away from the factors that we deem unimportant. We theorize, and we tell stories based on those theories. We retell anecdotes that encapsulate those theories to prove our points.

When it comes to questions of digital technologies and their roles in political change, and particularly dissent in authoritarian states, there is a whole vocabulary that has come into being that carries along a set of narratives. There are assumptions behind terms such as "digital activist", "internet freedom", and "network society" that carry over into the stories and the habitual grooves of thought that we take with us as we try to understand new developments.

One current that has been particularly influential in the debates around the "Arab Spring" uprisings is based on the idea of informational cascades, and this current provides the theory behind much of the optimistic talk about Facebook revolutions and the potential for digital technologies to undermine authoritarian regimes. It's not the only theory of uprisings, but it is a compelling one. So if you're going to tell a different story about digital technologies, you need a different theory. I went looking for what that theory should be, and didn't find one, so I put one together myself, and that's what these posts are about.

Next: a look at information cascades and what they leave out.

Writing Towards Zero

Five months have passed since I posted here because I found myself drawn to finishing an earlier project. I thought it would take a couple of weeks but it ended up taking five months. The end result is called "Identity, Institutions, and Uprisings", is available at SSRN, and over the next few weeks I'm going to provide a more accessible, more political, and less academic version of the project here. I suspect the potential audience is vanishingly small but hey, I'm pleased with it. I'll start tomorrow.

Open Data at Crooked Timber

Prompted by the flurry of activity around here, Henry Farrell of the highly-regarded Crooked Timber blog has organized a seminar on Open Data. In Timber-speak a seminar is a series of posts over the course of a week or so by a variety of guest bloggers, together with comments from the CT crowd, who are a very smart crowd indeed. So I'm thrilled at the seminar, and even more thrilled to be the first contributor; my contribution is Seeing Like a Geek.

Open Data Movement Redux: Tribes and Contradictions

Table of Contents

• 1 Introduction
• 2 The Open Government Data Landscape
• 3 Open Data: Is it a Movement?
• 4 Do Civil Liberties and Privatization Belong Together?
• 5 Civic and Commercial Interests: Complement or Conflict?
• 6 Summary

1 Introduction

I have two things to say to those who responded to Why the 'Open Data Movement' is a Joke:

1. Thank you for putting so much effort into providing such thoughtful, reflective, articulate affirmations of your point of view. You gave me (and others, I hope) a lot to think about, and a lot to read over the last several days.
2. Unfortunately, you're still wrong.

The original post was written in the heat of the moment, so here is a more detailed and considered, and therefore almost certainly less-likely-to-be-read, argument about the contradictions and problems of the "Open Data Movement".

2 The Open Government Data Landscape

First, here is a map of Open-Government-Dataland (click for a larger popup).

The longitude, marked across the x axis, indicates the impact of the data itself. The line x=0 is the Yu-Robinson Meridian¹ and separates Open-Government-Dataland into an eastern and a western hemisphere, each populated by a spectrum of data. Harlan Yu and David G. Robinson explain the hemispheres this way:

"A machine-readable bus schedule aims to promote convenience, commerce and a higher quality of life—it enhances service delivery. Disclosures of public contracting opportunities play a dual role, potentially enhancing both economic opportunity and public integrity. And core civic data, such as legislative or campaign finance information, serves a more purely civic role, enhancing transparency."

Tom Lee describes these two hemispheres as "'open-as-in-data.gov' and 'open-as-in-FOIA'".

The latitude, marked along the y axis, marks the incentive of the data users. Travelling from south to north along a meridian takes us from the purely commercial activities in the south to the purely non-commercial in the relatively poor northern lands. At y=0 we cross the Bates Parallel, named after Jo Bates who has this to say about the conflict between the commercial and non-commercial hemispheres:²

"Interviews and observations suggest that some at the periphery of the OGD [Open Government Data] initiative have tended to conceptualise OGD as being about small start-ups, voluntary 'civic hackers', and other micro/small enterprises. This is unsurprising given the heavy weighting towards micro/small businesses in the UK's IT sector and the large number of 'civic hackers' active in the OGD community; however, the potential re-user industry for OGD is broader than this. The PSI [Public Sector Information] re-use industry comprises a range of industries and includes multinational corporations (MNCs) such as Google and LexisNexis, conglomerates such as Daily Mail and General Trust whose DMG information division is the parent company of the UK based Landmark Information Group, as well as an array of SMEs, micro enterprises, independent developers, and voluntary 'civic hackers'."

While this landscape is rich, the inhabitants of different quadrants of the map can be distinguished easily:

• In the harsh mountains of the North-West³ live the hardy non-commercial "civic hackers" whose diet consists of transit timetables, weather forecasts, and other pragmatic, useful data obtained, often piece-by-piece, from municipal governments. The culture of the civic hackers displays a combination of civic goals and enjoyment in the intrinsic interest of programming, with a twist of start-up mentality. It is a tribe described vividly by David Eaves⁴: "geek, technically inclined, leaning left, and socially minded. There are many who don't fit that profile, but that is probably the average." This tribe includes organizations such as Code for America ("A new kind of public service") as well as small and loosely-coordinated groups of individuals working at a local level and who identify with their role as citizens over their role as consumers.
• The North-Eastern tundra is the home of civil liberties activists, a resourceful tribe bent on promoting government transparency. It is driven by a desire to make information such as lobbying activities, campaign funding, government operations, and legal statutes open and accessible. In the USA you will find organizations such as Sunlight Foundation ("Making Government Transparent and Accountable") and Public.Resource.Org ("Making Government Information More Accessible") living here. Tom Lee's response⁵ and Carl Malamud's comments hail from these lands.
• In the warmer lowlands of the South-East, food is a little more plentiful. Here you can find a long-nosed tribe of commercial organizations and individuals who get paid to hold government accountable: the tribe of "data-driven journalists" which focuses on working with data to carry out its function.
• Finally, by far the most comfortable quadrant in the land is the lush pastures of the South-West, where a tribe of comfortable and well-nourished commercial organizations lives. Some members of this tribe are small and fast moving, but others have grown to giant size and sport odd names such as Google, Microsoft, or ESRI. One member of this tribe, Socrata, wrote another retort to my post of last week.

Not everyone lives in a single spot; some organizations are nomadic, wandering from quadrant to quadrant. As just one example, mySociety in the UK has both a commercial and a charitable wing; it runs both transparency-oriented projects such as They Work For You, and service-oriented projects such as Mapumental, which is a commuter-mapping service offered as a commercial product. O'Reilly Media lives in the South-West, but Alex Howard (a member of that tribe) spends considerable time in the South East.⁶

3 Open Data: Is it a Movement?

I hope the geography tour is pretty uncontroversial, and that it helps to orient ourselves with respect to the three claims I made last week about the "Open Data Movement":

1. It's not a movement in a political or cultural sense of the word.
2. It's doing nothing for transparency and accountability in government.
3. It's co-opting the language of progressive change in pursuit of a small-government-focused subsidy for industry.

Of these three, I stand by about two and a half, although I do agree that the wording is sloppy and could be misleading. In my defense, I wrote quickly, expecting attention from my usual handful of readers (you know who you are; thanks for hanging around.) and not the much bigger audience that the post ended up attracting.⁷

It would take too long to engage in a defence of each of these claims⁸ so instead I'll set out what I see as the contradictions and confusions that come out of labelling all four tribes that inhabit Open-Government-Dataland as a single movement, and distinguishing them from tribes who do not appear on this map: those share a similar interest (eg, Civil Liberties groups and journalists) but who do not focus on data. I see little coherence in the interests or priorities of the Open-Government-Dataland tribes; in particular the giants of the south-west cast a dark shadow over the other quadrants, and the other tribes may have to drive them out of Open-Government-Dataland or succumb to their hegemony.

4 Do Civil Liberties and Privatization Belong Together?

Encyclopedia Britannica says that a social movement is a "loosely organized but sustained campaign in support of a social goal" and that's the definition I'll stick with. So what's the social goal of the Open Data Movement? There is a technological goal, spelled out a few years ago by some of its leading lights in terms of the formats, timeliness, completeness and licensing of the data,⁹ but what is its social goal? Pretty much any description I've seen gives two separate goals: improved government efficiency and transparency, corresponding to the west and east hemispheres of Open-Government-Dataland.

Being in favour of efficiency and transparency is a bit like being in favour of chocolate and cheese: both are good, but it's not clear that they have very much to do with each other. But the problem is deeper than this: Open Data advocates argue not just for efficiency, but for a particular vision of "efficiency" captured by Tim O'Reilly's phrases "Government as a Platform" and "Gov 2.0".¹⁰ This vision places the interests of "the public" or "the people" on the same side as corporations and in conflict with those of the state. The thinking of Open Data advocates is open to the same kind of critique that Jodi Dean makes today about Adbusters' Kalle Lasn: that an apparent populist leftism disguises (intentionally or otherwise) an economically neoliberal agenda.

"Lasn misrepresents the economic problem of neoliberal capitalism as a division between neoclassical economics and the "new ecological or bionomic or psychonomic discipline that is bubbling underneath the surface." Now maybe I just don't know what he's talking about, but it looks to me like the sort of stuff that is usually wrapped up as complexity theory, with all its talk about emergence and swarms and self-organization and criticality (I talk about this in the first chapter of Blog Theory). It's the same set of ideas part of New Economy thinking, which isn't opposed to neoliberalism at all but was a primary carr[ier] of it, especially insofar as regulation is bad and free flow is good. Thomas Friedman, after all, is like the poster boy of horizontality–The World is Flat"

It does seem to me that the ideologically neoliberal aspects of "Gov 2.0" have not been absorbed by some of those in the civil liberties tribe. Tim O'Reilly, for example, is both ambitious in his small-government vision ("Government 2.0… is government stripped down to its core, rediscovered and reimagined as if for the first time") and explicit in his commitment to market-based delivery of services. He approvingly quotes David G. Robinson's Government Data and the Invisible Hand:¹¹ "Private actors, either nonprofit or commercial, are better suited to deliver government information to citizens". When it comes to healthcare he writes "[government] should not [take part] by competing with the private sector to deliver health services, but by investing in infrastructure (and 'rules of the road') that will lead to a more robust private sector ecosystem".

The "Government as Platform" vision is even more market-driven than that of the "Cambridge Study" reported by Jo Bates (link), and to which Rufus Pollock of the Open Knowledge Foundation contributed. As Bates says, the Cambridge Study argued for "unrefined digital data to be available for re-use at marginal cost (general zero for digital resources), whilst the charging regime on refined PSI products should remain intact. These refined products, it is argued, would then be in fair competition with other suppliers, since there would be equal access to unrefined data inputs… In a further paper, Pollock goes on to argue that the optimal charging model would be direct state subsidy or, in some cases, charges to update the database. These economic arguments thus draw on a liberal economic paradigm with strong emphasis on supply-side policies based on removing constraints on commercial production through liberalisation and marketisation, combined with taxpayer subsidisation of infrastructural resources such as data."

Jo Bates's paper This is what modern deregulation looks like (link) explores the contradictions between the efficiency and transparency hemispheres in a thorough and lucid way and really you should just read that if you want a better-informed version of my own views. Here is one of the more abstract and general sections, that sums up her claims:

"the current 'transparency agenda' [of the UK government, supported by prominent Open Data advocates] should be recognised as an initiative that also aims to enable the marketisation of public services, and this is something that is not readily apparent to the general observer. Further, whilst democratic ends are claimed in the desire to enable "the public" to hold "the state" to account via these measures, there is an issue in utilising a dichotomy between the state and a notion of 'the public' which does not differentiate between citizens and commercial interests… The construction… encourages those attracted to civic engagement into an embrace of solidarity with profit seeking interests, distanced from the ever suspect notion of the state."

The "Government as Platform" vision widely accepted among Open Data advocates¹² thus overlaps significantly with the views of the UK government quoted by Jo Bates, culminating in Francis Maude's statement that Open Government Data is "what modern deregulation looks like". Is this neoliberal deregulation a vision that Tom Lee and David Eaves support?

The transparency agenda has been used by the PSI Reuse industry and by right-wing governments as a camouflage for other, economically neoliberal goals. Tom Lee describes the Open Data Movement as a "self-described nonpartisan activist movement" but while I accept his argument that the Civil Liberties tribe, including the Sunlight Foundation, are non-partisan (and, yes, are a movement), I do not think his characterisation can carry over to other tribes. The support, tacit or otherwise, of the Civil Liberties groups for the "Government as Platform" agenda, means that the Sunlight Foundation is promoting a neoliberal economic position with which its members may not agree.¹³ I don't dismiss the views of Kevin Merrit, CEO of Socrata, as "self-serving and profit-motivated" when he argues that the Open Data Movement has promoted transparency, but I do believe there is a conflict of interest (which is a structural fact, not a personal quality) between arguing for an Open Data policy and then making money by providing software to implement that policy. It's a conflict that makes episodes such as New York City's unfortunate release of individual teacher assessments more likely.

5 Civic and Commercial Interests: Complement or Conflict?

Most Open Data advocates don't phrase the issue in terms of private-sector provision of services, but instead phrase it in terms of civic engagement, non-profit groups, and "people". Tim O'Reilly often phrases his arguments purely in terms of a civic public (and may see it that way himself), as in "This is the right way to frame the question of Government 2.0. How does government become an open platform that allows people inside and outside government to innovate?"

Carl Malamud goes further, arguing that the Open Data Movement is a replacement for a regime in which "the commercial sector is raping and pillaging the public treasury, getting exclusive deals on data that not only keeps out other companies, but researchers, public interest groups, and everybody else who make up 'the public.' In many cases, the government data is so tightly behind a cash register that even government workers enforcing the law can't afford to buy copies of the data they produce or the rules they promulgated." Others see no conflict between commerce and civic activity in this area: Tom Lee writes "I think it's flatly wrong to consider private actors' interest in public data to be uniformly problematic."

David Eaves makes a strong argument for the vitality of the civic hacker tribe, and points out that Open Data has been largely ignored by Canadian corporations (although US companies such as Socrata have gained contracts for providing municipal "open data platforms"). The Canadian environment may be like that of the UK, where there is a "heavy weighting towards micro/small businesses in the … IT sector" to quote Jo Bates. Similarly, there are just are not that many Canadian companies deeply involved in government operations or in the use of public data.

(There have been positive statements from Open Text, and Desire2Learn has sponsored an "Edge Challenge" that has attracted app developers using open transit data [disclaimer: in my day job I have had some tangential involvement in that competition], but I can see what he means.)

So why would I focus on the private-sector, market-based actors of the south west quadrant when the civic hackers are perhaps more prevalent? Because of an argument made a year or two ago by Michael Gurstein, who asks "who is in a position to make 'effective use' of this newly available data?" and answers himself:

"'open data' empowers those with access to the basic infrastructure and the background knowledge and skills to make use of the data for specific ends. Given in fact, that these above mentioned resources are more likely to be found among those who already overall have access to and the resources for making effective use of digitally available information one could suggest that a primary impact of "open data" may be to further empower and enrich the already empowered and the well provided for rather than those most in need of the benefits of such new developments."

Data's value is combinatorial. It is most powerfully used by those who can combine it with other sources of data and who have the scale and resources to use it effectively. I think it's fine that "civic hackers" are developing transit apps, but in the end that market is likely to be won by a single company under the current licensing and standards approaches.¹⁴

While Open Data advocates appear "open" to many new ideas, everything I've read suggests that they are near-united on the principle of "non-discriminatory" licensing, meaning making data available to commercial enterprises (of any size) on the same terms as to the Civic Hackers. The economy of data-driven products is similar to the economics of cultural industries: it tends to end in winner-take-all outcomes and favours large-scale enterprises. In cultural markets, this tendency has led many countries to adopt a toolbox of techniques to maintain domestic cultural industries in the face of the scale of the American cultural industry, from quotas to subsidies to non-market providers.¹⁵ Such measures have much in their favour, yet the Open Data Movement is apparently united in opposing them.

Economically, Silicon Valley is likely to be the major winner in the Open Government Data game. It is difficult to see how to justify a subsidy to Silicon Valley companies as a priority for cash-strapped governments of smaller countries.

An example: Jo Bates (again) describes the interest in weather data. "In the context of the UK there has been significant lobbying by the financial industry to get better access to UK weather data so that it is able to compete in this [weather risk management] market. Groups such as the Lighthill Risk Network, of which Lloyds of London are a member, have lobbied government for better weather data so that they can develop risk based weather products. Similarly, the insurance industry has requested real time information on the pretext that they might respond more quickly to extreme weather events such as flooding. My own research and the recent announcement suggest that these demands have been met enthusiastically by well placed policy makers in national government who are keen to develop a UK weather derivatives market." Weather risk management might seem like an odd duck, but Bates reports that "This weather risk management market far outweighs the USA's commercial weather products market which in 2000 was estimated at approximately $500 million a year", touching over $45 billion in 2005-06.

The rhetoric of civic engagement is appealing, but blurring the boundary between small-scale civic "hackathons" and the major financial institutions is a position that simply ignores major economic and political issues.

The benefits of standards-driven formats are, for municipal activities, not obvious unless you want to attract global interest. I continue to believe that licensing and formats are an area where there is still room for innovation, and where a premature focus on standardization may shorten the lifespan of civic-hacker use of municipal data before the big players get to pull it into their own systems. I'd argue, as I have before, for some form of charging to be enabled, at least on large-scale commercial use of data. I'd also argue that standardization should not be high on the agenda for municipal governments looking to build and collaborate with a local community of hackers.

6 Summary

Let me return to my three claims:

1. It's not a movement in a political or cultural sense of the word.
2. It's doing nothing for transparency and accountability in government.
3. It's co-opting the language of progressive change in pursuit of a small-government-focused subsidy for industry.

I'd argue that (1) holds: there is simply too much incoherence, too much in the way of conflicting interests and non-overlapping goals, for the "Open Data Movement" to be a movement. And see also the footnote.¹⁴

Item (2) is harsh. There are many within the "Open Data" sphere who live in the Civil Liberties area who have made significant contributions to transparency and accountability. But as a net effect, I'd stand by the claim – the overall impact of open data initiatives could well be to promote a kind of government that is prone to secrecy, as "small government" parties have often been.

Item (3): the co-option is being done by a vocal and influential section of Open Data advocates, but I'd definitely hold to the claim that the language of progressive change is being used, and the actions of civil liberties activists used, by some whose agenda is closer to neoliberal than egalitarian.

If you are still with me after all that; thanks for reading.

Footnotes:

¹ Harlan Yu and David G. Robinson, /The New Ambiguity of "Open Government"/, Working Paper, draft of Feb 28, 2012. Retrieved from SSRN.

² Jo Bates, /"This is what modern deregulation looks like": co-optation and contestation in the shaping of the UK's Open Government Initiative/, The Journal of Community Informatics, 8 (2). Retrieved from ci-journal.net.

³ This article adopts a northern-hemispherical hegemonic worldview.

⁴ David Eaves, Open Data Movement is a Joke?, May 2, 2012. Retrieved from eaves.ca.

⁵ Tom Lee, Defending the Big Tent: Open Data, Inclusivity and Activism, May 2, 2012. Retrieved from sunlightfoundation.com

⁶ Alex Howard, No joke: Open data fuels transparency, civic utility and economic activity, May 2, 2012. Retrieved from govfresh.com.

⁷ If I had know the audience was to be so large, I would have written more cautiously, and then the audience would not have been so large.

⁸ While I don't want to trespass on everyone's attention for that length of time, if you are interested in discussing these do send me an email (tslee at web dot ca) and I'd be happy to respond.

⁹ Open Government Working Group, 8 Principles of Open Government Data, 8 December 2007. Retrieved from opengovdata.org.

¹⁰ Tim O'Reilly, Government as Platform, Chapter 1 of /Open Government: Collaboration, Transparency, and Participation in Practice/, Retrieved from oreilly.com.

¹¹ David Robinson, Harlan Yu, William P. Zeller, and Edward W. Felten, Government Data and the Invisible Hand, 11 Yale J.L. & Tech. 160 (2009). Retrieved from SSRN.

¹² The only dissenting view I've seen that tackles it explicitly, apart from my own, is that of Andrea DiMaio, who also has some smart things to say here.

¹³ On this I disagree with Catherine Fitzpatrick, who shares some of my views about the Open Data movement and argues forcefully for them here. I appreciate Catherine's robust arguments in the debate, but she does come at this from a very different political point of view to my own, as her comment on Tom Lee's post makes clear.

¹⁴ Aside: David Eaves' description of municipal-level civic-hackers (North-West quadrant) in Canada is compelling, but this group of people is also Not a Movement. Now some people claim not to care about the word "movement", and if you don't then skip back to the main text, but I think it matters.

There are many admirable socially and civically beneficial activities that are not movements. Many people coach children's sports teams (I've done it myself); millions of people take part in such activities, they are committed, involved, and do a lot of civic good, but they do not form a movement: I was a soccer coach, not a public sports activist. Similarly, birdwatching has a long and distinguished history of contributing to social goals (protection of birds and their habitats) and of sharing their observations in socially and scientifically useful ways (my brother Dorset Dipper contributes to the Hertfordshire Bird Atlas) but birdwatching is a hobby, not a movement. This thing about technologists claiming to be a movement is something that, perhaps irrationally, irritates me. Calling "Open Data" a movement is not quite as daft as calling "NoSQL" a movement, despite the arguments of O'Reilly's Mike Loukides, but to my mind invoking a "movement" is a way to give added weight and significance to activities that may be admirable and useful, but that are ultimately uncontentious: it smacks of self-importance and that rubs me the wrong way.

¹⁵ For a fine description of the economics of cultural products, and the toolboxes that smaller economies have used to maintain cultural diversity, see Blockbusters and Trade Wars by Peter S. Grand and Chris Wood, Douglas & McIntyre 2005. Link.

Date: 2012-05-08 21:40:38

Org version 7.8.06 with Emacs version 23

Is philosophy a joke?

No, not a continuation of the theme of the previous two posts.

My son, who has been pursuing a degree in philosophy over the last couple of years with steadily mounting frustration, has decided to "drop out", work for a while, and consider where he goes next. He signs off his blog here:

Although it is unfortunate to quit before I can figure out whether (the vast majority of) philosophers are actually as staggeringly incompetent as they appear to be, or are simply playing an elaborate practical joke, I simply couldn’t stand it either way.

Yes, he's disenchanted. I'm proud of him and his efforts to find a way to apply reason to important questions about life, and I've been dismayed to watch the discipline of philosophy lose someone as motivated as he has been, punishing originality rather than encouraging it, pandering to intuition, and giving up on reason while spending its time on issues such as whether proper names are rigid designators.

I'm no philosopher, but while I look forward to seeing what my son does next, I have no inclination whatsoever to read Naming and Necessity.

Reactions to 'Why the "Open Data Movement" is a Joke'

[Update: In the light of morning I dislike this post. I'll leave it up, but it is too defensive, explains too much, and is too conciliatory given what was thrown at me yesterday.]

This morning's post, Why the "Open Data Movement" is a Joke, attracted more attention than most of what I've written here. Largely this was a result of a Twitter debate between Evgeny Morozov (@evgenymorozov) and O'Reilly Media's Alex Howard (@digiphile). Thanks also to Lorenz Matzat (@lorz) and Ryan Shaw (@rybesh) for arguing broadly in favour of the post.

Alex Howard hated the piece, calling it "ill-informed", "lazy, ignorant writing" that "didn't even bother to cite the relevant scholarship", "demonstrably incorrect", "laughable" and more. He also writes that "The author has a habit of writing polemics that include errors or omissions of fact." I am terrible at expressing anything in 140 chars so I'll respond here.

First, it should be obvious that the post was prompted by events here in Canada - and yet no one has actually mentioned any of the Canadian content in any of the comments about the post. This is frustrating. The last five years have been terrible ones for accountability and transparency in this country and yet Canada has just joined and endorsed the Open Government Partnership (link). Does this conjunction say anything about "open data" as a goal? To me it says that a technological "open data" agenda does not indicate a political "open data" agenda, and that -- as I wrote about the Wikileaks cables a year ago (link) --the fault lines of political beliefs run perpendicular to attitudes about technology, not parallel. So it made me wonder if the idea of open data as a goal for a coherent movement holds water, and whether "opening government via technology" make sense. To Alex Howard - well I don't really know, because for all his outrage he doesn't actually say anything about the first half of the post or about the events that obviously moved me to write it and led to the frustrated tone I wrote it in.

Beyond that, he seems to confuse my contention that the idea of open data as a "movement" is a joke with a broader claim that "open data" is a waste of time or that people working on making data open are all dupes. No such thing! Open data can be a fine thing, but I'd much rather have a fully-staffed StatsCan charging for data than a half-staffed StatsCan providing it for free. Which would he choose? Obviously a fully-staffed StatsCan providing data for free would be ideal, but it doesn't look like we're getting that any time soon. The UK's Francis Maude, the incoming co-chair of the Open Government Partnership, says that "we want to create an army of armchair auditors who can hold government to account". This would be nice, but not at the cost of a real independent auditor.

I suspect that Alex Howard and I just see the world from different points of view. Que sera, sera. But from what I know the Sunlight Foundation is on the side of the angels so I was disappointed to see that Tom Lee of the Sunlight Foundation considered my post "a jumbled mess". I do realize that coalition politics makes strange bedfellows, and that broad coalitions can still be worthwhile, but some tents can be so big that they collapse in a shapeless pile of canvas. I worry that the open data tent is one such, and that the apparent common goals of some people under the canvas hide bigger differences.  I share the concerns of Alex Howard's colleague Nat Torkington when he writes this:

Obama and his staff, coming from the investment mindset, are building a Gov 2.0 infrastructure that creates a space for economic opportunity, informed citizens, and wider involvement in decision making so the government better reflects the community's will. Cameron and his staff, coming from a cost mindset, are building a Gov 2.0 infrastructure that suggests it will be more about turning government-provided services over to the private sector.

For me, the gap between the two visions is fundamental and makes the idea that these two goals are part of the same movement, well, a joke. The tension/contradiction between commercial and civic interests that these sentences highlight is one division that seems unresolved and yet fundamental. So having read the responses, I stand by what I wrote.

Why the "Open Data Movement" is a Joke

 Two recent announcements from Canada prompt my mood this morning:

• Canada Joins International Open Government Partnership
• Conservative Cuts put half of Statscan jobs at risk

A government can simultaneously be the most secretive, controlling Canadian government in recent memory and be welcomed into the club of "open government". The announcements highlight a few problems with the "open data movement" (Wikipedia page):

• It's not a movement, at least in any reasonable political or cultural sense of the word,
• It's doing nothing for transparency and accountability in government,
• It's co-opting the language of progressive change in pursuit of what turns out to be a small-government-focused subsidy for industry.

In short, the open data movement is a joke. Those who are on the political left who lend their support to it have some hard decisions to make.

The Canadian Case: Open is an empty word

The Harper government's actions around "open government", and the lack of any significant consequences for those actions, show just how empty the word "open" has become. For readers outside the country, here is a selection:

• Cancelling the compulsory long-form census (link), thereby demolishing a source of reliable statistical data that guides government decisions and debates over national priorities. "The information previously collected by the long-form census questionnaire will be collected as part of the new voluntary National Household Survey (NHS)." The decision prompted Statscan head Munir Sheikh to resign (link).
• Firing Health Canada scientists who speak publicly on drug safety issues (link). [Update: as "d" comments below, this was the Liberal government in 2004]
• Muzzling Canada's public scientists in other departments, with one example being Fisheries and Oceans scientist Kristi Miller (link). More broadly, "Natural Resources Canada scientists have had to get pre-approval from Minister Christian Paradis's office to speak with journalists. They must also get ministerial approval for everything they say to the news media." (link).

It's got to the stage where the Canadian Association of Journalists recently awarded its "Code of Silence" award for Canada's most secretive government or publicly-funded agency to the entire federal government (link).

While there has been opposition to these moves, I think it's fair to say that the "open data movement" has not been central to it. But never mind, Statistics Canada data is now available for free on the government's web site (link). There seems to be no link between the government's actions and the actions of this "movement", and basically that's because the Open Data Movement is more focused on formats, digitally-acessible data sets, free access to postal codes, and so on than it is focused on actual government transparency around issues that matter. It's a movement that has had no impact on government accountability.

Who is the Open Data Movement?

Am I being unfair? Who, after all, is the Open Data Movement? Well it turns out there isn't one really, at least when it comes to "open data" in the sense of "open government data", which along with "open scientific data" is one of the two most common uses of the term.

"Open Data Movement" is a phrase dragged out by media-oriented personalities to cloak a private-sector initiative in the mantle of progressive politics. Along with other cyberculture terms ("hacktivism", "unconferences", "hackathons") the word "movement" suggests a countercultural grass-roots initiative for social change, but there isn't anything of the sort that I can see.

Take Tim O'Reilly, who has thrown the phrase around for some time (see here for an example from a couple of years ago). Like others who use the phrase, he sees no conflict between civic culture and corporate interests, so the Strata conferences and Open Government conferences he has run have been sponsored by major software, hardware, and computer services companies (including, I think, my employer, for whom I do not speak). Strata 2012, for example, is co-hosted by Cloudera, sponsored by EMC and MapR, and many others.

Or take the "Code for America" initiative, which uses language that is explicitly about promoting an alternative vision of how government works ("it's about citizenship and how the internet is fundamentally reshaping the way government can work", It's "a Peace Corp for geeks") and which has many well-intentioned people involved. Yet when it comes to it, there's a lot more here about making uncontroversial data available (including for commercial use) than there is about anything like challenging government on actual accountability or transparency. So it's no surprise that the list of donors includes major corporations like EMC (again), ESRI, Google, O'Reilly Media, and Microsoft.

It's not that there's necessarily anything wrong with Code for America, more that it's not a movement in any political or even cultural sense. Another member of the CfA donor list is the Omidyar Network, set up by the eBay billionaire Pierre Omidyar, and it reflects his view that private sector corporate profit-making activity and civic activity are not in tension, but complement each other.

As a result, the actual activities of this "movement" end up being to push for government subsidies of private-sector activity. It's "big society" all over again. This is the TED worldview, so it's no surprise that the recent Open Government Partnership wraps itself in noble goals such as fighting "corruption, closed doors, the consolidation of power" (see Hillary Rodham Clinton's remarks) and basks in the reflected virtue of TED fellow Walid Al-Saqaf (Open data vital for a new Yemen) when the most likely outcomes are privatisation initiatives of the kind promoted by Francis Maude.

Abandoning "Free for Commercial Use"

Progressives involved in open data work, of which there are many, could do something useful here. In order to prevent actions around government transparency being hijacked as a front for corporations to get at subsidised raw material, they could promote a "non-commercial use" license of the kind that is an option under some variants of Creative Commons content license.

Until now, groups and individuals with some credibility on the left have maintained constructive if arms-length relations with the corporate/civic almagam that is the Open Data Movement. It's time for them to draw a line, and not let their own often-admirable initiatives get used as a smokescreen for privatisation and small-government initiatives.

Are any doing so, or are there explicitly-progressive initiatives that are making a difference? I'd be interested to hear.

More reading

Basically, this post follows on from this and this, which I wrote a couple of years ago.

Click to Judge

I recently contributed an essay to the always-excellent Literary Review of Canada, which is out in the April issue and is one of the contributions that they have put online so you can read it in its entirety for free here, you lucky people. It's a review of an essay collection called "The Reputation Society", edited by Hassan Massum and Mark Tovey. Unfortunately I didn't like the book much. The concluding paragraph of my review is this:

There is a need for inventive and serious thought about the issues of reputation and trust in an increasingly digital world. Our social and commercial interactions will be increasingly mediated by large-scale software systems, and we need ideas about how best to design, navigate and regulate these systems. Unfortunately, by avoiding real-world cases and thorny problems, The Reputation Society provides no answers.

My starting point for the review, although it's not explicit in the essay, is that information asymmetry is at the heart of anything to do with trust, reputation, and the market; and despite what some optimists claim, sheer volume of opinions does not solve the problem. Recently, I even deleted this paragraph from the Wikipedia article on Information Asymmetry:

Although information asymmetry has recently been noted to be on the decline with the rise of the internet, which allows ignorant users to acquire hitherto unavailable information such as the costs of competing insurance policies, or the price of used cars, it is still heavily applied to human resource and personnel economics regarding incentive schemes when the employer cannot continually observe worker effort.

"The costs of competing insurance policies, or the price of used cars" has nothing to do with information asymmetry. The question is, how do you know if I'm telling the truth, when I have an incentive to distort it? How can we establish trust?

Wikipedia itself provides a useful case study. For many topics there is little incentive for a contributor to provide false information beyond the thrill of vandalism, and the fact that such vandalism can be undone with a single click means that trust is not a significant issue throughout many parts of the encyclopedia. Why would anyone lie about the birth date of Henry III of Castile being 4 October 1379? As a result, "anyone can edit" applies and the encyclopedia has become the phenomenon it is.

There is a small number of pages (several thousand of the 3.3 million in English Wikipedia, or 0.1%) that are the subject of repeated edit wars, and which "are semi-protected to reduce the risk of inappropriate editing". The policies and practices around these pages have undergone continual revision (Seth Finkelstein will know more) and the costs of maintaining neutrality are high; one way or another, when there is a conflict of interest, it needs special treatment.

When it comes to reputation, there's always a conflict of interest. A hotel has an interest in promoting its own image, and in worsening the image of its competitors. Simply throwing open the floodgates doesn't resolve this basic problem, and the paragraphs on TripAdvisor at LRC explain why. (That said, I have no doubt my cousin's place Riad Africa, in Marrakesh, is indeed a wonderful place to stay).

On the web, as elsewhere, there will be many ways explored to solve the proble of trust. Angie's List takes an approach that is almost the opposite of TripAdvisor in its reviews of contractors and services: it requires membership, and all reviews are checked by a human. A result is that, success or not, Angie's List is run much more like non-web reputation services: restaurant review guides, holiday home guides, and so on. If the main benefits of the web are free-for-all Wikipedia-style contributions, then it remains to be seen how reliable it can be for issues like reputation, where conflict of interest and asymmetric information are the sticking points. 

 

Wikibollocks: Mathew Ingram and Seth Godin on publishing

Here's most of Matthew Ingram's article about a Seth Godin interview with added intemperate and unedited and probably ill-judged commentary from yours truly. The Mathew Ingram article is indented.

Thanks to the rise self-publishing tools, from Amazon’s Kindle platform to Apple’s iAuthor software, anyone who wants to write a book can do so and theoretically reach an audience of millions — as self-publishing superstars such as Amanda Hocking and John Locke have shown.

This is meaningless. In one sense it's been the case for a hundred years that "anyone...can..reach an audience of millions". You didn't even have to go through a publisher (Virginia Woolf). But for most people, going through a publisher is easier than self-promotion, which for most people is very very difficult. So to claim "anyone" can self-publish to millions is just ridiculous. We can write off Salinger, Pynchon, Harper Lee, Samuel Becket, or Cormac McCarthy for a start. Promotion (not to mention editing and more) takes a certain kind of person. Not everyone can do it. I can't. 

But this explosion of amateur authors and publishers also means a lot more competition for an audience.

I'm sorry, "amateur authors"?? Most authors have always been amateur in the sense of not having a regular job as an author. If Amanda Hocking is an amateur (income from her books, millions) and I'm a professional (income from my book, hundreds) then someone is looking at the world upside down. Plus, in the first paragraph MI is implicitly claiming that it's easier than ever to reach an audience, and in this paragraph he's saying it's harder than ever (more competition). Make your mind up Mathew!

So how do writers make money? First of all, according to author and marketer Seth Godin, they have to give up the idea that they somehow deserve to be paid for their writing.

It's straw man time. Who are these authors? I know several authors, and none who feels the sense of entitlement of which Godin accuses them.

In a recent interview with Digital Book World, the writer and creator of the Domino Project — a joint publishing venture with Amazon that he recently wound up — was asked about his advice that authors should give their books away for free and that they should worry more about spreading their message and building a fan base instead of focusing on how to monetize it right away.

This is a model that will work for a very few people, but for most will fail. It's bad advice.

And how would he respond to writers concerned about their ability to make a living from their writing? Godin’s response:

Who said you have a right to cash money from writing? Poets don’t get paid (often), but there’s no poetry shortage. The future is going to be filled with amateurs, and the truly talented and persistent will make a great living. But the days of journeyman writers who make a good living by the word — over.

Anyone who thinks "the truly talented and persistent will make a great living" should have talked to Vincent Van Gogh. Are there still Van Gogh's in today's world? Of course - we just don't know who they are.

Perhaps musicians should look to digital royalties? Sadly no. Making money online is a winner-take-all competition, a few will get very rich (and luck has a big say in that, as the well-known Watts-Salganik experiments proved), but most will get somewhere between little and nothing.

This probably isn’t the kind of message that most authors (or creative professionals of any kind) want to hear, but that doesn’t make it any less true. The rise of the amateur, powered by the democratization of distribution provided by the Web and social media, is something that is disrupting virtually every form of content that can be converted into bits.

This is a mix of two poses. First there's hard-headed realism: the old world is gone, with a sympathetic wave of the hand. Then there's blind faith that everything is for the best in the best of digital worlds and please don't mention that "the rise of the amateur" is also "the rise of the plutocrat" (Bezos, Zuckerberg, Jobs) and "the democratization of distribution" is also "the monopolization of profits" in the hands of the aggregators. If you promote your work through Twitter, it's not "direct", it's mediated by Twitter, and it's as well to remember that. If you self-publish on Amazon then Amazon is your publisher.

To take just two examples, the news industry is struggling to adapt to an era where anyone can commit “random acts of journalism” with a blog or smartphone - 

"Random acts of journalism" sounds cool and fun, but cute phrases don't make reality. Matthew Hindman actually did some hard looking into the democratization of the media and concluded that the spectrum of voices getting heard now is narrower, if anything, than pre-Internet. The rise of the amateur citizen journalist is only half the story. (My comments on Hindman here: self-promotion!) Here is Hindman:

From the beginning, the Internet has been portrayed as a media Robin Hood - robbing audience from the big print and broadcast outlets and giving it to the little guys. But the data in this chapter suggest that audiences are moving in both directions. On the one hand, the news market in cyberspace seems even more concentrated on the top ten or twenty outlets than print media is. On the other, the tiniest outlets have indeed earned a substantial portion of the total eyeballs... It is the middle-class outlets that have seen relative decline in the online world. Moreover, it is overwhelmingly smaller, local media organizations that have lost out to national sources. (p.100)

And back to Mathew Ingram:

 — and where sources of news have the ability to publish their own content instead of having to go through a middleman — and photography has been battling the rise of the amateur for years now.

The "having to go through a middleman" link is to a piece about Rupert Murdoch opening a Twitter account. No longer does Rupert have to go through a middleman! Three cheers for the democratization of distribution! There are many sources of inequality, and attention is one. No one publicizes Jane Minor Author's twitter account, but Rupe - well he's different.

Nice picture.

The crucial principle at work in all of these areas is the idea that your real competition isn’t the book or news outlet that is better than you; it’s the one that is good enough for a majority of your audience.

So maybe the Huffington Post version of that news story isn’t as good as the one in the New York Times, but it is good enough for many readers. And maybe those vampire books by Amanda Hocking or the detective novels from million-selling author John Locke aren’t as good as yours, but for hundreds of thousands of weekend readers they are probably good enough.

So on the one hand we hear that "the truly talented and persistent" can still make money, but now we hear that quality isn't the issue. Well, one way or another I'm sure the new world will be just dandy. And what is that way?

Godin’s point isn’t that you can’t make money; it’s that you have to think differently about how to accomplish that task.

If you’re a mystery writer, can you find 1000 true fans to pay a hundred dollars a year each to get an ongoing serial from you?

The "1000 true fans" reference is to a Kevin Kelly daydream from 2008 (*). Plausible, but if it sounds like a plan to you, first read John Scalzi and then tell me that's the path you're going to take.

 It’s not the market’s job to tell authors how to monetize their work. The market doesn’t care. If there’s no scarcity of what they want, it’s hard to get them to pay for it.

The idea, implicit here, is that the market gives people what they want. Let's just agree to differ on that one.

Who says that artists have a right to make money?

No one I know. I'll skip the next few paragraphs.

As media theorist Clay Shirky has pointed out before, abundance breaks a lot of content-related business models that were built on scarcity, and that includes the ones that have supported the book-publishing industry for so long.

My response to the Clay Shirky piece on content-related business models is here.

That’s why publishers have been scrambling to try to lock down their content — including jacking up the prices that libraries pay for e-books — and it’s why authors who have a built-in audience are using the Web to connect directly with that audience.

You really want to talk about locking down content while pointing to publishing on Amazon and iBooks as a way forward? Come on Mr. Ingram, you know better than that.

Godin’s message may not be a popular one, but it is the way that content works now.

Again with the hard-headed technological inevitability. But politics, law, and culture has a lot to do with the way that content works. I have hopes that there will be room for many publishing models, with a collection of models - digital and physical. But the biggest threat to a diversity of revenue streams is a reliance on a single deterministic future governed by Amazon, Apple and Google. Fortunately, resistance to Silicon Valley's increasingly disconnected rhetoric is growing, and I think there are still more constructive possibilities for would-be authors than Seth Godin's self-help, positive-thinking brand of marketing.