« March 2008 | Main | May 2008 »

Linux Grows Up and Gets a Job

Linux started as a hobby project. But it is now 17 years old and it has grown up, and a recent report by the Linux Foundation, which extends a series of investigations by Jonathan Corbett, shows that it is no longer a hobby, it is out in the working world.

The report looks at the Linux kernel, and so does ignores all those other pieces of the operating system (drivers, applications, desktop interfaces and so on). Still, the kernel is the heart of the OS and "one of the largest individual components on almost any Linux system. It also features one of the fastest-moving development processes and involves more developers than any other open source project." So this report, which "looks at how that process works, focusing on nearly three years of kernel history as represented by the 2.6.11 through 2.6.24 releases." is a valuable window on what may happen to successful open source projects as they mature.

One of the highlights: "over 70% of all kernel development is demonstrably done by developers who are being paid for their work". 14% is contributed by developers who are known to be unpaid and independent, and 13% by people who may or may not be paid (unknown), so the amount done by paid workers may be as high as 85%. The Linux kernel, then, is largely the product of professionals, not volunteers.

So Linux has become an economic joint venture of a set of companies, in the same way that Visa is an economic joint venture of a set of financial institutions. As the Linux Foundation report makes clear, the companies are participating for a diverse set of commercial reasons. Some want to make sure that Linux runs on their hardward. Others want to make sure that the basis of their distribution business is solid. And so on, and none of these companies could achieve their goals independently. In the same way, Visa provides services in many different locations around the world in different sizes and types of stores. Some banks need their service mainly in one country, some in another, but when they work together they all get to provide their services all around the world.

This does not mean that Linux was always a commercial venture, or that all open source projects are commercial ventures. To invoke another parallel, open source software is a creative venture like music. Many people create music for all kinds of reasons. Most people create music for an audience of one when they hum in the kitchen or sing in the shower. A smaller (but still huge) number of people get together to form groups or participate in orchestras or bands. They don't earn a living from it, but they love doing it and enjoy their performances. Some might dream of hitting the big time, others are happy being part of their community. Then a much smaller set of people take it a step further. Maybe they are paid to be in an orchestra; maybe they are in a band with a manager and bar gigs around the country. And a lucky few, of course, hit the big time. They get a record deal, find a big audience, and make some real money.

So is music produced for love or for money? Both of course, in various proportions across the spectrum. Open source started off as a small-scale set of projects done mainly by volunteers. As the scale and scope of open source projects an increasing number have provided their contributors with some money (augmented perhaps by a waitressing job). Now a few of the most successful have hit the big time and become full-scale economically important commercial enterprises.

Things change. As open source software has matured and expanded it has become both more unlike the rest of the world and more like it. It will be fascinating to see what comes next, but the Linux Foundation report has made clear that open source has crossed its commercial Rubicon, and there is probably no going back.

Update: follow-up and response to some discussion here.

Talking of war...

My brother just put up a picture of my Grandad's very familiar German binoculars, picked up on the battlefield in World War 1.

Taxi to the Dark Side/Talking to the Taliban

This is not what I usually write about here, as I have little to add to the discussion over the war in Afghanistan, but two things have made a big impression on me in the last couple of weeks.

The first is the series of interviews with 42 members of the Taliban organized by Graeme Smith of the Globe and Mail (link). In particular, there is this:

Almost a third of respondents claimed that at least one family member had died in aerial bombings in recent years. Many also described themselves as fighting to defend Afghan villagers from air strikes by foreign troops.

...and this, which I guess many people better informed than I already know:

Aerial bombings and civilian deaths have both increased: The United Nations estimates more than 1,500 civilians were killed last year, as compared with the 900 to 1,000 civilian deaths counted by two studies of the previous year. An analysis of the first nine months of 2007 found the number of air strikes was already 50 per cent higher than the total for 2006.

Civilian bombings emerged as a major theme of the war last year. President Hamid Karzai shed tears in public as he spoke about civilian deaths. In June, a coalition of Afghan aid agencies published a controversial report suggesting that the rate of civilian casualties had doubled from the previous year, and that international forces were starting to rival the Taliban as the greatest source of civilian deaths.

The second was watching Taxi to the Dark Side, the Oscar-winning documentary about torture in Afghanistan, Iraq, Guantanamo, and elsewhere. It made very clear that most detainees (83,000 at the time the film was made) were not detained by the allied forces, many have committed no crime whatsoever, and that torture of the detainees is common. The effect that the torture and murder of Dilawar, the taxi driver of the title, has on his village is not followed up but is not difficult to imagine.

Aerial bombing and torture are not at the centre of the debate over NATO's role in Afghanistan, but they should be. The debate would then not be about how we can best help the people of Afghanistan; it would be about how we can stop damaging them and driving them to desperate measures. As long as these tactics are part of the military operation in Afghanistan then the longer NATO is there, the more enemies it will create.

Taxi to the Dark Side, which includes many interviews with soldiers involved in torture, shows how the ethics of groups can differ from the intentions of the people that make up those groups. Even an army composed largely of well-meaning individuals can act in fundamentally immoral ways. Just as NATO soldiers cannot tell a Taliban veteran from a villager who abhors the Taliban, so Afghan citizens cannot tell the difference between a soldier who will treat them decently and one who will arrest them for nothing and condemn them to torture. In both cases the safe assumption to make is the cautious one. So all Afghan citizens will be suspicious of and hostile to NATO soldiers and NATO soldiers will be suspicious of and hostile to Afghan citizens whether or not that hostility is deserved by the individuals. And each such hostile encounter justifies the hostility of the other side. While NATO is there it is not easy to see how things can improve.

Within Canada, we need to address the question of what kind of war we are part of. The assumption to date is that we are fighting a decent war. As long as aerial bombing and torture are part of the military operation then this is not a decent war.

Here Comes Everybody

Here Comes Everybody: The Power of Organizing Without Organizations, Clay Shirky, Penguin, 2008.


"We are living", says Clay Shirky, "in the middle of a remarkable increase in our ability to share, to cooperate with one another, and to take collective action, all outside the framework of traditional institutions and organizations" [p. 20]. Digital technologies are now part of our social fabric, and all "the phones and computers, the e-mail and instant messages, and the webpages are manifestations of a more fundamental shift. We now have communications tools that are flexible enough to match our social capabilities and we are witnessing the rise of new ways of coordinating action that take advantage of the change" [20].

Here Comes Everybody is an accessible and challenging introduction to these changes - to the many ways every Shem, Shaun and Issy can now share, collaborate, and act together. It combines some great stories with non-technical introductions to some of the key ideas (a little game theory, a little network theory, a few power laws) and is well worth reading -- but it should be read with caution.

The reason for this mixed verdict is the dual nature of the book itself. Here Comes Everybody has two voices. One (let's call him 'Shirky') is a perceptive and creative interpreter of the ways that digital technology is changing society. I like and respect Shirky. He is blunt and provocative enough to cut through the mess of questions that come up when tackling something as far-reaching as the Internet, while being even-handed and reserved enough to respect the complexity of his subject. Here is Shirky on the erosion of journalism and photography:

There is never going to be a moment when we as a society ask ourselves, "Do we want this? Do we want the changes that the new flood of production and access and spread of information is going to bring about?" It has already happened; in many ways, the rise of group-forming networks is best viewed not as an invention but as an event, a thing that has happened in the world that can't be undone. As with the printing press, the loss of professional control will be bad for many of society's core institutions, but it's happening anyway. [p73]

I've struggled with this message for a few days because I don't like its determinism, but he has convinced me. Shirky separates what is happening from what is desirable (not that everything digital is undesirable) and that's an important separation to make if we are to be at all clear-headed.

Shirky is not a techno-inevitabilist in a broad sense. Nuclear power, for example, "is a technology that society can, for the moment, make a decision about" [299]. As with driving a car, we "have a good deal of control over both the route and the speed with which nuclear power progresses, including the option to simply pull over" [299]. But when it comes to digital technologies we are steering a kayak: "We are being pushed rapidly down a route largely determined by the technological environment... Our principal challenge is not to decide where we want to go but rather to stay upright as we go there." [300]

The other voice (let's call him 'Clay') is a techno-enthusiast and an inveterate story-teller. When Clay looks at the Internet he sees no reason for worry - he sees freedom and unlimited potential. It's "the largest increase in expressive capability in the history of the human race" [106], bringing with it a world in which, when "people care enough, they can come together and accomplish things of a scope and longevity that were previously impossible; they can do big things for love." [142] Shirky may warn that there are losses from social changes, but Clay is breezily dismissive: "The spread of cheap and widely available creative tools is sad for people in the advertising business in the same way that movable type was sad for scribes -- the loss from this kind of change is real but limited and is accompanied by a generally beneficial social change." [209] This is the logic of The Lottery - the short story in which one person is stoned so that others can be better off.

I blame Clay and his enthusiasms for the two major flaws of this book.

The first is a fallacy of composition. Clay looks at the Internet and sees lots of groups forming (and things are easy to see on the Internet because even our most casual utterances get stored on someone's servers for posterity to investigate) and he concludes that the world is alight with a new groupiness, the likes of which we have never seen. From time to time Shirky pipes up to remind his alter ego that this is not enough, that "treating the internet as some sort of separate space... was part of the problem" [194]. "The internet augments real-world social life rather than providing an alternative to it" [196]. One implication of Shirky's caution is that, to evaluate the state of groups in our world, we also have to look at how our use of the Internet may have displaced other forms of group building. But while Shirky knows all about Bowling Alone, Clay is too busy running off to tell us a story about Meetup.com to take a really close look at how those positives and negatives are adding up.

While Clay is telling us all about the use of digital technology to spark innovative forms of protest in Belarus, which is a fascinating story, we really need Shirky to ask why, with all these group-forming tools at our disposal and despite the documented disillusionment with the war in Iraq, there is so little coherent protest happening compared to previous wars? Is it really the case that society now is becoming, thanks to the internet, more democratic, more collaborative, and more cooperative than before? I am not convinced. Clay is in danger of making the same mistakes that William Greider made in One World, Ready or Not, and which Paul Krugman demolishes - of finding lots of examples of groups and inferring that the world must have more groups in it than it used to - but that logic is flawed.

Now Shirky is far too well-informed to fall into this kind of trap. Shirky recognizes that, just as removal of a bottleneck at one point in a highway may prompt a new bottleneck to form a few miles down the road, so the "removal of technological limits has exposed a second set of social ones" [91]. But Clay's enthusiasms mean that the book is unbalanced, and this second set of limits never really gets investigated closely. Clay the enthusiast wins out over Shirky the dispassionate observer. If you are going to argue that groups are forming as never before, and if you are going to use minor events like angry airline passengers protesting about being trapped on runways to claim that "Consumers now talk back to businesses and speak out to the general public, and they can do so en masse and in coordinate ways" [179] then you really have to think about consumer activity before digital technology. Here is "talking back to business" with a vengeance:

August [1800] - Notwithstanding that the last day of this month was a Sunday, it was marked as the commencement of a serious riot. A great increase in the price of provisions, more especially of bread, had roused the vindictive spirit of the poorer classes to an almost ungovernable pitch. They began late in the evening, by breaking the windows of a baker in Millstone Lane, and in the morning proceeded, with an increase of numbers and renewed impetuosity, to treat others of the same trade in the same unwelcome manner. Granaries were broken into at the canal wharfs, and it was really distressing to see with what famine-impelled eagerness many a mother bore away corn in her apron to feed her offspring. [link]

Do consumers have a stronger voice now than in the past? I don't know, but I do know that a story or two about American Airlines passengers is not going to convince me. If Clay wants to tell us about a student group using Facebook to protest about British Bank HSBC's cancellation of interest-free loans then perhaps he should think about the longstanding student boycott of Barclays Bank during the 1970's and 1980's that contributed to the end of apartheid. But he doesn't, and that's disappointing.

The second flaw - also a common one - is a reluctance to follow the money. Between them, Clay and Shirky convince me that the Internet marks a change; that a society with an Internet is different from a society without one. Also, they convince me that the Internet is not a single model of sharing/collaboration/collective action; it's many models. So let's talk about these models, about which ones have legitimacy and longevity and which don't. To do so requires Shirky to take the broad brush that Clay is using and to start to make smaller, more detailed points.

Perhaps we no longer need books telling us that the Internet is a big thing. It is time to treat that fact, as Shirky sometimes does, as the starting point for a discussion rather than the conclusion. The questions then become ones of what kind of structures will form and persist in the online world, and if you are going to talk about these questions then you have to address the economics of the problem.

Money matters.

People's willingness to contribute to Wikipedia, Shirky points out, is tied to the non-profit status of Wikipedia. A threat from Spanish participants to start an alternative version convinced founder Jimmy Wales to "formally forgo any future commerical plans for Wikipedia, and to move the site from Wikipedia.com to Wikipedia.org, in keeping with its nonprofit status. Similarly, he decided to adopt the GNU Free Documentation License for Wikipedia's content [which] assured contributors that their contributions would remain frerely available" [274]. There is an ongoing tension between contributors to sites and the "owners" of those sites that is visible on MySpace (see Billy Bragg's efforts to gain musicians rights over their content), on Facebook (the failed "Beacon" program as one example) and even Google and the line it walks with advertising content on its search results. Contributor-driven movie database IMDB was taken over by Amazon, but Amazon does not put its name on the pages of the site - does it fear that people would not contribute so freely if they realized they were just making Jeff Bezos even richer? The book ends with a story by Clay about a successful revolt by participants at digg.com - but once again, a story is not enough to support the thesis he is trying to make. 

There are many issues of democracy, ownership, privacy and control that, depending on how they are resolved, will influence our chances of staying upright as we get carried along by this technological flood. Clay Shirky shows in this book that he has many insights that we can use, but to really help steer this kayak we need more Shirky and less Clay.

Circular References

  • Not a Blogger
    This here is a relaxed, slow-moving weblog. It ain't one o' them hyperactive updated-all-the-time weblogs. Slow down a little.

Book

Blog powered by TypePad
Member since 11/2005

Tools

  • Sitemeter

Books