Here's a draft of something I've been working on over the weekend...
On a hot summer’s day a few years ago I was waiting at a train
station in the UK.
The train was late. A voice came over the public address system saying “We
apologize for the late arrival of the 10:30 to Brighton”. Fair enough. Then ten minutes later, the
same voice, in the same intonation, said the same words: “We apologize for the
late arrival of the 10:30 train to Brighton”.
Ten minutes more, the same thing. And so on for an hour or so until the train
finally arrived.
What stayed with me was not the lateness of the train — the
day was so hot that tracks were buckling and we knew on setting off that we
were in for delays — it was that announcement. The words themselves sounded
like an apology, but, just like “your call is important to us”, the message
that came through to me was the exact opposite of what the words themselves
said.
* * *
Literature scholars have a notion of something called
“performative language”. The idea is that while most utterances simply talk
about the world (“That daisy is yellow”) others actually perform an action. “I
resign”, when spoken to your boss, is not just a statement, it is an action
that changes your status from employed to unemployed. “I do” when said in the
right place and in front of the right audience, actually changes the speaker’s
status from unmarried to married.
“We apologize” can be a performative utterance. When offered
in the correct way, saying you’re sorry is not just a statement about the world,
it is an act: the act of apologizing. But it’s not always a performative
utterance. It needs two things to make it so.
First, and most obviously, someone needs to say it. Without
an apologizer, there can be no apology. An automated system cannot apologize.
It could be argued that on that railway platform, even though there was no human individual speaking, the one making the apology was the
company. The recorded voice actually comes from the railway company, and
was simply transmitted by the automatic system, just like a letter carrier
delivers a letter. But it turns out there is a problem with this too, and the
problem is cost.
Apologizing, as we all know, can be a difficult thing to do,
and that difficulty is part and parcel of the act of apologizing. Imagine
receiving an apology for a deeply painful betrayal delivered casually, in a way
that made it clear it carried no more emotional cost than commenting on the
weather. You would understand immediately that it was not an apology at all, even
if it contained the words “I apologize”. It is “cheap talk”, and while it could
be an attempt to get you to be quiet and leave the offender alone, or to
postpone the inevitable recriminations, or one of several other things, cheap
talk can’t be an apology.
So when an automated system says “we apologize” then there
is, in fact, no apology happening, no matter how often it says it. When the
only action it takes to deliver the words is pushing a button somewhere, at the
most, then the statement is “cheap talk”: cost-free communication that should
carry no weight.
* * *
We are getting to understand the business of “speech without
a speaker”. When we hear an automated voice saying “Your call is important to
us” most of us do receive the real message. We know that if our call was really
important then there would be a person there to receive it. The words are not
the message; the automated system that delivers it is.
We understand that the letters we get from company CEOs offering
us new deals do not, of course, come from the CEO at all. He (or occasionally
she) may not even have read it, let alone written it. May not even know of its
existence.
“Have a nice day” is a variant on the same theme: the phrase
is often said by a real person, so it seems like there is a speaker, but if the
hapless individual behind the till is reciting a script, and if delivering that
script is a condition of their employment, then they are not really the
speaker; they are simply passing on a message from someone else. And that
someone else has no interest in the niceness (or otherwise) of your day: it is
simply a calculated policy to give the appearance of friendliness.
On occasion, of course, someone behind a till is in a
friendly mood, and actually gives the words real meaning. But most of us can
tell the difference between such a sincere utterance and someone delivering a
programmed script.
But while we are adjusting to the problem of speech without
a speaker, we have not really started adjusting to the other changes that cheap
communications technology are bringing. We know that our mediums of
communication are changing, with new forms of internet-based communications
mutating every few years, from e-mail to instant messaging to weblogs and more.
What we need to adjust to is the way that the economics of these new media
change the meaning of the messages we send and receive.
Many of us do adjust to these changes in meaning in our
personal communications. A Christmas letter sent by e-mail to a long list of
recipients means less to us than an individually handwritten letter. But our
social systems are slower to change. In particular, there are forms of legal
communication that have built into them an assumption of cost, but which can
now be made very cheaply.
* * *
A new communication mutant was described recently by Mark
Rasch, former head of the US Justice Department’s computer crime unit, in The Register, an online journal of technology comment. Rasch writes about “robolawyers” or “lawyerbots”: programs designed to search the Internet for suspected violations of a company’s copyright or trademark. A lawyerbot can be designed not only to detect possible violations, perhaps by searching for keywords or
combinations of words, it can also be designed to send out legal warnings (Rasch calls them “takedown notices”) to the owners and hosts of the sites involved.
The legality of these automated takedown notices has not
been tested. But they don’t have to go to court to be effective. Internet
Service Providers and other web site hosts such as eBay don’t like the
possibility of lawsuits, and their legal liability for the content of the sites
that they host is uncertain, so these notices are often enough to persuade them
to take down the problematic material, regardless of whether there is anything
wrong with it or not.
Rasch writes about Brian Kopp, who wrote a book based on the
popular online game “World of Warcraft” and offered it for sale on the eBay
auction site. The book was, by all accounts, in the spirit of those “Dummies”
guides that tell you how to use Microsoft Word and other software products – an
unauthorized third party guide, but not defamatory or illegal. Once Kopp posted
his book, however, eBay began receiving a stream of “takedown notices” telling
them that the book violated the rights of Blizzard, the company that owns World
of Warcraft, and that they should remove it from their site. eBay responded by
not only removing the book, but cancelling Kopp’s account so that he could not
auction anything else either.
It is impossible to know for sure whether there were any
real people involved at any stages of this campaign against Brian Kopp (who is
almost certainly real, although not a very good writer). Rasch notes that Kopp’s eBay auction generated “a slew of takedown notices from
various parties. As soon as he reposted the auction, it generated a new
takedown notice. Human lawyers are generally not that efficient. So these
autonomous agents may in fact be the ones generating these takedown notices.”
So did any real person at Blizzard know they were sending takedown
notices to Mr. Kopp? Quite possibly not. Had anyone (apart from a computer) “read”
his book? Probably not. Did an individual at eBay weigh the evidence and decide
what action to take, or does eBay have a program to disable accounts if it
receives a sufficient number of complaints? Unclear.
The fact that legal letters rarely go to court, and the fact
that they can be generated programmatically, means that automated legal notices
are likely to increase. We are already subject to automated legal checks
whenever we install new software, in those End User License Agreements that we
have to say we’ve read even though we obviously haven’t, and even though they
are not designed to be read. Increasingly, those agreements include clauses
that let software report back to the company that made it about the use we make
of it, or details about our computer. Such information can have legitimate
uses, enabling better fixing of software problems for example, but it can also
have illegitimate uses.
There is an assumption in our legal system that a legal
notice follows some kind of investigation, however cursory, on the part of the
party that issues it. That is, it is a document to be taken seriously. But
auto-generated and widely distributed legal notices are “cheap talk”, and their
increasing ease of generation means that, like the apology at the railway
station, they should be treated as meaningless.
* * *
It would be easy to dismiss Brian Kopp’s case as absurd if
it wasn’t so clearly a harbinger of things to come. As lawyerbots proliferate, companies
will find it worthwhile to develop other lawyerbots to handle incoming
“letters” and we will have legal communications that nobody writes and nobody
reads. If a letter is delivered and nobody reads it, does it carry any meaning?
If lawyerbots were just sending meaningless messages back and forth, nothing
would be lost except for a bit of Internet bandwidth, but it seems that the
legal system has not yet adjusted to this new mutant, and so such
communications can be damaging.
Increasing automation means that forms of communications are
mutating ever more quickly. Some developments are not difficult to foresee. As
with all things computer-related, it is clear that automated communication is
going to be more common – a lot more
common – in the future than it is now. The near future will bring automated translation
software that can provide good enough translation in a wider and wider set of
circumstances. It will also bring speech comprehension software, and
text-to-speech software of increasing subtlety: no more jerky sentences with
those pasted together words – the audio equivalent of the ransom note pasted
together from newspapers. Put them all together and programs will soon be
carrying out conversations with people around the world, persuading us to part
with our money in new and grammatically sophisticated ways, using voices
designed to exploit our sympathies and prejudices.
Except they are not voices and not conversations at all, they
just sound like they are. We need to be consistently on our guard if we are to
avoid becoming confused by these mutant communications. As a first step, we
need to acknowledge that machine-delivered apologies are not apologies, no
matter how sincere they sound. And we need to decide that lawyerbot-delivered threats
and warnings carry as much weight as the cost of delivering them, which is
zero.
