I want to thank everybody for joining us this
afternoon, for those of you who are joining us by
video conference at ATRDC it is a gloomy and rainy
day outside, probably not something you're not used
to up in Boston, but the mood of the city is
bright, after completing their sweep of the
Cardinals last night the Nationals have advanced to
their first-ever World Series.
[APPLAUSE]
And with the Caps as Barry is very proud
of their Stanley Cup victory.
Last year DC could be on its way to becoming a city
of champions, though there's always the
Redskins.
Today we're excited to have Professor Shoshana
Zuboff with us.
She studies digitization and its relationship with
both the history and future of capitalism.
Her work also focuses on the consequences of
digitization on individuals,
organizations, and society at large.
Her recent scholarship finds that the digital age
has led to a new form of capitalism based on the
provisions of free services in exchange for
behavioral data from users.
Her talk at the Milkin Global Conference was a
big hit.
I had the great pleasure of meeting her in person
earlier this year, in Los Angeles, and I'm just
delighted that she agreed to come and join us.
Earlier this year she released her latest book,
entitled: The Age of Surveillance Capitalism
the Flight -- The Fight For a Human Future At the
New Frontier of Power.
If you don't have a copy, I don't think I can
endorse the book, but I -- I do recommend you coming
-- (Laughter).
Shut the cameras off.
Nina, our ethics officer counsel is sitting right
here, so I just want to be very clear about what I
can say.
She argued that the new surveillance capitalism
represents a potentially final phase in the
evolution of capitalism.
The precursor to this book was an article entitled:
Big Other Surveillance Capitalism and the
Prospects of an Information Civilization,
which won the best paper award at a 2016
international conference on information systems
scholars.
Professor Zuboff is also well known for her first
book, The Age of the Smart Machine, the Future of
Work and Power, which was the first book of its kind
to track how then nascent digital technology changed
the workplace, both for management and rank and
file employees.
Professor Zuboff has been featured in a number of
news publications and has been showcased on CNBC.
She earned her Ph.D.
from Harvard University and her bachelor's in
philosophy from the University of Chicago.
We invited Professor Zuboff to share her
perspectives with us and I'm sure it will be
fascinating.
In the same spirit of our Jackson Nash address
series, and the talk of Professor Feng Zhu, her
colleague from Harvard, earlier this year, this
past August, we hope that her remarks cause us to
think critically about our economic framework for
understanding competition, digital markets, and our
economic framework more generally.
After her talk we'll open up for questions and after
we conclude I hope that you will all join us for
reception at the back of the auditorium right here.
So Professor Zuboff, welcome to the Department
of Justice.
It's my pleasure to welcome you here.
Thank you so much.
I can get up there now.
[APPLAUSE]
So I just want to get our timing
straight, since we're starting a half hour late,
how long do we have, can somebody tell me?
45 minutes to an hour is fine.
Okay.
For our talking and Q and A?
Yeah.
Okay.
All right.
45 minutes to an hour.
Okay.
Good.
That's great.
So -- Or as much time as you think we would need to
be educated.
All right.
We'll be here for awhile folks, don't worry.
And then there's going to be a quiz, so everybody's
got to listen to me.
Okay.
So let it be known that not even the seasons first
nor'easter would prevent me from getting here
today.
I don't know if Denise is around, but thanks to
Denise Adams I think -- I think we tried like, 500
dates or something before we were able to do this,
and I really didn't want to miss the day.
So I'm a little bit the worst for wear, but here I
am, and it's -- it's such an honor to be here with you.
You know, this is the paperback version of my
book, which is just out in London in paperbacks, just
out in London, has just been, but you know the
real version is really thick, big, and usually
when I have events and talk about the book
I start over here and I end over here, and in a
discussion about law and regulation and, you know,
what can we do, and is it too late and all that.
But thinking about what we were going to do this
afternoon I decided that I'm going to start over
here and then I'm going to work my way back to the
beginning and follow through because finally
talking to some people who can do something about this.
So I'm very committed to that, and very excited
about that, and have -- have a lot of ideas about
it, which probably are going to be different from
-- from your ideas.
And so what I'm hoping is, I'll unspool my -- my
framework and thinking about surveillance
capitalism, and then in the Q and A I'm imagining
that is the time when you'll come from your
spaces and I'll come from mine and we'll use that
Q and A to sort of build those bridges and figure
out how our ideas can inform one another.
So that's what -- that's what I'm after today.
So I'm going to begin with my conclusion:
The internet is now owned and operated by what I call
surveillance capitalism.
That is to say, the internet is owned and
operated by private surveillance capital.
That's where we are right now.
So it's kind of a crazy thing, because it wasn't
that long ago, 20 years, 15 years, even 10 years,
plenty of people still thinking, you know,
the internet was the answer to democratization, to voice,
to individual empowerment, and a lot of what I've
done over the many years of working on this project
is to understand the way in which that vision and
that dream -- dream morphed into something
very, very different, surveillance capitalism,
the unique form of power that it produces, its
vision for society, for human nature and the
consequences of all of that for some very basic
things, like democracy.
So it's interesting to me that despite the fact that
we are a nation of laws, and in these spaces we
have laws on privacy and we have antitrust laws,
which you specialize in, surveillance capitalism
rooted and flourished over the last 20 years, largely
unimpeded by law, and that becomes a big question for
us to try and understand how was that able to occur.
It -- it flowed into blank spaces, you know, the --
some of the pioneers of surveillance capitalism
celebrated the ways in which the internet was
sort of the last ungoverned space, as they
called it, and so it -- it -- it moved like a -- like
a sludge into these spaces that were essentially
lawless, free of regulation, free of
democratic governance, and the rapid growth was
facilitated by a whole slew of things.
There's 16 reasons that I examine in depth, not
going to go into all 16 of them right now, but -- or
this afternoon, for that matter -- but it's
important to note this rapid growth was enabled
by the unprecedented character of the
mechanisms and methods that it invented and their
inherent indecipherability.
And that is exemplified in technical systems and
public rhetoric carefully, intentionally designed to
camouflage, misdirect, confuse, bamboozle, and
generally engineer the ignorance of populations
and our lawmakers.
And this has been a pretty successful undertaking.
So anybody familiar with Tom McGraw's book The
Profits of Regulation?
You know, he wrote it a little bit ago, but it won
a Pulitzer prize, and it is a very wise, insightful
history.
A historian of business, at Harvard, and this is a
book that bears you looking at it, unlike my
book it's a fairly slender book, but it really bears
your taking a look.
So I wanted to bring back some -- some of the key
things that he wrote about, and one is that,
you know, he looked over the arc of the 20th
century, US, US 20th century, and he wrote
about these different regulatory regimes that
have arced across time, beginning in the 1870s,
the early period of industrialization,
the early 20th century, especially 1900, 1916,
the 1930s, of course, the era of the new deal, the 1970s
and 80s, `80s, accompanying that new
phase of industrialization.
But the key point he made was that the people who
forged these regulatory visions, and implemented
these regimes, were different in each one of
these eras, beginning early century with
the muckrakers and the progressives, mid-century
it was really the lawyers who were in charge, and
only late century was it the economists, something
that he famously called: "The Economists' Hour."
And of course he anticipated, Tom was
writing in the -- in the `80s, he anticipated that
the economists' hour would certainly end, and
wondered what would come next.
And in wondering about that he noted that across
this long arc of regulation in the 20th
century that concerns for justice and fairness,
he wrote, generally overshadowed the more
narrow aims of economic growth.
And he thought that that was going to be -- that
was going to be some kind of hint as to what was
going to come after the economists' hour,
a doubling back to -- to these -- to this emphasis
on fairness and justice.
And, of course, in my view, this next regulatory
era that I think, you know, many people believe
we're at least on the cusp of right now, this next
regulatory era is going to be championed by those who
care about democracy and who are deeply concerned
about the threats that surveillance capitalism
poses for democracy.
So McGraw ends with a very important warning, and a
very important warning for all of us, and that is:
He says that looking back over this history
regulators failed.
They failed when this one criteria was -- one
criterion was fulfilled, and that is when, quote,
they were unable to frame strategies appropriate to
the particular industries they were regulating.
So this -- this is a big deal for me, because
I spent a decade of my life on this book, I spent
about seven years in my pajamas just writing
what's in between these two covers, and I did it
because I believed that this new economic logic
needed to be named; and that naming was a prelude
to taming; and that without a careful
understanding of its unprecedented mechanisms
and methods we would be shadow boxing in -- in the
20th century when we needed to move into the
digital century and understand the
unprecedented.
So that was a lot of my motivation, and certainly
key motivation for my being here today.
When you understand the novel economic logic of
surveillance capitalism essentially every daily
atrocity, you know, you open up the paper every
day, I always tell my students, you can open up
the paper any day and you can basically recapitulate
every argument in this book, right?
Which gives its face validity.
But this stuff keeps happening.
When you understand the framework, when you
understand the economic imperatives, each daily
atrocity is extremely both predictable and
understandable.
So -- so the bottom line for me is that it says to
me that this is a moment when we need to look
beyond existing paradigms.
We're going to build on the shoulders, of course,
of what has come before, but we are going to need
21st century digital century paradigms, laws,
regulatory regimes if we are going to target,
interrupt, outlaw this distinctly 21st century --
this distinctly 21st century born digital novel
economic logic.
All right.
So I want to talk about what surveillance
capitalism is, but before I do that I want to talk a
little bit about what it is not, because there's a
lot of confusion about that, you know, category
errors, when you say something, when you think
something is one thing but it's actually not, it's
actually another thing.
So what are some of the key category errors?
Well, part of the public rhetoric of surveillance
capitalism has been to get us to believe that its
methods and mechanisms are the inevitable
consequences of digital technology.
The surveillance capitalism is conflated
with the digital.
This is simply false.
I write about many examples, examples of
telemedicine, examples of the Smart home, examples
that were designed by cutting edge digital
scientists, engineers, in, for example, 1999, 2000,
2001, 2002, 2003, no one even knew about
surveillance capitalism until around 2004.
And when you look at what their plans were back then
they're completely different from anything we
have now.
I want to give you one very, very quick example,
this project that came out of the top folks at
Georgia Tech, they called it the aware home, the
year is 2000.
So very quickly, there are digital devices in the
walls, there are people that live in the house,
the digital devices are spitting out information,
the information is going to one node and one node
only, that's a wearable little doodad, not unlike
this, that the occupants of the house are wearing
or having with them.
So when they drew the designs, when they drew
the schematics, it's a closed loop.
The device is the occupants.
And the scientists wrote about this, and they said,
look, this is very intimate data.
Our systems have to maintain privacy above all else.
That's the way they looked at it, even though they
had the same objectives for the Smart home that we
have today, elders being able to age in place,
security, energy efficiency, feedback that
can improve quality of life and so forth.
Okay.
That was what year?
2000.
Now we fast forward to 2017, two scholars,
University of London, doing an analysis of one
Nest thermostat.
You know what Nest is, and you know that it's now
wholly absorbed into Google.
So there is no more Nest.
All right, one Nest thermostat, they concluded
that any mildly vigilant consumer, who installs one
Nest thermostat should review a minimum of 1,000
privacy contracts, because not only is the Nest
aggregating data and sending it to all the
usual suspects, mostly Google, mostly Facebook,
some other big advertising groups in there, but also
to a range of other third parties.
And those third parties are sending it to other
third parties and other third parties -- you know
this is infinite regress, right?
It goes on.
And of course at each stage, starting with Nest,
no one takes responsibility or
accountability for what the other third parties
are going to do with the data.
So you're just out there.
And Nest says, look, we understand if you don't
want to agree with this privacy policy.
We just want you to be aware that if you don't
agree we're not going to be able to upgrade the
functionality of your thermostat, we won't be
updating the software, so you're going to want to be
on guard in case, you know, frozen pipes, smoke
detector doesn't work, other things that could go
awry if you don't agree.
So this is the world that we've moved from 2000,
that's 17 years.
So the question is, what happened?
Digital.
Right?
So what happened in between?
What happened in between is an economic logic,
nothing to do with digital technology, an economic
logic that essentially -- think of the Trojan horse
analogy, essentially invaded the technology,
hijacked the technology, the beast hiding in the
belly of the Trojan horse.
So that's where we are now.
So -- okay.
So bottom line, we can easily imagine digital
technology without surveillance capitalism,
impossible to do the reverse, impossible to do
the reverse, can't image surveillance capital
without the digital.
Okay.
Second category error, surveillance capitalism is
not a corporation or not a group of corporations.
Invented by Google, migrated to Facebook,
became the default logic of accumulation in the
tech sector, now migrates through the normal economy
in virtually every sector, insurance, retail, health,
education, finance, every Smart product, every
product called Smart, every service called
personalized, now complex ecosystems of data
suppliers, market makers, and everything in between.
You know about the CEO of the Ford Motor Company and
his vision for Ford Motor.
You know, people aren't buying cars the way they
used to, and that's a worldwide phenomenon.
So what we would like is for the CEO of Ford Motor
to do something that Henry Ford might have done a
century ago.
Henry Ford was a terrible person, by the way, you
know, nothing good to say about him except what he
did as -- as an innovator in business.
So -- so what we would like the CEO of Ford to
say, you know what, we're never going to sell cars
the way we used to, let's invent a car that doesn't
burn any carbon.
And, you know, that's going to be the shot in
the arm that the car industry all over the
world can use.
We're going to -- we're going to be rich again.
Unfortunately, that's not what he's saying.
What he's saying is: W want PEs like Google and
Facebook.
So I've got a great idea, there are 100 million
people driving Ford vehicles.
Let's stream data from those drivers.
Let's combine it with the data, Ford credit, where
we already know everything about you.
Now we're on a par with Google and Facebook, we've
got all this data, what investor would not want to
invest in us?
Okay?
We're going to come back to that thought,
so remember, what investor would not want to invest
in us.
Okay.
So we can no longer just be talking about Facebook
or Google or anything like that or Amazon or
Microsoft.
We're talking about a surveillance economy, now.
This whole thing is shot through.
All right.
Now, third category error, this is not a person.
We can't reduce this to a person.
Lot of shots at Mark Zuckerberg these days.
Every day that happens I guarantee you Larry Page
and Sergey Brin are waking up
very happy, Sundar Pichai and Satya Nadella,
this is all on Zuckerberg right now.
This is great.
But this is not about individuals.
There are individuals who have too much power, and
their power must be curbed, but at the end of
the day even curbing the power of key individuals
is not going to stop this economic logic.
We're beyond that now.
So that's something to take into consideration.
Okay.
So what is surveillance capitalism?
All right.
The year is 2000, Larry Page was sitting in a room
with a bunch of other people, including the
first ever brand manager of Google, and they were
trying to say, okay, what is Google, what is Google,
like they didn't know how to describe what business
they were about to be in, and it was Larry Page that
early who said, if we did have a category it would
be personal information.
People will generate enormous amounts of data,
everything you've ever heard or seen or
experienced will become searchable.
Your whole life will be searchable.
So this was an idea that already existed in the
year 2000 and Page's vision perfectly reflects
the history of capitalism.
So for a long time, you know, people who write
about capitalism, ideas that begin with me, have
talked about this -- this sort of foundational
pattern, goes like this, capitalism evolves and
grows by claiming things that live outside the
market dynamic and bringing them into the
market dynamic, turning them into commodities that
can be sold and purchased, right?
So you're familiar with that, like industrial
capitalism claims nature for the market dynamic,
turns it into land and real estate that can be
sold and purchased.
I don't have to give you a lot of examples of that,
right?
So surveillance capitalism, as we will
see, diverges fundamentally from the
history of capitalism in many, many ways, except
this one, because it does exactly the same thing,
it claims something from outside the market, brings
it into the market to make it a commodity, but in
this case with a dark and unexpected twist, because
what it claims is private human experience.
Private human experience, going right back to that
year 2000, was understood to be the last virgin wood.
Where do we look for new sources of
commoditization, private human experience?
So this is -- this is the birth.
So private human experience is claimed as
free, raw material for new processes of production
and sales.
So what happens is private human experience is
unilaterally taken for a process that people call
datafication, taken and converted, transformed
into behavioral data.
Some of these behavioral data, of course, used for
product and service and improvement.
We saw that a lot at the beginning of Google.
But what happened at Google was -- well, I can
come back to that in a minute.
Let me give you the overview first.
All right.
So we're taking our human experience, making it into
behavioral data, using some of it for
improvement, but discovering that much more
data than what we need for improvement is available.
And these data are valued for their rich, predictive
signals.
This is more than what is needed for product and
service improvement.
And I call this behavioral surplus.
These behavioral surplus flows are hunted and captured.
Some of them come out of, you know, what we -- how
we search and how we browse and what we post,
but most of it comes from stuff that is taken from
us, that we're not even aware of, for example, on
a Facebook post do you use exclamation points or
bullet points?
The level of saturation in the photos that you
post, what color.
These are rich, predictive signals that are lifted
from what we voluntarily give, but also are hunted
and captured in the online and the offline world,
without our knowledge, without our awareness, and
therefore, most certainly, without our consent.
This is why, for example, the tech companies fought
for the right to take our faces when we are in
public without having to inform us and, therefore,
without having to acquire our consent, for example.
So there's a broad-based activity that's moving in
many directions, hunting and capturing behavioral
surplus, these data flows are essentially, like,
every -- every one of these data flows
represents a supply chain interface.
So picture, now, complex webs, pipes, supply chains
that are out, you know, they're out there on the
street, grabbing your face, they're online,
taking signals from what's online, they're looking at
your posture as you walk down the street, the slope
of your shoulders, for example, and your gait.
They're picking up the muscles in your face in
order to translate that into emotional analysis
because emotions turn out to be excellent predictors
of behavior.
All right.
So all of this is converging, finally, into
pipes that are bringing it into a new kind of
factory, a 21st century factory that we call
machine intelligence, artificial intelligence.
And like all factories this factory produces
products.
But these products specifically are
predictions of human behavior.
They are prediction products.
So probably many of you saw this, about a year ago
there was a leaked Facebook memo that gave us
some insight into this -- this central manufacturing
operation, this -- the heart of its factory,
they call it the AI hub.
And that glimpse, among many interesting things
revealed in the memo, we see it describing the fact
that in the AI hub it is now ingesting, their term,
trillions of data points every hour and producing
six million predictions of human behavior, their
language, per second.
All right.
So that's -- that's the -- that's what's happening in
the factory.
And where do these prediction products go?
Obviously they're -- they're predicting our
behavior, but they're not for us.
They are extracted from our behavior, they're
about us, but they're not actually being used on our
behalf.
They come from us, but they aren't being cycled
to us.
Where they go now is to a new kind of marketplace,
constituted by business customers who have a keen
interest in knowing what we will do in the future.
So -- so essentially what's happening here is
the game of these markets is to sell certainty.
And the way you sell certainty is to have the
best predictions that come closer and closer to
approximating observation.
And these predictions are now sold to business
customers in a new kind of marketplace that trades
exclusively in human futures.
So these are human futures markets in much the same
way that we have markets that trade in pork belly
futures or markets that trade in oil futures or
markets that trade in wheat futures and so forth,
commodity futures.
So now we're trading in futures of human behavior.
So -- so it maybe goes without saying, but I will
just underscore it for a moment, these -- these
activities that I've told you about, that
I characterize as surveillance capitalism,
the word surveillance in that phrase is not
intended to be mellow dramatic or -- or -- or
provocative, it's intended to be a practical
description, because from the very beginning it was
understood that when people are exposed to
these kind of practices they actually reject them.
They don't want any part of them.
And so the issue was, how do we hide these practices
so people don't actually know that we're doing
these things?
They don't actually know that we're taking these
data flows, they don't actually know that we're
hunting these predictive signals.
And so from the beginning these -- there are very
explicit discussions about the necessity of the
one-way mirror, that we can see them but they
can't see us, that we can hunt and capture but they
never know we were there.
We don't leave a footprint.
So the socializations with the one-way mirror with
the implications for social hierarchy, the
implications for inequalities of knowledge
the inequalities of power were there from the start,
ergo surveillance capitalism.
Okay.
So -- so I started to mention a moment ago that
this was invented at Google.
And I'm not -- I'm not going to go into the whole
-- all the details of the history here, but I want
you to know -- Can you move back to the podium so
they can here you on the far side?
Okay.
I'm sorry.
You couldn't hear me?
You couldn't hear all of that?
We can hear all of that.
The far side.
Oh, okay.
I'm sorry.
I usually never stand behind the podium.
I'm blocked from you.
But I will do that for the sake of the greater good,
here.
Okay.
So -- so -- but here's something that I really
want you to know about this invention at Google.
So what happened at Google -- I'm just going to go
over the top line so I can get to the punch line --
financial emergency, dot com bust, even their
investors were threatening to pull out, so a state of
emergency was declared, before that the Google
founders rejected advertising as something
that would disfigure the internet, disfigure the
search engine, but now it came time to double down
on advertising as a way out of financial
emergency, stated exception.
And -- and that's when they discovered that they
had all this collateral data, they called it
digital exhaust, sitting around in their servers,
that's when they discovered the rich
predictive signals buried in these data, and even
though they weren't using them for product
improvement they one day discovered that they could
use these data to create predictions of human
behavior.
What were those predictions called?
Click through rates.
First globally successful prediction product.
And what was the first globally successful human
futures market?
We call it online targeted advertising.
But indeed, those were business customers who
gave up the long traditions of brand values
leading advertising placement in order to take
the computation that would be spit out of Google's
black box and would tell the advertiser where to
place the ad.
And the rest is history.
So what's interesting -- and here's the punch line
that I want you to know -- is that between the year
2000 when -- when this first stirred, the problem
first became clear and they first began working
on it, and the year 2004, which was what, that was
the year that Google IPO'd, first time they
were public documents, and any numbers that anyone
could look at to try and see what had happened in
the interim years, because the investor didn't pull
out, good reason why, between 2000 and 2004
Google's revenue line increased by 3,590 percent.
That is what I call the surveillance dividend.
Once Google achieved the surveillance dividend with
this new logic it raised the bar for every
business, first of all, in Silicon Valley, first of
all, in the tech sector.
What venture capitalist, what investor wants to
invest in a business that's going to innovate
the old and slow way, like inventing a car that
doesn't burn carbon -- I told you I was going to
come back to that -- when you can invest in a
business that is producing a surveillance dividend,
raising the bar and, in effect, disfiguring the
competitive field for anyone who wants to truly
innovate, who anyone who wants to innovate in the
way we used to think about innovation, like, oh,
I know how to solve this, this -- this competitive crisis.
I'm going to create a product or service that is
just better than anything that anyone has imagined
heretofore, I'll call it the Model-T, for example,
right?
Okay.
So -- so now we're ready to talk about
the competitive dynamics, here.
So a lot of what I spent my time doing is trying to
understand the competitive dynamics of these new markets.
And it turns out when you deconstruct, reverse
engineer the competitive dynamics of these markets
you -- you -- you -- you uncover what the -- what
are the iron laws, the economic imperatives that
rule all practice in this domain.
And I'm summarizing them, at a high level, and it's
pretty simple.
The first one is -- and I know that you're familiar
with this -- you're going to feed an AI to get great
predictions, you need a lot of data, right?
Everybody knows that.
So the first economic imperative is economies of
scale in the production of behavioral surplus.
Getting these economies of scale into the pine lines,
into the factories.
Then in the second phase of competitive intensity
it becomes clear that volume is critical, but
volume's not enough.
We also need varieties of data.
Can't just be what people are doing online,
what they're browsing, searching, what they're posting.
We want their faces.
We want their bodies.
We want their bloodstreams.
We want their brainwaves.
We want their breakfast conversations.
We want to know the decibel levels in the kitchen.
We want to know the map of their living room.
We want their car.
We want their homes.
So varieties of data, economies of scope.
So now we've got scale and scope, competitive
dynamism continues to intensify, and pretty soon
surveillance capitalists learn that the very best
predictive data comes from actually intervening in
the state of play in order to actually touch human
behavior, to intervene in human behavior, to coax,
to herd, h-e-r-d, human behavior in the direction
of its guaranteed outcomes.
This is something new under the sun and
impossible to achieve without the increasingly
global, ubiquitous digital architecture that
saturates our environment.
You heard Makan refer to my earlier paper, big
other, this architecture is what I call big other
for a very specific reason, that hopefully
I'll have a minute to explain to before we
-- before we end.
So economies of scale, economies of scope, now
we're looking at economies of action.
It's not enough to automate information flows
about us, now the aim is to automate us, to
engineer behavior, to modify human behavior at scale.
So, you know, I asked myself the question, I'm
sure you're asking yourself the same question
right now, and that is: Wow, that's very powerful,
to be able to command the digital infrastructure, to
actually modify human behavior, this is a very
unique kind of power.
What is this power?
And I became riveted on the fact that, you know, a
lot of folks have been talking about big brother,
we talk about Orwell, we talk about digital
totalitarianism and all the rest.
But, in fact, this is pretty much the opposite
of totalitarianism.
This is a kind of power.
Remember, totalitarianism operates through terror
and murder, through the constant threat of
physical violence.
Remember Stalin's famous phrase: The engineering of
the soul.
This power is almost the opposite of that.
This power really doesn't care what you do.
Maybe some of you saw that famous Andy Bosworth,
another leaked memo, when he said, you know, we
connect and we grow.
And when -- and when we connect if people use us
to do a terrorist plot, so be it.
We continue to connect and grow.
And if people use us to fall in love and get
married, so be it.
We continue to connect and grow.
This is what I call radical indifference.
It's the experimenters' point of view now applied
to real-world behavior on a global scale.
I call this instrumentarian power.
It works remotely.
It doesn't want to beat us up.
It has no intention to drag us from our bed at
night and ship us to the gulag or the camp.
What it does want, relentlessly, is our
behavior, at any cost, in order to fill its supply
chains and drive its markets.
All right.
So economies of action now represent this shift the
data scientists talk about, I interviewed a
whole slew of 52 data scientists, which
represented about 600 years of experience over
the years that I was doing this work.
And, you know, they talk about this important shift
in their work from what they call monitoring to
actuation.
And this is, you know, you've heard of knowledge
power, right?
Knowledge turns into power.
So that's what they mean.
Monitoring is you get a critical mass of
information, at a certain point, and that allows
your monitoring to pivot over, cascade over into
actuation, where you actually have enough
information to be able to remotely program, to
automate a machine system.
The same thing is now happening with the human system.
Enough critical mass of information in order to
now be able to actually actuate modify human
behavior at scale remotely.
Okay.
So this may sound a little science fictiony to you,
maybe you think I spent too long behind closed
doors or whatever, but, in fact, there are examples
of this all around us, all around us.
And we have to assume that there are many more
examples of this that we have not seen and probably
will never see.
I'm going to just throw out a couple of good
examples because you will already know about them.
Very, very quickly, the Facebook mass of scale,
contingent experiments, anybody remember that,
2012, 2013?
In 2012 they planted some simple cues on Facebook pages
to see if they could get more people to go vote in the
midterm elections.
The answer was, yes, we can.
In 2013, while the whole world was erupting in
outrage over the published findings of the 2012
experiment, 2013 they were conducting another one,
this one planting civil malfeasance to see if they
could get people to feel happier or sadder.
Do you remember that?
So it turns out in both cases they were
successful.
And when they wrote up the conclusions in prestigious
scholarly journals they celebrated two facts, one,
we now know that we can engineer subliminal cues
on Facebook pages in the online milieu, including
the engineering of social comparison dynamics, in
order to change real world behavior and feeling.
Number two, we know that we can do this while
consistently bypassing user awareness.
Thus, depriving users of, obviously, choice,
obviously the right to consent, more importantly,
the right to combat.
All right?
Bypassing user awareness becomes a central --
a central plank of economies of action, amplifying the
one-way mirror, social relations that we saw
right from the beginning.
So, you know, it's interesting, and for you,
as regulators and lawmakers, you know, this
has to be very interesting, that one of
the key ways that we find out about this stuff is on
the basis of leaked documents and whistle
blowers.
So why is that?
So now we have another leaked document, this one
a 23-page document written by Facebook executives in
Australia to their Australian and New Zealand
business customers, bragging about the fact,
laying out the -- all these products for these
business customers, these prediction products, where
they understand that they can now -- they can now --
they now have so much data on 6.6 million teenagers
and young adults in those two countries that they
actually can track them through the emotional
cycles of the day and the week.
They can pinpoint when a young person feels
stressed, defeated, overwhelmed, anxious,
nervous, stupid, silly, useless, and a failure.
And they can see the patterns in these
emotional cycles of the week.
They understand by training the models for
the patterns, taking individual data, fitting
them into the pattern.
They can then tell a business customer what
kind of message to send on what day in what hour in
order to provide the antidote to whatever that
salient emotion is in that particular part of the cycle.
So for example, Shoshana feels like a failure,
she's filled with anxiety anticipating her Saturday
night date, it is now late Thursday afternoon,
if you've got a sexy black leather jacket I suggest
you send it to her immediately, with a push,
let her know that you will deliver it to her door
free of charge, and you might as well throw in a
discount coupon.
You will make the sale, okay?
So this is the kind of thing that they're doing.
And in both of these cases what we're seeing is these
increasing experimental zones of economies of
action, learning how to modify human behavior at
scale.
Now, of course, what I just said to you will
sound familiar, I hope it sounds familiar, because
it should echo in your head something called
Cambridge Analytica.
Remember Chris Wylie, whistle blower, talking
about Cambridge Analytica.
He said: We had so much data about so many
individuals we knew what their inner demons were,
and we could -- we could devise triggers, messages
to send to them to trigger those inner demons to get
them to do something that they wouldn't have
ordinarily done, join a group that they wouldn't
have joined, read material that they wouldn't have
read, maybe even vote in a way they wouldn't have voted.
We don't know.
Right?
So this is: What is Cambridge Analytica?
A pivot a few degrees, taking surveillance
capitalism's, daily mechanisms and methods,
bread and butter, pivoting them a few degrees from
commercial to political outcomes.
That's the way it works.
Experimentally developed in Facebook, wholly
imported by Cambridge Analytica.
I just came from Chicago, where I spoke at a
conference over the weekend, at the University
of Chicago, God love it, another speaker at the
conference was Aleksandr Kogan, you remember that
name, right?
The one who got all the Facebook data with his
crackpot personality quiz, and also just happened not
only to get the data of 37 million Facebook users but
of all their friends, so he ended up with about 85
million Facebook users and all their data, all their
data, which he then, most accounts say, sold, he
claims not, so maybe you can find out the truth one day.
But it was using this same mechanisms and methods
that Cambridge Analytica got -- got out of the
starting gate in the very first place.
And when it was my turn to speak I'm proud to say
that I called Aleksandr Kogan out on his practices
and -- and the effort to dazzling whitewash what he
actually did, because he mentioned the fact that he
was really shocked at how upset people were that he
took their data and how they thought that it was creepy.
Hmm.
Okay. So very quickly, you may not know this, the next
big experimental zone for economies of action didn't
come from Facebook, it came from Google, in the
form of a game.
How many people in this room went out in the
streets with family and friends and played
Pokemon-Go?
Come on, fess up, fess up, fess up.
Okay.
Did you know that Pokemon-Go was incubated
in Google over many years?
Did you know that it was designed, engineered by
its famous street view boss John Hanke, who
famously told the Villages of Bon, England, when they
came out to the borders of their village and tried to
block the street view cars, and John Hanke said
to them, and I'm paraphrasing now, get over it.
The west has decided this is how the future's going
to be so get with the program you villagers.
Okay.
So yes, and before that, of course, John Hanke
invented Google Earth, it used to be called keyhole,
it was owned by the CIA, purchased by Google,
turned into Google Earth, and John Hanke went to
Google and had an illustrious career filling
the pipelines with behavioral surplus from
every aspect of human behavior.
So it turns out that Pokemon-Go was a game
within a game.
And that, in fact, the bigger game was that
establishments, not only McDonalds and Starbucks,
all over the world, but also Joe's Pizza and Ed's
Tire Shop were paying Pokemon-Go for guaranteed,
what, not click-through rates, because this is the
real world, what's the real world analogy to
click-through rates?
footfall, right?
Guaranteed real bodies, with their real feet,
falling on the floors of my real establishment to
spend real money.
So these establishments, maybe you went to some,
did you go to McDonalds with your kids?
Huh?
No?
Okay.
No?
Yeah, I'm looking over there, yeah, yeah.
So -- so -- so here we have these establishments
having the opportunity to buy lure, l-u-r-e, lure modules.
So the point is that between lure modules and
the rewards and punishments of gamification
they learned how to herd populations of
people through the city streets to their
guaranteed destinations in order Ka-ching, Ka-ching,
to have that transaction, highly lucrative
predictions of footfall.
Okay.
So now, let's get to the end of this.
I'm skipping some things that I was going to say,
because you're going to ask me questions, and
that's when I'm going to say them, because I want
to get to this big final question.
What is to be done?
That's the question everybody has on their minds.
So first of all, I want to say that I, unlike a lot
of other people who listen to me at this stage of my
talk, and who feel depressed and morose,
I am very optimistic.
I am very optimistic about our what happens next.
First of all, because, begin on this note,
surveillance capitalism thrived in what kind of
environment?
A lawless environment.
So it's not like we've been working for 20 years
to stop this thing and it just keeps growing.
The way I see it, we actually hadn't really
started yet.
I mean, yeah, we've had a fine here, a fine there.
We've had some stuff in Europe, you know, we've
recently had a five billion dollar fine here.
But we all know that those fines aren't what's going
to do it.
So, in fact, I'm quite optimistic because this is
sort of the -- the kind of cycle time when you have
massive technological transformation and
democracy, right?
So the -- the tech guys like to say, well,
democracy is slow and, therefore, hopeless.
Democracy impedes competitiveness,
innovation, too slow, will never catch up with us.
I quote Larry Page saying, you can't expect us to
follow laws that were invented before the internet.
I always think of habeas corpus, such an important law.
But in any case, so we have a situation where --
where democracy is slow and that's a good thing.
Democracy is slow because it's democratic, and
because it reflects the interests and experience
of people in our society and how we move to be
aligned with our aspirations, with our
democratic aspirations for our future society.
So -- so it's right that democracy is slow.
The idea that democratic intervention degrades
competitiveness is absolutely -- it would be
laughable if it wasn't one of the most pernicious
myths perpetuated by -- by the surveillance
capitalists, because, in fact, it is the
surveillance dividend that has been the great enemy
of competition in these spaces.
There are hundreds, thousands, hundreds of
thousands of businesses, of entrepreneurs, of
people with great ideas, who want to be taking
advantage of digital technology to actually
address our needs and to address the needs of society.
But right now, unless they are willing to go after
the surveillance dividend, their chances of getting
the investment support that they need, their
chances of getting the -- the analyst support that
they need, are very, very low.
So it's, in fact, the surveillance dividend
essential to this logic that it suppressing
competition.
So the way I look at it, the big challenge for us
is how do we interrupt and outlaw this logic so that
we eliminate the ascendence for the
surveillance dividend?
And when we do that then stand back and watch
competition flourish?
That's my conclusion.
How do you do that?
Well, first of all, you aim at supply and you aim
at demand, supply and demand.
I started to say this to you before, every time you
tell people about what's going on backstage they hate it.
Every piece of research, every survey, every
interactive thing that's written about every time
people find out about these backstage operations
they hate it.
They want encryption.
They say, how do I get away from it?
And so on and so forth.
But they keep using it.
And so the tech companies point to this and they
say, see, they actually -- they actually love us.
They're still using us.
And their researchers, academics point to this
and they say: This is what we call the privacy
paradox.
There's no paradox, here, folks.
We continue to use this for 16 reasons, but I'm
only going to mention three right now, one is
because it's unprecedented and most people still have
no idea what's going on.
Number two, because a tremendous amount of
surveillance capital is used to engineer our
ignorance to keep us not knowing what's going on.
And number three, it's called the dictatorship of
no alternatives.
They have created a world of no escape.
Ergo, my very first sentence, the internet is
now owned and operated by private surveillance
capital, a world of no escape, I want to make
plans with my friends and family for dinner, I'm
marching through their supply chains.
I want to get my health data from my doctor's office?
I'm marching through their supply chains.
I want to get my kids grades from school?
I'm marching through their supply chains, the Google
platform.
In fact, two weeks ago I was on a government
website, trying to download something that
I had to send to my German publisher, an IRS kind of
form, and my tracking thing in the corner of my
screen, three trackers blocked, five trackers
blocked, on a US government website.
Okay.
Owned and operated by surveillance capitalism.
No exit.
So my view is, how do we -- how do we make sense of this?
Well, the only way to make sense of this is to say
that surveillance capitalism is a market
failure for all the ways in which it's been
rewarded for the surveillance dividend.
When you actually talk about supply and demand,
the way capitalists have always talked about supply
and demand, i.e., you know, because the big
change now is -- you notice this, right --
we're not the customers and we're not the product.
We are the raw material.
We are now human nature to be harvested.
That's it.
Abandon the carcass, harvest the predictive
signals, that's it.
So -- so this is a market failure, because it's not
really meeting our needs.
It's about us, but not for us.
Okay.
So -- so how do we -- how do we interrupt this?
Well, my view is we begin with supply at the
beginning of that value chain I described to you
and we end with demand.
By supply I mean -- I'm not talking about data
ownership, accessibility, and portability.
If we -- if we start off talking about data we've
already lost.
We have to go back to that very first step -- whoops
-- that very first step, which was -- what was that
very first step?
Claiming -- remember the new commodity -- claiming
private human experience and translating it into data.
If we start already arguing about data that's
like arguing with US Steel about the number of hours
a day a child should work in that steel mill.
That's not an argument we had, we said,
comprehensive child labor laws, federal, not state.
All right?
So we don't want to just bargain about the data.
We want to go right to the start where data is
illegitimately taken, that's actually called
stealing, that's where we need to start, because,
you know what, they have no right to my face,
because with my face they get my emotions, with my
emotions they predict my behavior.
I say I don't want them to have my face.
If I should decide that somebody can have my face
for some kind of analysis, maybe I've got a palsy and
I want to contribute to the disease group that's
going to work on that palsy, that's my decision.
It doesn't get taken from me.
That's my decision.
Okay.
So we interrupt supply, at the far end, what's the
furthest end we talked about?
Right, these markets.
Markets that trade in human futures.
And I've been talking to you about the competitive
dynamics of those markets and how they lead to these
imperatives.
Remember, what were they, scale, scope, action,
right?
With all the pernicious effects of knowledge
pivoting into power in ways that are
fundamentally undermining to democracy.
Why?
Because economies of action undermine,
disregard human autonomy, bypassing awareness,
robbing us of the right to combat, a fundamental
assault on individual sovereignty, individual
agency without which a democratic society is
impossible to imagine.
And of course, from above, creating these
unprecedented asymmetries of knowledge and the power
that that knowledge affords.
Okay.
So the answer is to make human futures markets
illegal.
We make other kinds of markets illegal.
We say, no trading in human organs, because the
effects are predictably pernicious.
We say, no trading in human slaves, because the
effects are predictably pernicious.
So we can say no trading in human futures because
the effects are predictively pernicious
and inevitably on a collision course with
democracy.
All right.
So that's a lot of what I wanted to say to you.
I really want to hear your questions.
I know we've gone later than everybody wanted
because -- because of the nor'easter, but please let
me -- let me hear your questions and let's bat
some things around, but by all means, ask questions
and challenge, some other material will come out as
we do that.
Can I leave the podium now?
Yes.
Thank you.
I feel like I've just been freed.
I just think he prefers you to stand on this side
so you're in the possible -- that's my value add.
I love having assistance and direction, I find it
very relaxing.
Thank you.
So I do have a question if I -- so I think it is the
case that the only way -- Jeff, can you -- Yes.
Turned it around.
I think it's the case that the only way -- this is a
strong claim -- that you can reliably make money on
the internet today is using data and
advertising.
It's a little strong, but let's just assume that,
for now, if -- if that is the case, when we outlaw
these firm's ability to surveil do we risk under
-- undermining their ability to actually create
value for consumers that we care about, right?
So there's lots of things that these firms do that I
think we value.
I'm a little bit concerned, if we just say
stop, those business -- those businesses are not
going to be able to thrive anymore.
Okay.
Such a good question, of course.
All right.
So one of the -- you know, one of the things that we
really need to get away from is imaging that we
are stuck in this time warp of click-through rates and
micro targeting for online advertising, okay?
Because we have gone beyond that now.
As I intimated, but didn't really have time to get
into it with examples and so forth.
So I want you to think about it this way: Saying
that surveillance capitalism is only the
online ad market is like saying that mass
production is only to make Model-Ts.
It's like that.
It's where it started.
It was the original context.
But we have moved beyond that context.
So let's look at some examples: Wheere we were talking
about -- well, for example, in the insurance
sector, right?
So in the insurance sector we're using digital, right?
So it's not -- it's internet-enabled, but, you
know, get out of your head the idea that it's just
like me on my phone or me on a keyboard, right?
So it's using automotive telematics to track the
minutia of your behavior as a driver to shape your
behavior, modify behavior in realtime, by offering
rewards and punishments in realtime, raising your
premium rate, punishment because you just took that
curve too fast, lowering your premium rate, reward
because you slowed down just as the light was
turning yellow, right?
So -- so, you know -- so -- so that's one kind of
example in the insurance industry.
Now, you also want to be thinking about these
ecosystems.
Because now we have, for example, firms that
specialize in -- I was writing this years ago,
and I called it surveillance as a service,
you know, because people write about software as a
service, but surveillance, SVAAS, I thought I'll just
riff on that, surveillance as a service.
Lo and behold, a couple years later Amazon -- when
Amazon decided to go over to become not only a
ruthless capitalist but also now a ruthless
surveillance capitalist -- they started calling their
new products and services: Surveillance as a service,
as if, like, that was a good thing.
But that's definitely not the way I meant it.
But now we have firms that are, you know, in these
ecosystems specializing in different niches of
behavioral surplus.
So, for example, there are firms that will just get
data about peoples' renting behavior, and then
they -- and then they, you know, they will have
deals, maybe with IBM or with some other AI, and
they come up with predictions, and then they
sell either the raw data flows or they sell the
prediction products to these -- these home -- you
know, home development companies, landlords who
specifically have, you know, Joe Schmo just
applied for an apartment, and they're using these
data to be able to predict what kind of renter will
Joe be.
So these are just a couple of examples, we're seeing
them in the health, in health insurance, we're
seeing them in education, we're seeing them in real
estate, we're seeing them in transportation,
different ways in which we're using the digital
surround, right, to produce the data flows,
either selling those niches of data flows or
turning them into products that are then sold in
these markets.
So -- and they're changing the face of every
industry, right?
So -- so what's -- what's -- what is absolutely
right about what you said is that it's hard to make
money without the surveillance dividend.
And that's now starting to be true in the insurance
industry just as, you know, starting 20 years
ago it turned out to be true in Google and search,
you know, and then Zuckerberg figured out
that he needed to do the same thing because he
couldn't figure out how to make money fast enough.
And so that's when he hired Sheryl Sandberg, and
I call her the Typhoid Mary of surveillance
capitalism because she brought it to Facebook and
we kind of went from there.
So -- so -- so we're -- so -- okay.
So that's one part of your question.
The second part of your question is, are we going
to mess up these things that we value?
Okay.
I have a good answer and -- and then another
answer.
The good answer is that there are I don't know how
many people waiting in the wings who want to do
social network for the people who use it without
the -- the varieties of consequences that Facebook
has produced by following this economic logic.
I don't have to repeat the news headlines of the past
two years to you as these consequences have become
more clear to the public.
But Facebook is -- is an ungoverned entity that is
now deciding on the, you know, the content of
social connection all around the world, to the
great detriment of governments, of
democracies, and of individual human beings.
So -- so -- and again it's not because Mr. Zuckerberg
began as an evil guy.
I don't know him, can't opine.
Point is that these are economic imperatives with
predictable consequences, that's what I'm getting at
and that's why we need to interrupt them.
So -- so can we do social networks without the
overhang of the surveillance dividend and
its consequences?
Absolutely we can.
Are there people eager to do it now?
Yes.
Has innovation in that space ground to a halt?
Yes.
Why?
Because any time there's another platform Facebook
buys it or, in the case of Snapchat, they imitate it.
So they have killed innovation in that space
and everything is now this one vast behemoth which,
yes, from a certain point of view it, you know,
looks like a classic monopoly.
But it's -- it's not, because it's a huge size,
right?
Because they actually employ fewer people today,
at their height of market capitalization, than
General Motors employed at the height of the Great
Depression.
So it's not -- you know, it's not size.
It's data flows.
It's behavioral surplus flows.
That's what they're monopolizing.
Okay.
So let's -- so obviously we could keep going, but
that's some -- some response.
Were any of those -- Yeah.
-- potential competitors that Facebook or Google
bought, were any of them, like you mentioned
Snapchat, based on an economic model that was
something other than surveillance capitalism?
Yes.
Well, yes, because, you know, WhatsApp was all
about encryption, and Snapchat was the
disappearing thing, so you know these were --
They were not collecting the behavioral data to then
try to sell to advertisers at that time?
They do now.
They do?
But they were not at the beginning.
And this -- this is why, for example, you know
people were shocked at the price that Mark Zuckerberg
paid for WhatsApp because it wasn't yet making money.
And he -- he -- I -- it was in the billions, I
forget exactly what it is, I have it here in my
notes, but the point is, you know, he paid what was
then regarded as an outlandish price.
But again, from the point of view of this economic
logic, it's very easy to understand why that made sense.
Are you with me on that?
Yeah?
Yes.
So -- and then when it -- and then when Snapchat
became so popular, because it was disappearing, and
that was pulling people away from Facebook, who
were already, you know, feeling the malaise of
this sense of, you know, multidimensional surveillance.
And -- and then, of course, they -- they
adapted that and -- so -- so the short answer is,
yes, and there are lots of interesting details there.
But the short answer is yes.
Yes?
So when you talked earlier in your lecture about how
this is my face, it's my right to my face, and I
give people permission to use it or not, it sounded
to me like you were describing an opt-in
regime on the privacy side, in which a lot of
countries have done that, and it hasn't changed
consumer behavior.
So how do you break that?
Not just about giving -- it cannot be just about
giving permission, because we do it all the time when
it comes to our data.
Yeah.
Well, you see, you're in -- I just spent
practically the last month in Europe, and, you know,
you -- so you're on a website, and thanks to
GDPR there's that box, and it says blah, blah, blah,
blah, blah, blah, blah, blah, blah, but then you
still don't get to move the next step unless you agree.
So you're a little bit more informed, a lot more
depressed, and then you're still clicking agree.
So I've spent a lot of time in Brussels,
you probably know the name Giovanni Buterelle,
who was a colleague that I admired
greatly, and we spent a lot of time together in
these discussions, and we actually launched my book
in Brussels in January, and a lot of folks there
were members of European Parliament, you know,
a lot of people who had worked on GDPR, and they
wanted to hear that they've been working on
this for so long that GDPR was the answer, time to
pop the champagne, time to put our feet up, you know,
we can relax for awhile.
And I came in and said, GDPR is only the
beginning, not the end, data ownership, data
accessibility, data portability are not going
to do us any good, for two reasons, number one, it
only applies to the data we give them.
Remember the trillions of data points ingested every hour?
Tiny fraction of that is what we give them.
Most of that is what they've hunted from us
that we don't even know about, so that's number one.
Number two is, even if we give back what we gave
them we're never going to get those six million
predictions per second or what happens to them when
they find their way into human futures markets and
how they're used.
We're never going to get those trillions of data
points or insights into those training models,
because that is the proprietary core, that's
what's driving the value of the predictions, and
that's what's driving the revenue streams out of
these futures markets.
So that's -- that's never coming to us.
And Giovanni knew that.
And at the end we have this big illustrious panel
introducing the book and so forth, he was the
moderator, and at the end he surprised everybody in
this vast hall, and he asked for the mic, and he
-- he spoke to the room, unscripted, and he said --
paraphrasing now -- we thought we were at the
frontier but we're only at the beginning.
And we now understand what the -- what the frontier
really is, because we're moving into a situation
where, you know, data is -- knowledge is used for
power, we have to contest the very most fundamental
mechanisms that produce these data in the first place.
And the fact that they are being used in markets that
actually fund our own domination.
So you know, it was a very quiet hall that day.
But I think the point here is that as I move around
the world, talking to lawmakers here, but even
more so in Europe, where I have more experience in
that -- in that zone, you know, people are
understanding that we've only just begun and that
notice and consent is just as empty a set of
mechanisms under GDPR as it -- as it is here with
our kabuki notice and consent operations.
And that as far as GDPR goes everything right now
depends on the plaintiffs.
Everything right now depends on how people in
communities, you know, take up the regulations
that are there and bring them before the courts and
how the courts build on them and what sense they
make of them, right?
But ultimately we're going to have to go further.
Maybe some of you saw Margrethe Vestager
interview last week, where, you know, she's
coming into her second fifth year term as the
competition Czar of the EU commission, and she was
interviewed -- she did a big press thing last week, and
she said, you know, antitrust is an important
tool in our toolkit, but we -- we've also got to
understand surveillance capitalism and we've got
to figure out how we're going to deal with that.
So I think there's lot of movement there and a lot
of understanding that certainly notice and
consent in any way it currently exists on planet
Earth is not going to get us where we need to go.
So we mentioned this idea that using data with the
human experience to improve product's quality
of consumer experience, is something that you
recognize is there, but then you kind of crafted
that and didn't come back to it.
Yeah.
So I guess the question is if the proposal is to
effectively ban markets from commodatizing human
experience would -- would you -- is there a way that
you could break it off, and at least capture the
benefits to bring products and services that might
come if you allow companies to rely in some
form on that, or would it be, look, we have to ban
it outright because ultimately the benefits of
doing so would outweigh the costs?
Okay.
So I -- I brought this to show just in case there's
an opportunity.
So this is an expensive piece of advertising real estate.
It's the back cover of New Yorker magazine.
And it's a Google ad.
And it says, at the top, helping you, and at the
bottom, every day, and in between are 17 things that
Google is doing to help us, okay?
Lest you think I'm biased, shout out a number.
Give me a number between one and seventeen.
Six.
One, two, three, four, five, six, unsend that email
Give me another number.
12.
12?
Okay.
One, two, three, four, five, six, seven, eight,
nine, ten, eleven, twelve, move that meeting.
What is my point?
Cancel that meeting, do the floss, take better
pictures at night, find a Thai place open now, name
the 21st president, 17 reasons all like that.
So here's Google, on the most expensive piece of
real estate, telling us all the things that it's
doing for us.
And they're all completely trivial unless you're -- I
don't know, does anyone still use the word yuppie?
Unless you're a yuppie living in San Francisco,
riding the Google bus, most of this is completely
immaterial, right?
So my point about right at the beginning about the
digital versus the economic logic is that we
deserve the digital, as individuals.
As a society we deserve big data.
What should we be using big data for?
We should be using big data to solve the climate
crisis, to figure out how to eliminate -- evaporate
all those little plastic particles that are now
even in arctic snow, to cure cancer.
This is what it was supposed to be for.
I assure you, all of the data to support what I'm
about to say is in this book.
Right now big data is wholly owned and operated
by surveillance capitalism.
There are 10,000 data scientists on Earth who
know how to do this stuff.
Most of them now are going to work for the tech
sector or already work there.
You can't even sign out for a computer science
course now in most colleges and universities
unless you are a declared major and even then they
have to ration courses because there is a brain
drain in our country and in Europe because all the
computer scientists are going to work for private
companies to do this.
So how do we get back so that we get to have what
we deserve?
We are citizens of a democratic society moving
into the third decade of the digital century.
And this is the best they've got for us?
In the meantime, their market capitalization, you
put together the four top surveillance capitalists,
Google, Facebook, Microsoft, and Amazon,
what their market capitalization in the
trillions, over a trillion, I mean, you see, right?
And these -- these markets are extremely lucrative,
right?
They have suffocated competition.
And this was supposed to be the new Guttenberg
century, this was supposed to be the century of
intense democratization and empowerment of the
individual on the strength of the digital.
And instead we have asymmetries of knowledge
and power that literally throw us back to a kind of
futile pattern.
So bottom line is, I think we can handle this.
Number one, because -- I -- I write about what I
call the behavioral value reinvestment cycle, the
very -- before Google discovered surveillance
capitalism they took data off what people were doing
online and they used it to improve search.
They used it to invent translate.
They used it to invent auto complete.
They used it to improve their products and
services, right?
That's great.
That's what every company should be doing.
But then, when they discovered behavioral
surplus, right, that behavioral value
reinvestment cycle became second fiddle to this
massive new operation.
And that's where -- when you -- you know, right
now, 87 percent of Google's revenues and 98
percent of Facebook's revenues come from
behavioral microtargeting, online targeted
advertising, right?
They all come from this logic I just described to you.
And now that improving their -- their products
and services is simply like a lost leader to get
the behavioral surplus flowing through the supply
chains into the pipes into the factory into the markets.
You see what I'm saying?
So it's not an end in itself and that's why it's
become a market failure because it's not giving us
what we really need.
When you mentioned market failure it has a significance --
Yes.
To a room like this.
Yes.
So market failure usually involves, you know, some
kind of remedial action, maybe not necessarily
antitrust, but other things.
Are there examples in other jurisdictions, you
mentioned the GDPR, but are there other countries,
regions that are trying to address this in a
different way, either giving individuals some
kind of property right to it, or are there solutions
other than, you know, banning the market for
this, that's you and your book or since then have
proposed or considered.
Well, Makan, I am keenly interested in the answer
to this question.
And I'm trying to understand that myself.
My travels and, you know, I've also been connected
with the international grant committee, I think
-- and you and I corresponded a little bit
about that -- but, you know, I was at the Ottawa
meeting, and it was interesting at the Ottawa
meeting I came there thinking I would be one
witness among others, and I would, like, have to
pitch this point of view and get people to
understand it, and how do I get people ramped up
quickly.
When I got there I was surprised to find that
almost every member of the committee had a copy of my
book and most of them had thousands of little
Post-It notes sticking out, little dog-eared
pages -- something that I always tell my children to
do when they read a book so you know it's really read.
So surveillance capitalism is a framework.
I was the last witness to testify, because my name
begins with Z, and everybody else was talking
about surveillance capitalism before I even
opened my mouth.
So this had become a framework in those deliberations.
And -- and, you know, obviously I was able to
help build on that and deepen that and so forth.
But, you know, this is -- this is the -- what I see
happening in Europe.
I can't say that the -- the solutions are there
yet, that people know exactly what to do, but I
can say that the understanding is really
growing.
So when -- you know, when the committee first met in
January, in London, and then they published their
report authored by Damian Collins, who is the
British MP, you may recall the headline, you know,
calling Facebook digital -- digital gangster.
And that, itself, is very important, because that
came out in, like, February 2019, calling --
calling these guys digital gangsters.
A year earlier, 2018, I had the almost final draft
of my manuscript, and I had to give it to my
publisher to review, for the legal department to
review.
And they read all the 700 pages and they came back
with one thing they wanted me to do, they wanted me
to take the word theft or thief out of anywhere it
appeared in the manuscript, because they
were scared by that word.
One year later, Damian is calling Facebook digital
gangsters.
So that's a big leap in one year.
And that's pretty much down to Chris Wylie, and
Alexander Nicks, and all those bad boys, and some
of the things that they did.
And of course Chris coming -- coming back to the
fold, prodigal son, as a whistle blower.
So I feel like from my vantage point, I've been
on the road for 10 months now, from my vantage point
I know two things, one is that audiences in every
city and every country that I've been in I begin
every event -- is this interesting to you?
I don't know.
I'll tell you anyway -- is, just so you know, what
the public -- what's happening with the public,
every single event I begin in exactly the same way, I
say, it's a sunny day or it's a Saturday night, why
are you here with me?
What made you come here?
Think about it.
Reduce it to one word and shout it out.
And so they do that and I get somebody to write it
down, I get the same list of words in every city in
every country, starting in January, ending two days
ago, last time I did this was two days ago, in
Chicago, and it's the same list of words, anxiety,
fear, control, freedom, democracy, autonomy,
malaise, revolution, resistance, law,
regulation, same list of words.
In fact, if you don't believe me, I carry it
around in my book here.
I can show you what people have written down, same
list of words.
So one thing I know is that they call us users,
like an anonymous group that is only defined in
relationship to the mechanisms of this
economic logic.
But people are no longer experiencing themselves
that way.
People are discovering their shared not only
economic but -- but social and political and even
psychological interests, and coming together out of
that, that -- that -- those interests, and
looking for how to name it and how to understand it,
and wanting their governments, wanting their
democratic institutions, wanting their lawmakers to
take action.
This is not the future that we want.
The experimental zones for economies of action, it
began online, moved to Pokemon-Go.
The next phase of that experimentation is the city.
And that work is already underway.
They used to call it the Google city, now they're
afraid to call it that.
And so now they just call it the Smart city.
And now they're in a contest right now with
many of the citizens right now in Toronto trying to
get the Toronto waterfront as the first exemplar of
the Google city.
And you look, you read those proposals in detail
-- as unfortunately that I have done -- and
they're everything that I just said to you in spades.
So -- so this is moving.
And the end game here is really substituting
computational governance for democratic governance.
This is where we're going, computational governance,
with a view to commercial outcomes, to democratic
governance.
And the city is the first effort.
So I know that the public is on the move.
And the other thing is that I feel like lawmakers
are engaging in a discussion that has not
been engaged in for 20 years.
And from where I sit, the solutions are not clear yet.
There's this piece of legislation and that bill
and this bill and that bill, but so far I haven't
seen anything that is trenchant enough to move
in this direction, but that things are happening
where they have not been happening is undoubtedly
the case, not only here but in Europe.
Do you see the solution as needing to be something
that's an international effort among many people?
Because I know -- I mean, who's the US put our
values on other nations and whatever, and so what
about nations that are aren't democracies and
aren't, you know, subscribers to that form
of government, and I know the GDPR has issues with
American companies complaining that they also
need to put those click-ins, because the
person in EUX's website, they would have to do that
as well.
So I guess how do you see that international level
working out to actually make this happen?
Well, a purely international new
framework is unlikely, because part of the globe
consists of China and other authoritarian
countries who have a very different relationship to
what I've just described to you in China, the state.
Obviously China is not a democratic civilization,
never had democracy, not -- you know, so we are --
we're talking about apples and oranges or what's
further away than that, apples and porcupines or
something like that.
So you know, in China the decision has been seeing
this tremendous knowledge and power, specifically
instrumentarian power coming out of the market
sector in China, and deciding that that power
is too big to be wasted on commercial outcomes, and
essentially annexing it to the state, currently in an
experimental phase, again, social credit system and
so forth.
But -- so the idea that you know -- the idea is
not to do -- compete with China, to do what China is doing.
The idea is to found a democratic alternative to
a future which is a future that is very congenial to
the surveillance capitalists.
And that is a future where they get to continue to do
what they're doing and process the knowledge and
power that they produce.
And also the state uses it for its ends and means,
right?
And that -- so I mean that is essentially in China
we see the conflation of market and state,
the conflation of instrumentation and power
and state power.
That is not the future that we are anticipating
as a democracy.
That is not the future that we -- not only that
we want but that is the future that all of us
should be mobilized to prevent, because that is
an ugly end game.
And that's when Orwell does come back into the
picture, not quite as -- as the old big brother,
but certainly a new kind of totalitarianism.
So that -- so what is the alternative to that?
Right now I -- I have a folder, I left my
briefcase in the car, but I have a folder like this
of all these examples of how we already, here in
our country, are producing examples of this sort of,
you know, very dangerous market state conflation,
you know?
So it's -- it's not -- it's not out of the
question that it could happen here in our county.
So this is something that we have to understand and
we have -- and we have to be -- so it's not going to
be a global solution.
But it could be a democratic solution that
will be an alternative and a beacon in the face of
authoritarian regimes that annex these powers to
their total statistic aims.
And where do you think everybody's going to want
to live? I mean.
How about the philosophical comments,
one I wonder, aren't many of these companies
actually not profitable?
So I'm wondering what you see as the role of
financial market, for example, in 2000 you had
companies with vast revenue growth and
earnings with financial market reckoning similarly
disinvestment campaigns, whose companies were not
aligned with our values, tobacco and other products
along those lines, and third, I wonder if there's
potentially a distortive effect --
I'm having trouble hearing you.
I'm wondering if there's a potentially distortive
effect with anthropomorphizing
corporate behaviors, using words like theft and
punishment, where I think is maybe incentives and
disincentives would more accurately describe some
of this conduct.
Indeed, I think incentives is a very apropos word.
I think when we talk about disrupting human futures
markets we're talking about it eliminating the
incentives that drive the surveillance dividends.
So that certainly is a language that I embrace.
As far as behavior modification regimes go,
you know, behavior modifications is scheduled
of reinforcements and includes rewards and
punishments.
So that's kind of why I use those languages.
And this language, for example, is familiar to
the insurance companies, their own materials use
that kind of language.
As far as anthropomorphizing the
companies?
I completedly agree, which is why I'm stressing, this is an
economic logic and no matter what we think about
individual companies or individual executives
it would be wrong to reduce the problem to either of those.
This is an economic logic that now runs across.
And I'm sure many of the people that are
participating in this economics logic are
fantastic people who are waiting for an opportunity
to be liberated from the necessity of producing a
surveillance dividend, whether they're already
doing it or whether they are on the road doing it,
trying to do it and being revenue positive and,
you know, not yet making profit.
So have I exhausted you?
This is sort of similar to something that was brought
up early on, and I wonder, these companies, as you
point out, several many of them have trillion dollar
capital and then tons of shareholders, and we have
our TSP funds, and I'm sure many people's 401(k)s
are invested in these companies, and if you do
knock off some of their valuation I'm just
wondering that their incentive is to do a lot
of lobbying along the lines of you're going to
impact everybody, not just us.
Oh, yeah, I mean, you know, none of this is
going to happen without a fight, that's for sure.
We're talking about a lot of capital, here, but
we're also talking about a lot of force on the other
side that may not be well represented by lobbyist
right now, not only the public, but many, many
other businesses that -- that are -- that are not,
you know -- the businesses that want to proceed
differently are not organized together the way
businesses that want to keep expanding the
surveillance dividend have organized, right?
So we've got -- we've got a gap there.
I've had conversations with some very large
corporations who are not in the surveillance capitalists
game, and I am very hopeful that they will get
themselves better organized and they will be
walking around on Capitol Hill and knocking on
people's doors the way the other lobbyists are.
So I think, again, this is part of their historical
lag that we're working on, I also think that we're
increasingly going to see citizens caring about this
and voting with these issues in mind.
And the 2020 elections may be the first where some of
this actually begins to get a front row seat and a
dialogue about what kind of future we're going to
have and what kind of society are we going to be.
And are we going to be a society of certainty or a
society of freedom.
And where on that spectrum we're going to fall?
Is it going to be under democratic governance or
under the governance of companies that have no
democratic brief and -- and no shareholder brief,
for that matter.
So, you know, this is -- this is a dynamism,
this is part of being at a -- at a threshold in history
when the answers are not all cooked and -- and
we have to cook them.
Well, thank you so much.
[APPLAUSE]
You've certainly given us a lot
to think about, a lot to lose sleep over, and --
and -- but for us, and especially our economists
to think about, you know, is this -- are the issues
raised something that can be addressed for the
antitrust laws?
As you were talking about a lot of this I couldn't
help but think about is the data being used,
perhaps, to predict between competing
companies, you know, what is the future activity of
one competitor in response to a lot of competition is
premised on the fact that if I lower my price I
might gain market share.
And you come up to a point where you maximize your
profit by stealing market share from a competitor.
But that's -- then you hope that the other person
lowers the price and the consumer ultimately
benefits.
You know, if some of this data can be used between
competitors, because it might be to their
advantage, if they know the other person is not
going to lower the price but maybe I'll increase
the price by a dollar if I know that the competitor's
going to increase it, and I don't have to have an
agreement or a policing mechanism for that cartel
activity, it makes me worry yet another way
about the competitive aspects and this type of issue.
So it's something that we need to be doing, that
we have our chief of our technology section,
Aaron Hogue sitting there, and our chief economist here,
Jeff Wilder.
They both will figure this out in the next week or
two and we'll have the clear crystal answer.
I cannot thank you enough for making the time --
Thank you.
-- to come and speak to us.
And I think everybody who is here and everyone
who will watch it, I know our deputy attorney general,
who's very much in tune with these issues,
has expressed an interest, had a desire,
had initially planned to be here and wanted to listen,
and I think he will be watching this, as well,
to see kind of the scope of the issues we're facing as
a matter of, I think, broader public policy that
you touch on.
So thank you again.
Thank you.
We'll look forward to working with you as you
continue to seek for the right answers.
Thank you.
Thank you so much.
Thank you. Again.
[APPLAUSE].