I’m very pleased to post a draft of my forthcoming essay with Professor Woodrow Hartzog (BU Law), Kafka in the Age of AI and the Futility of Privacy as

Quelle: Kafka in the Age of AI and the Futility of Privacy as Control

Quelle: The Social Dilemma

Set in the dark underbelly of Silicon Valley, The Social Dilemma fuses investigative documentary with enlightening narrative drama. Expert testimony from tech whistle-blowers exposes our disturbing predicament: the services Big Tech provides-search engines, networks, instant information, etc.-are merely the candy that lures us to bite. Once we’re hooked and coming back for more, the real commodity they sell is their prowess to influence and manipulate us.

Mark Zuckerberg’s manifesto outlines his vision for a centralised global colony ruled by the Silicon Valley oligarchy. I say we must do the exact opposite and create a world with individual sovereignty and a healthy commons.

English | Nederlands | Français | Catalan | Español | Deutsch

A hand holding a mobile phone – the symbol of cyborg rights
We are sharded beings.
Let’s build a future where we own and control all aspects of our selves.

Mark Zuckerberg has released a manifesto titled Building Global Community in which he details how he – one of the top 8 billionaires in the world – and his byzantine American/multinational corporation, Facebook Incorporated, are going to solve all of the world’s problems.

In his grand vision for humanity, Mark keeps returning to how Facebook fundamentally “brings us closer together” by “connecting friends and families.” What Mark fails to mention is that Facebook does not connect people together; Facebook connects people to Facebook, Inc.

Facebook’s myth: that it connects you to each other.
Facebook: the myth
Mark wants you to think Facebook connects you to each other.
Facebook’s reality: it connects you to Facebook, Inc..
Facebook: the reality
Facebook connects you to Facebook, Inc.

Facebook’s business model is to be the man in the middle; to track every move you, your family, and your friends make, to store all that information indefinitely, and continuously analyse it to understand you better in order to exploit you by manipulating you for financial and political gain.

Facebook isn’t a social network, it is a scanner that digitises human beings. It is, for all intents and purposes, the camera that captures your soul. Facebook’s business is to simulate you and to own and control your simulation, thereby owning and controlling you.

Where Mark asks you to trust him to be a benevolent king, I say let us build a world without kings.

I call the business model of Facebook, Google, and the venture-capital-funded long tail of Silicon Valley startups “people farming”. Facebook is a factory farm for human beings. And Mark’s manifesto is nothing more than a panicked billionaire’s latest sophomoric attempt to decorate an unpalatable business model grounded in the abuse of human rights with faux moral purpose to stave off regulation and justify what is unabashedly a colonial desire: to create a global fiefdom by connecting all of us to Facebook, Inc.

Avoiding a Global Colony

Mark’s manifesto isn’t about building a global community, it is about building a global colony – with himself as king and with his corporation and the Silicon Valley oligarchy as the court.

Facebook wants us to think that it is a park when it’s actually a shopping mall.

It is not the job of a corporation to “develop the social infrastructure for community” as Mark wants to do. Social infrastructure must belong to the commons, not to giant monopolistic corporations like Facebook. The reason we find ourselves in this mess with ubiquitous surveillance, filter bubbles, and fake news (propaganda) is precisely due to the utter and complete destruction of the public sphere by an oligopoly of private infrastructure that poses as public space.

Facebook wants us to think that it is a park when it’s actually a shopping mall. The last thing we need is more privately owned centralised digital infrastructure to solve the problems created by an unprecedented concentration of power, wealth, and control in a tiny number of hands. It’s way past time we started funding and building the digital equivalents of parks in the digital age instead of building ever-larger shopping malls.

Others have written detailed critiques of Mark’s manifesto. I will not repeat their efforts here. Instead, I want to focus on how we can build a world that stands in stark contrast to the one in Mark’s vision. A world in which we – individuals – instead of corporations, have ownership and control of our selves. In other words, where we have individual sovereignty.

Where Mark asks you to trust him to be a benevolent king, I say let us build a world without kings. Where Mark’s vision is rooted in colonialism and the perpetuation of centralised power and control, mine is based on individual sovereignty and a healthy, distributed commons.

Individual Sovereignty and the Cyborg Self

We can no longer afford the luxury of not understanding the nature of the self in the digital age. The very existence of our freedoms and democracy depend on it.

We are (and we have been for a while now) cyborgs.

We must resist any attempt to reduce people to property with the greatest of fervour.

In that, I don’t mean to conjure up the stereotypical representation of cyborgs as prevalent in science fiction wherein technology is implanted within biological tissue. Instead, I offer a more general definition in which the term applies to any extension of our minds and our biological selves using technology. While technological implants are certainly feasible, possible, and demonstrable, the main way in which we extend ourselves with technology today is not through implants but explants.

We are sharded beings; the sum total of our various aspects as contained within our biological beings as well as the myriad of technologies that we use to extend our biological abilities.

An illustration showing a person’s phone within the boundaries of her self.
We must constitutionally protect the dignity and sanctity of the extended self.

Once we understand this, it follows that we must extend the protections of the self beyond our biological borders to encompass those technologies by which we extend our selves. Wherefore, any attempt to own, control, and trade in these technologies by third parties is an attempt to own, control, and trade in the constitutional elements of people. It is, in short, an attempt to own, control, and trade in people.

Needless to say, we must resist any attempt to reduce people to property with the greatest of fervour. For to not do so is to give our tacit consent to a new slavery: one in which we do not trade in the biological aspects of human beings but their digital aspects. The two, of course, do not exist apart and are not truly separable when manipulation of one necessarily affects the other.

Beyond Surveillance Capitalism

Once we understand that our relationship to technology is not one of master/butler but cyborg/organ; once we understand that we extend our selves with technology and that our technology and data lie within the boundaries of the self, then we must insist that the constitutional protections of the self that we have enshrined within the Universal Declaration of Human Rights and implemented within our myriad of national laws are extended to protect the cyborg self.

It also follows, then, that any attempt to violate the boundaries of the self must be considered an assault on the cyborg self. It is exactly this abuse that constitutes the everyday business model of Facebook, Google, and mainstream Silicon Valley-inspired technology today. In this model, which Shoshana Zuboff calls surveillance capitalism, what we have lost is individual sovereignty. People have once again become property – albeit in digital, not biological, form.

To counter this, we must build new infrastructure to enable people to regain individual sovereignty. Those aspects of the infrastructure that concern the world around us must belong to the commons and those parts that concern people – that make up the organs of our cyborg selves – must be owned and controlled by individuals.

So, for example, smart city architecture must be in the commons and data about the world around us (“data about rocks”) must belong to the commons, while your smart car, smart phone, smart watch, smart teddy bear, etc., and the data they collect (“data about people”) must belong to you.

An Internet of people

Imagine a world where everyone has their own space on the Internet, funded from the commons. This is a private space (an organ of the cyborg self) that all our so-called smart devices (also organs) link into.

Instead of thinking of this space as a personal cloud, we must consider it a special, permanent node within a peer-to-peer structure wherein all our various devices (organs) connect to one another. Pragmatically, this permanent node is used to guarantee findability (initially using domain names) and availability (as it is hosted/always on) as we transition from the client/server architecture of the current Web to the peer-to-peer architecture of the next generation Internet.

Everyone has their own place on the Internet that all their devices connect to.
An Internet of people.

The infrastructure we build must be funded from the commons, belong to the commons, and be interoperable. The services themselves must be constructed and hosted by a plethora of individual organisations – not governments or corporate behemoths – working with interoperable protocols and competing to provide the best service possible to the people they serve. Not coincidentally, this severely limited scope of corporate function marks the entirety of a corporation’s role within a democracy as I see it.

The sole purpose of a corporation should be to compete with other organisations to provide the best service to the people it serves. This is in stark contrast to the wide remit corporations have today to attract people (whom they call “users”) under false pretences (free services wherein they are the product being sold) only to addict them, entrap them with lock-in using proprietary technology, farm them, manipulate their behaviour, and exploit them for financial and political gain.

In the corporatocracy of today, we – individuals – serve corporations. In the democracy of tomorrow, corporations must serve us.

The service providers must, of course, be free to extend the capabilities of the system as long as they share their improvements back into the commons (“share alike”), thus avoiding lock-in. For providing services above and beyond the core services funded from the commons, individual organisations may set prices for and charge for value-added services. In this way, we can build a healthy economy of competition on top of an ethically sound core instead of the system of monopolies we have today on top of an ethically rotten core. And we can do so without embroiling the whole system in convoluted government bureaucracy that would stifle experimentation, competition, and the organic, decentralised evolution of the system.

A healthy economy built upon an ethical core.
A healthy economy built upon an ethical core.

Interoperability, free (as in freedom) technology with “share alike” licenses, a peer-to-peer architecture (as opposed to client/server), and a commons-funded core are the fundamental safeguards for preventing this new system from decaying into a new version of the monopolistic surveillance web we have today. They are how we avoid economies of scale and break the feedback loop between the accumulation of information and wealth that is the core driver of surveillance capitalism.

To be perfectly clear, we are not talking about a system that can flourish under the dictates of late-stage surveillance capitalism. It is a system, however, that can be constructed under present conditions to act as the bridge from that status quo to a sustainable, post-capitalist world.

Building the world you want to live in

In a talk I gave at a European Commission event in Rome recently, I told the audience that we must “build the world we want to live in.” For me, that is not a world owned and controlled by a handful of Silicon Valley oligarchs. It is a world with a healthy commons wherein we – as a community – collectively own and control those aspects of our existence that belong to us all and where we – as individuals – individually own and control those aspects of our existence that belong to our selves.

Imagine a world where you (and those you love) have democratic agency; where we all enjoy basic welfare, rights, and freedoms befitting cyborg dignity. Imagine a sustainable world freed of the destructive short-term greed of capitalism where we no longer reward sociopaths for finding ever more ruthless and destructive ways to accumulate wealth and power at the expense of everyone else. Imagine a free world removed from the feedback loop of manufactured fear and ubiquitous surveillance that has us spiralling deeper into a fresh vortex of fascism. Imagine a world where we grant ourselves the mercy of an intellectually rewarding existence where we are free to explore the potential of our species among the stars.

That is the world that I wake up every day to work towards. Not because it is charitable. Not because I’m a philanthropist. In fact, for no reason at all other than because that is the world that I want to live in.

Lanier writes how he was scared witless when he learned how humans could be conditioned and brainwashed through selective feedback responses. It was then it dawned on him, what he later referred to as “the Thought”, that VR is the ultimate technology for a skinner box; the perfect tool for human manipulation.

Lanier summarised an equation for the terrifying outlook of the ultimate Skinner box, and its reads as follows:

Turing^Moore’s Law * (Pavlov, Watson, Skinner) = Zombie Apocalypse

In other words: if we combine our ever growing computational powers with behavioural manipulation, shit will hit the fan. Those in power of defining the behaviour of the people in the box will have absolute power to bend the world to their will. The result, according to Lanier, will be catastrophic.
In a recent Forbes article, Lanier is quoted:

„If you run [the metaverse] on a business model that’s similar to the one that Facebook runs on, it’ll destroy humanity. I’m not saying that rhetorically. That is a literal and specific prediction that humanity could not survive that. (…) VR can either be beautiful art and sympathy or terrible spying and manipulation. We set its meaning”

Quelle: How Jaron Lanier, “The Father of VR”, Warned us about Meta.

In „The Every,“ Dave Eggers imagines a monopoly so vast that resistance is futile.

Quelle: What if Facebook and Amazon merged? Dave Eggers imagines our dystopian future

What was first? Dave Eggers‘ novel or this reality?

A former Facebook employee has told US politicians that the company’s sites and apps harm children’s mental health and stoke division in society.

Frances Haugen, a 37-year-old former product manager turned whistleblower, heavily criticised the company at a hearing in the Senate.

Facebook has faced growing scrutiny and increasing calls for its regulation. (…)“

https://www.bbc.com/news/world-us-canada-58805965

Henry Farrell

January 16, 2018

This essay is featured in Boston Review’s print issue, Global DystopiasOrder your copy today.

This is not the dystopia we were promised. We are not learning to love Big Brother, who lives, if he lives at all, on a cluster of server farms, cooled by environmentally friendly technologies. Nor have we been lulled by Soma and subliminal brain programming into a hazy acquiescence to pervasive social hierarchies.

Dystopias tend toward fantasies of absolute control, in which the system sees all, knows all, and controls all. And our world is indeed one of ubiquitous surveillance. Phones and household devices produce trails of data, like particles in a cloud chamber, indicating our wants and behaviors to companies such as Facebook, Amazon, and Google. Yet the information thus produced is imperfect and classified by machine-learning algorithms that themselves make mistakes. The efforts of these businesses to manipulate our wants leads to further complexity. It is becoming ever harder for companies to distinguish the behavior which they want to analyze from their own and others’ manipulations.

This does not look like totalitarianism unless you squint very hard indeed. As the sociologist Kieran Healy has suggested, sweeping political critiques of new technology often bear a strong family resemblance to the arguments of Silicon Valley boosters. Both assume that the technology works as advertised, which is not necessarily true at all.

Standard utopias and standard dystopias are each perfect after their own particular fashion. We live somewhere queasier—a world in which technology is developing in ways that make it increasingly hard to distinguish human beings from artificial things. The world that the Internet and social media have created is less a system than an ecology, a proliferation of unexpected niches, and entities created and adapted to exploit them in deceptive ways. Vast commercial architectures are being colonized by quasi-autonomous parasites. Scammers have built algorithms to write fake books from scratch to sell on Amazon, compiling and modifying text from other books and online sources such as Wikipedia, to fool buyers or to take advantage of loopholes in Amazon’s compensation structure. Much of the world’s financial system is made out of bots—automated systems designed to continually probe markets for fleeting arbitrage opportunities. Less sophisticated programs plague online commerce systems such as eBay and Amazon, occasionally with extraordinary consequences, as when two warring bots bid the price of a biology book up to $23,698,655.93 (plus $3.99 shipping). 

In other words, we live in Philip K. Dick’s future, not George Orwell’s or Aldous Huxley’s. Dick was no better a prophet of technology than any science fiction writer, and was arguably worse than most. His imagined worlds jam together odd bits of fifties’ and sixties’ California with rocket ships, drugs, and social speculation. Dick usually wrote in a hurry and for money, and sometimes under the influence of drugs or a recent and urgent personal religious revelation. 

Still, what he captured with genius was the ontological unease of a world in which the human and the abhuman, the real and the fake, blur together. As Dick described his work (in the opening essay to his 1985 collection, I Hope I Shall Arrive Soon):

The two basic topics which fascinate me are “What is reality?” and “What constitutes the authentic human being?” Over the twenty-seven years in which I have published novels and stories I have investigated these two interrelated topics over and over again. 

These obsessions had some of their roots in Dick’s complex and ever-evolving personal mythology (in which it was perfectly plausible that the “real” world was a fake, and that we were all living in Palestine sometime in the first century AD). Yet they were also based on a keen interest in the processes through which reality is socially constructed. Dick believed that we all live in a world where “spurious realities are manufactured by the media, by governments, by big corporations, by religious groups, political groups—and the electronic hardware exists by which to deliver these pseudo-worlds right into heads of the reader.” He argued: 

the bombardment of pseudo-realities begins to produce inauthentic humans very quickly, spurious humans—as fake as the data pressing at them from all sides. My two topics are really one topic; they unite at this point. Fake realities will create fake humans. Or, fake humans will generate fake realities and then sell them to other humans, turning them, eventually, into forgeries of themselves. So we wind up with fake humans inventing fake realities and then peddling them to other fake humans. 

In Dick’s books, the real and the unreal infect each other, so that it becomes increasingly impossible to tell the difference between them. The worlds of the dead and the living merge in Ubik (1969), the experiences of a disturbed child infect the world around him in Martian Time-Slip (1964), and consensual drug-based hallucinations become the vector for an invasive alien intelligence in The Three Stigmata of Palmer Eldritch (1965). Humans are impersonated by malign androids in Do Androids Dream of Electric Sheep? (1968) and “Second Variety” (1953); by aliens in “The Hanging Stranger” (1953) and “The Father-Thing” (1954); and by mutants in “The Golden Man” (1954). 

This concern with unreal worlds and unreal people led to a consequent worry about an increasing difficulty of distinguishing between them. Factories pump out fake Americana in The Man in the High Castle (1962), mirroring the problem of living in a world that is not, in fact, the real one. Entrepreneurs build increasingly human-like androids in Do Androids Dream of Electric Sheep?, reasoning that if they do not, then their competitors will. Figuring out what is real and what is not is not easy. Scientific tools such as the famous Voight-Kampff test in Do Androids Dream of Electric Sheep? (and Blade Runner, Ridley Scott’s 1982 movie based loosely on it) do not work very well, leaving us with little more than hope in some mystical force—the I Ching, God in a spray can, a Martian water-witch—to guide us back toward the real. 

We live in Dick’s world—but with little hope of divine intervention or invasion. The world where we communicate and interact at a distance is increasingly filled with algorithms that appear human, but are not—fake people generated by fake realities. When Ashley Madison, a dating site for people who want to cheat on their spouses, was hacked, it turned out that tens of thousands of the women on the site were fake “fembots” programmed to send millions of chatty messages to male customers, so as to delude them into thinking that they were surrounded by vast numbers of potential sexual partners. 

These problems are only likely to get worse as the physical world and the world of information become increasingly interpenetrated in an Internet of (badly functioning) Things. Many of the aspects of Joe Chip’s future world in Ubik look horrendously dated to modern eyes: the archaic role of women, the assumption that nearly everyone smokes. Yet the door to Joe’s apartment—which argues with him and refuses to open because he has not paid it the obligatory tip—sounds ominously plausible. Someone, somewhere, is pitching this as a viable business plan to Y Combinator or the venture capitalists in Menlo Park. 

This invasion of the real by the unreal has had consequences for politics. The hallucinatory realities in Dick’s worlds—the empathetic religion of Do Androids Dream of Electric Sheep?, the drug-produced worlds of The Three Stigmata of Palmer Eldritch, the quasi–Tibetan Buddhist death realm of Ubik—are usually experienced by many people, like the television shows of Dick’s America. But as network television has given way to the Internet, it has become easy for people to create their own idiosyncratic mix of sources. The imposed media consensus that Dick detested has shattered into a myriad of different realities, each with its own partially shared assumptions and facts. Sometimes this creates tragedy or near-tragedy. The deluded gunman who stormed into Washington, D.C.’s Comet Ping Pong pizzeria had been convinced by online conspiracy sites that it was the coordinating center for Hillary Clinton’s child–sex trafficking ring. 

Such fractured worlds are more vulnerable to invasion by the non-human. Many Twitter accounts are bots, often with the names and stolen photographs of implausibly beautiful young women, looking to pitch this or that product (one recent academic study found that between 9 and 15 percent of all Twitter accounts are likely fake). Twitterbots vary in sophistication from automated accounts that do no more than retweet what other bots have said, to sophisticated algorithms deploying so-called “Sybil attacks,” creating fake identities in peer-to-peer networks to invade specific organizations or degrade particular kinds of conversation.

Twitter has failed to become a true mass medium, but remains extraordinarily important to politics, since it is where many politicians, journalists, and other elites turn to get their news. One research project suggests that around 20 percent of the measurable political discussion around the last presidential election came from bots. Humans appear to be no better at detecting bots than we are, in Dick’s novel, at detecting replicant androids: people are about as likely to retweet a bot’s message as the message of another human being. Most notoriously, the current U.S. president recently retweeted a flattering message that appears to have come from a bot densely connected to a network of other bots, which some believe to be controlled by the Russian government and used for propaganda purposes. 

In his novels Dick was interested in seeing how people react when their reality starts to break down. A world in which the real commingles with the fake, so that no one can tell where the one ends and the other begins, is ripe for paranoia. The most toxic consequence of social media manipulation, whether by the Russian government or others, may have nothing to do with its success as propaganda. Instead, it is that it sows an existential distrust. People simply do not know what or who to believe anymore. Rumors that are spread by Twitterbots merge into other rumors about the ubiquity of Twitterbots, and whether this or that trend is being driven by malign algorithms rather than real human beings. 

Such widespread falsehood is especially explosive when combined with our fragmented politics. Liberals’ favorite term for the right-wing propaganda machine, “fake news,” has been turned back on them by conservatives, who treat conventional news as propaganda, and hence ignore it. On the obverse, it may be easier for many people on the liberal left to blame Russian propaganda for the last presidential election than to accept that many voters had a very different understanding of America than they do. 

Dick had other obsessions—most notably the politics of Richard Nixon and the Cold War. It is not hard to imagine him writing a novel combining an immature and predatory tycoon (half Arnie Kott, half Jory Miller) who becomes the president of the United States, secret Russian political manipulation, an invasion of empathy-free robotic intelligences masquerading as human beings, and a breakdown in our shared understanding of what is real and what is fake. 

These different elements probably would not cohere particularly well, but as in Dick’s best novels, the whole might still work, somehow. Indeed, it is in the incongruities of Dick’s novels that salvation is to be found (even at his battiest, he retains a sense of humor). Obviously, it is less easy to see the joke when one is living through it. Dystopias may sometimes be grimly funny—but rarely from the inside.

This is a rush transcript. Copy may not be in its final form. AMY GOODMAN: A longtime Facebook executive has admitted the company’s platform helped Donald Trump win the 2016 election, and predicted it may happen again this year. In an internal memo, Facebook Vice President Andrew Bosworth wrote, “So was Facebook responsible for Donald […]

“Democracy For Sale”: Cambridge Analytica & Big Tech’s History of Manipulating Elections — Radio Free

https://www.youtube.com/embed/aDx2-mdInhI?version=3&rel=1&fs=1&autohide=2&showsearch=0&showinfo=1&iv_load_policy=1&wmode=transparent

Robert Scheer interviews Noam Chomsky edited by O Society Jan 11, 2019 Robert Scheer: I always say the intelligence comes from my guests; in this case, I’m saying it with great respect and awe. My guest is Noam Chomsky. And actually, this is my first real encounter with this man. But I obviously, as many […]

Noam Chomsky (1): On the Global Dystopia of America — O Society

“… private surveillance capital has institutionalized asymmetries of knowledge unlike anything ever seen in human history. They know everything about us; we know almost nothing about them.” (S. Zuboff)

Democracy Now interviews Shoshana Zuboff: “The Age of Surveillance Capitalism”, Part 1. —