The architecture of the modern web poses grave threats to humanity. It’s not too late to save ourselves.
Story by Adrienne LaFrance | The Atlantic
The doomsday machine was never supposed to exist. It was meant to be a thought experiment that went like this: Imagine a device built with the sole purpose of destroying all human life. Now suppose that machine is buried deep underground, but connected to a computer, which is in turn hooked up to sensors in cities and towns across the United States.
The sensors are designed to sniff out signs of the impending apocalypse—not to prevent the end of the world, but to complete it. If radiation levels suggest nuclear explosions in, say, three American cities simultaneously, the sensors notify the Doomsday Machine, which is programmed to detonate several nuclear warheads in response. At that point, there is no going back. The fission chain reaction that produces an atomic explosion is initiated enough times over to extinguish all life on Earth. There is a terrible flash of light, a great booming sound, then a sustained roar. We have a word for the scale of destruction that the Doomsday Machine would unleash: megadeath.
Nobody is pining for megadeath. But megadeath is not the only thing that makes the Doomsday Machine petrifying. The real terror is in its autonomy, this idea that it would be programmed to detect a series of environmental inputs, then to act, without human interference. “There is no chance of human intervention, control, and final decision,” wrote the military strategist Herman Kahn in his 1960 book, On Thermonuclear War, which laid out the hypothetical for a Doomsday Machine. The concept was to render nuclear war unwinnable, and therefore unthinkable.
