It can just have goals that do us damage.” Read more. An AI doesn’t have to hate humans in the way Hollywood often shows them disliking us. It’s an absurd idea, but it shows the possibility of inadvertent damage. Then it turns its gaze towards the stars and thinks, ‘Well there’s an awful lot of planets out there and I can turn all those into paperclips!’ So it develops a space program and travels around the cosmos and turns the entire universe either into paperclips or something that makes paperclips. After a little while, it realizes, ‘Well these humans, they’re made of atoms, they could be turned into paperclips.’ So it turns us all into paperclips. The AI has the goal of maximizing the production of paperclips. Nick Bostrom is Swedish-born philosopher and polymath with a background in theoretical physics, computational neuroscience, logic, and artificial intelligence, as well as philosophy.

He warns about the possibility not so much of a superintelligence going rogue-like Skynet, or HAL in 2001-but more simply of an immensely powerful entity that would not set out to damage us but have goals that could do us harm…He uses what he calls a ‘cartoon’ example: The first AGI turns out to be developed by someone who owns a paperclip manufacturing company. sort by Note: these are all the books on Goodreads for this author. In human history, wars and plagues have done so on more than one occasion, and misguided ideologies and totalitarian regimes have darkened an entire era or a region. Books by Nick Bostrom (Author of Superintelligence) Books by Nick Bostrom Nick Bostrom Average rating 3.89 18,604 ratings 1,778 reviews shelved 102,597 times Showing 25 distinct works. If we get AGI, the outcome could be absolutely wonderful. A global catastrophic risk is one with the potential to wreak death and destruction on a global scale. He’s said, for a long time, that Kurzweil is half right. “He is a philosophy professor at Oxford University and runs the Oxford Martin School’s Future of Humanity Institute. Foreign Policy & International Relations.

Nick bostrom books