Nick Bostrom
Nick Bostrom | |
---|---|
PhD) | |
Spouse | Susan[1] |
Awards |
|
Era | Self-sampling assumption |
Website | nickbostrom |
Nick Bostrom (
Bostrom is the author of
Bostrom believes that advances in artificial intelligence (AI) may lead to superintelligence, which he defines as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest". He views this as a major source of opportunities and existential risks.[4][7]
Early life and education
Born as Niklas Boström in 1973 in Helsingborg, Sweden,[8] he disliked school at a young age and spent his last year of high school learning from home. He was interested in a wide variety of academic areas, including anthropology, art, literature, and science.[1]
He received a
Research
Existential risk
Bostrom's research concerns the future of humanity and long-term outcomes.
In 2005, Bostrom founded the Future of Humanity Institute which,[1] until its shutdown in 2024, researched the far future of human civilization.[13] He is also an adviser to the Centre for the Study of Existential Risk.[11]
In the 2008 essay collection, Global Catastrophic Risks, editors Bostrom and Milan M. Ćirković characterize the relationship between existential risk and the broader class of global catastrophic risks, and link existential risk to observer selection effects[14] and the Fermi paradox.[15]
Vulnerable world hypothesis
In a paper called The Vulnerable World Hypothesis,[16] Bostrom suggests that there may be some technologies that destroy human civilization by default[a] when discovered. Bostrom proposes a framework for classifying and dealing with these vulnerabilities. He also gives counterfactual thought experiments of how such vulnerabilities could have historically occurred, e.g. if nuclear weapons had been easier to develop or had ignited the atmosphere (as Robert Oppenheimer had feared).[18]
Superintelligence
In 2014, Bostrom published Superintelligence: Paths, Dangers, Strategies, which became a New York Times Best Seller.[19] The book argues that superintelligence is possible and explores different types of superintelligences, their cognition, the associated risks. He also presents technical and strategic considerations on how to make it safe.
Characteristics of a superintelligence
Bostrom explores multiple possible paths to superintelligence, including whole brain emulation and human intelligence enhancement, but focuses on artificial general intelligence, explaining that electronic devices have many advantages over biological brains.[20]
Bostrom draws a distinction between final goals and instrumental goals. A final goal is what an agent tries to achieve for its own intrinsic value. Instrumental goals are just intermediary steps towards final goals. Bostrom contends there are instrumental goals that will be shared by most sufficiently intelligent agents because they are generally useful to achieve any objective (e.g. preserving the agent's own existence or current goals, acquiring resources, improving its cognition...), this is the concept of instrumental convergence. On the other side, he writes that virtually any level of intelligence can in theory be combined with virtually any final goal (even absurd final goals, e.g. making paperclips), a concept he calls the orthogonality thesis.[20]
He argues that an AI with the ability to improve itself might initiate an
Bostrom argues that giving simplistic final goals to a superintelligence could be catastrophic:
Suppose we give an A.I. the goal to make humans smile. When the A.I. is weak, it performs useful or amusing actions that cause its user to smile. When the A.I. becomes superintelligent, it realizes that there is a more effective way to achieve this goal: take control of the world and stick electrodes into the facial muscles of humans to cause constant, beaming grins.[22]
Mitigating the risk
Bostrom explores several pathways to reduce the
Bostrom warns that an existential catastrophe can also occur from AI being misused by humans for destructive purposes, or from humans failing to take into account the potential moral status of digital minds. Despite these risks, he says that machine superintelligence seems involved at some point in "all the plausible paths to a really great future".[7]
Public reception
Superintelligence: Paths, Dangers, Strategies became a New York Times Best Seller and received positive feedback from personalities such as Stephen Hawking, Bill Gates, Elon Musk, Peter Singer and Derek Parfit. It was praised for offering clear and compelling arguments on a neglected yet important topic. It was sometimes criticized for spreading pessimism about the potential of AI, or for focusing on longterm and speculative risks.[23] Some skeptics such as Daniel Dennett or Oren Etzioni contended that superintelligence is too far away for the risk to be significant.[24][25] Yann LeCun considers that there is no existential risk, asserting that superintelligent AI will have no desire for self-preservation[26] and that experts can be trusted to make it safe.[27]
Raffi Khatchadourian wrote that Bostrom's book on superintelligence "is not intended as a treatise of deep originality; Bostrom's contribution is to impose the rigors of analytic philosophy on a messy corpus of ideas that emerged at the margins of academic thought."[23]
Digital sentience
Bostrom supports the substrate independence principle, the idea that consciousness can emerge on various types of physical substrates, not only in "carbon-based biological neural networks" like the human brain.
Anthropic reasoning
Bostrom has published numerous articles on
Bostrom believes that the mishandling of
In later work, he has proposed the phenomenon of anthropic shadow, an observation selection effect that prevents observers from observing certain kinds of catastrophes in their recent geological and evolutionary past.[32] Bostrom claims events that lie in the anthropic shadow are likely to be underestimated unless statistical corrections are made.
Simulation argument
Bostrom's simulation argument posits that at least one of the following statements is very likely to be true:[33]
- The fraction of human-level civilizations that reach a posthuman stage is very close to zero;
- The fraction of posthuman civilizations that are interested in running ancestor-simulations is very close to zero;
- The fraction of all people with our kind of experiences that are living in a simulation is very close to one.
Ethics of human enhancement
Bostrom is favorably disposed toward "human enhancement", or "self-improvement and human perfectibility through the ethical application of science", as well as a critic of bio-conservative views.[34]
In 1998, Bostrom co-founded (with David Pearce) the World Transhumanist Association[34] (which has since changed its name to Humanity+). In 2004, he co-founded (with James Hughes) the Institute for Ethics and Emerging Technologies, although he is no longer involved with either of these organisations.
In 2005, Bostrom published the short story "The Fable of the Dragon-Tyrant" in the Journal of Medical Ethics. A shorter version was published in 2012 in Philosophy Now.[35] The fable personifies death as a dragon that demands a tribute of thousands of people every day. The story explores how status quo bias and learned helplessness can prevent people from taking action to defeat aging even when the means to do so are at their disposal. YouTuber CGP Grey created an animated version of the story.
With philosopher Toby Ord, he proposed the reversal test in 2006. Given humans' irrational status quo bias, how can one distinguish between valid criticisms of proposed changes in a human trait and criticisms merely motivated by resistance to change? The reversal test attempts to do this by asking whether it would be a good thing if the trait was altered in the opposite direction.[36]
Bostrom's work also considers potential dysgenic effects in human populations but he thinks genetic engineering can provide a solution and that "In any case, the time-scale for human natural genetic evolution seems much too grand for such developments to have any significant effect before other developments will have made the issue moot".[37]
Technology strategy
Bostrom has suggested that technology policy aimed at reducing existential risk should seek to influence the order in which various technological capabilities are attained, proposing the principle of differential technological development. This principle states that we ought to retard the development of dangerous technologies, particularly ones that raise the level of existential risk, and accelerate the development of beneficial technologies, particularly those that protect against the existential risks posed by nature or by other technologies.[38]
In 2011, Bostrom founded the Oxford Martin Program on the Impacts of Future Technology.[39]
Bostrom's theory of the Unilateralist's Curse has been cited as a reason for the scientific community to avoid controversial dangerous research such as reanimating pathogens.[40]
Awards
Bostrom was named in Foreign Policy's 2009 list of top global thinkers "for accepting no limits on human potential."[41] Prospect Magazine listed Bostrom in their 2014 list of the World's Top Thinkers.[42]
Public engagement
Bostrom has provided policy advice and consulted for many governments and organizations. He gave evidence to the House of Lords, Select Committee on Digital Skills.[43] He is an advisory board member for the Machine Intelligence Research Institute,[44] Future of Life Institute,[45] and an external advisor for the Cambridge Centre for the Study of Existential Risk.[46]
1996 email controversy
In January 2023, Bostrom issued an apology for a 1996 email where he had stated that he thought "Blacks are more stupid than whites", and where he also used the word "niggers" in a description of how he thought this statement might be perceived by others.[47] The apology, posted on his website,[48] stated that "the invocation of a racial slur was repulsive" and that he "completely repudiate[d] this disgusting email". In his apology, he wrote “I think it is deeply unfair that unequal access to education, nutrients and basic healthcare leads to inequality in social outcomes, including sometimes disparities in skills and cognitive capacity.”[49]
In January 2023, Oxford University told The Daily Beast, "The University and Faculty of Philosophy is currently investigating the matter but condemns in the strongest terms possible the views this particular academic expressed in his communications."[47] In August 2023, the investigation concluded (according to a letter Bostrom posted on his website) that "we do not consider [Bostrom] to be a racist or that [he holds] racist views, and we consider that the apology [he] posted in January 2023 was sincere."[48]
Selected works
Books
- 2002 – ISBN 0-415-93858-9
- 2008 – ISBN 978-0-19-857050-9
- 2009 – ISBN 0-19-929972-2
- 2014 – ISBN 978-0-19-967811-2
Journal articles
- Bostrom, Nick (1998). "How Long Before Superintelligence?". Journal of Future Studies. 2.
- — (January 2000). "Observer-relative chances in anthropic reasoning?". Erkenntnis. 52 (1): 93–108. S2CID 140474848.
- — (October 2001). "The Meta-Newcomb Problem". Analysis. 61 (4): 309–310. JSTOR 3329010.
- — (March 2002). "Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards". Journal of Evolution and Technology. 9 (1).
- — (April 2003). "Are You Living in a Computer Simulation?" (PDF). Philosophical Quarterly. 53 (211): 243–255. JSTOR 3542867.
- — (2003). "The Mysteries of Self-Locating Belief and Anthropic Reasoning" (PDF). Harvard Review of Philosophy. 11 (Spring): 59–74. .
- — (November 2003). "Astronomical Waste: The Opportunity Cost of Delayed Technological Development". Utilitas. 15 (3): 308–314. S2CID 15860897.
- — (June 2005). "In Defense of Posthuman Dignity". Bioethics. 19 (3): 202–214. PMID 16167401.
- with Tegmark, Max (December 2005). "How Unlikely is a Doomsday Catastrophe?". Nature. 438 (7069): 754. S2CID 4390013.
- — (2006). "What is a Singleton?". Linguistic and Philosophical Investigations. 5 (2): 48–54.
- with Ord, Toby (July 2006). "The Reversal Test: Eliminating Status Quo Bias in Applied Ethics" (PDF). Ethics. 116 (4): 656–680. S2CID 12861892.
- with Sandberg, Anders (December 2006). "Converging Cognitive Enhancements" (PDF). Annals of the New York Academy of Sciences. 1093 (1): 201–207. S2CID 10135931.
- — (January 2008). "Drugs can be used to treat more than disease" (PDF). Nature. 452 (7178): 520. S2CID 4426990.
- — (2008). "The doomsday argument". Think. 6 (17–18): 23–28. S2CID 171035249.
- — (2008). "Where Are They? Why I hope the search for extraterrestrial life finds nothing" (PDF). Technology Review (May/June): 72–77.
- with Sandberg, Anders (September 2009). "Cognitive Enhancement: Methods, Ethics, Regulatory Challenges" (PDF). Science and Engineering Ethics. 15 (3): 311–341. S2CID 6846531.
- — (2009). "Pascal's Mugging" (PDF). Analysis. 69 (3): 443–445. JSTOR 40607655.
- with Ćirković, Milan; Sandberg, Anders (2010). "Anthropic Shadow: Observation Selection Effects and Human Extinction Risks" (PDF). Risk Analysis. 30 (10): 1495–1506. S2CID 6485564.
- — (2011). "Information Hazards: A Typology of Potential Harms from Knowledge" (PDF). Review of Contemporary Philosophy. 10: 44–79. ProQuest 920893069.
- Bostrom, Nick (2011). "THE ETHICS OF ARTIFICIAL INTELLIGENCE" (PDF). Cambridge Handbook of Artificial Intelligence. Archived from the original (PDF) on 4 March 2016. Retrieved 13 February 2017.
- Bostrom, Nick (2011). "Infinite Ethics" (PDF). Analysis and Metaphysics. 10: 9–59.
- — (May 2012). "The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents" (PDF). Minds and Machines. 22 (2): 71–84. S2CID 7445963.
- with Armstrong, Stuart; Sandberg, Anders (November 2012). "Thinking Inside the Box: Controlling and Using Oracle AI" (PDF). Minds and Machines. 22 (4): 299–324. S2CID 9464769.
- — (February 2013). "Existential Risk Reduction as Global Priority". Global Policy. 4 (3): 15–31. .
- with Shulman, Carl (February 2014). "Embryo Selection for Cognitive Enhancement: Curiosity or Game-changer?" (PDF). Global Policy. 5 (1): 85–92. .
- with Muehlhauser, Luke (2014). "Why we need friendly AI" (PDF). Think. 13 (36): 41–47. S2CID 143657841.
- Bostrom, Nick (September 2019). "The Vulnerable World Hypothesis". Global Policy. 10 (4): 455–476. .
See also
Notes
- ^ Bostrom says that the risk can be reduced if society sufficiently exits what he calls a "semi-anarchic default condition", which roughly means limited capabilities for preventive policing and global governance, and having individuals with diverse motivations.[17]
- ^ Bostrom notes that "the concept of a singleton is an abstract one: a singleton could be democracy, a tyranny, a single dominant AI, a strong set of global norms that include effective provisions for their own enforcement, or even an alien overlord—its defining characteristic being simply that it is some form of agency that can solve all major global coordination problems"[20]
References
- ^ ISSN 0028-792X.
- ^ "Infinite Ethics" (PDF). nickbostrom.com. Retrieved 21 February 2019.
- ^ "nickbostrom.com". Nickbostrom.com. Archived from the original on 30 August 2018. Retrieved 16 October 2014.
- ^ a b c Shead, Sam (25 May 2020). "How Britain's oldest universities are trying to protect humanity from risky A.I." CNBC. Retrieved 5 June 2023.
- ^ "Nick Bostrom's Home Page". nickbostrom.com. Retrieved 19 April 2024.
- ^ a b "Nick Bostrom on artificial intelligence". Oxford University Press. 8 September 2014. Retrieved 4 March 2015.
- ^ a b "Nick Bostrom on the birth of superintelligence". Big Think. Retrieved 14 August 2023.
- ^ a b Thornhill, John (14 July 2016). "Artificial intelligence: can we control it?". Financial Times. Archived from the original on 10 December 2022. Retrieved 10 August 2016. (subscription required)
- ^ Bostrom, Nick. "CV" (PDF).
- ^ Bostrom, Nick (2000). Observational selection effects and probability (PhD). London School of Economics and Political Science. Retrieved 25 June 2021.
- ^ a b Andersen, Ross. "Omens". Aeon Media Ltd. Archived from the original on 18 October 2015. Retrieved 5 September 2015.
- ^ Andersen, Ross (6 March 2012). "We're Underestimating the Risk of Human Extinction". The Atlantic. Retrieved 6 July 2023.
- ^ "Future of Humanity Institute". web.archive.org. 17 April 2024. Retrieved 17 April 2024.
- S2CID 4390013. Archived from the original(PDF) on 3 July 2011.
- ^ Overbye, Dennis (3 August 2015). "The Flip Side of Optimism About Life on Other Planets". The New York Times. Retrieved 29 October 2015.
- ^ Bostrom, Nick (2018). The Vulnerable World Hypothesis.
- ^ Abhijeet, Katte (25 December 2018). "AI Doomsday Can Be Avoided If We Establish 'World Government': Nick Bostrom". Analytics India Magazine.
- ^ Piper, Kelsey (19 November 2018). "How technological progress is making it likelier than ever that humans will destroy ourselves". Vox. Retrieved 5 July 2023.
- ^ Times, The New York (8 September 2014). "Best Selling Science Books". The New York Times. Retrieved 19 February 2015.
- ^ OCLC 943145542.
- ISSN 0013-0613. Retrieved 14 August 2023.
- ^ a b Bostrom, Nick (27 April 2015), What happens when our computers get smarter than we are?, retrieved 12 August 2023
- ^ a b Khatchadourian, Raffi (16 November 2015). "The Doomsday Invention". The New Yorker. Retrieved 13 August 2023.
- ^ "Is Superintelligence Impossible? | Edge.org". www.edge.org. Retrieved 13 August 2023.
- ^ Oren Etzioni (2016). "No, the Experts Don't Think Superintelligent AI is a Threat to Humanity". MIT Review.
- ^ Arul, Akashdeep (27 January 2022). "Yann LeCun sparks a debate on AGI vs human-level AI". Analytics India Magazine. Retrieved 14 August 2023.
- ^ "Almost half of CEOs fear A.I. could destroy humanity five to 10 years from now—but 'A.I. godfather' says an existential threat is 'preposterously ridiculous'". Fortune. Retrieved 14 August 2023.
- ^ "Are You Living in a Computer Simulation?". www.simulation-argument.com. Retrieved 5 July 2023.
- ISSN 0362-4331. Retrieved 5 July 2023.
- ^ Fisher, Richard. "The intelligent monster that you should let eat you". www.bbc.com. Retrieved 5 July 2023.
- ISBN 978-0-415-93858-7. Retrieved 22 July 2014.
- ^ "Anthropic Shadow: Observation Selection Effects and Human Extinction Risks" (PDF). Nickbostrom.com. Retrieved 16 October 2014.
- ^ Nesbit, Jeff. "Proof of the Simulation Argument". US News. Retrieved 17 March 2017.
- ^ a b Sutherland, John (9 May 2006). "The ideas interview: Nick Bostrom; John Sutherland meets a transhumanist who wrestles with the ethics of technologically enhanced human beings". The Guardian.
- ^ Bostrom, Nick (12 June 2012). "The Fable of the Dragon-Tyrant". Philosophy Now. 89: 6–9.
- S2CID 12861892.
- ^ "Existential Risks: Analyzing Human Extinction Scenarios". nickbostrom.com. Retrieved 6 July 2023.
- ^ Bostrom, Nick (2002). "Existential Risks: Analyzing Human Extinction Scenarios". Journal of Evolution and Technology. Oxford Research Archive
- ^ "Professor Nick Bostrom : People". Oxford Martin School. Archived from the original on 15 September 2018. Retrieved 16 October 2014.
- ^ Lewis, Gregory (19 February 2018). "Horsepox synthesis: A case of the unilateralist's curse?". Bulletin of the Atomic Scientists. Archived from the original on 25 February 2018. Retrieved 26 February 2018.
- ^ "The FP Top 100 Global Thinkers – 73. Nick Bostrom". Foreign Policy. 30 November 2009. Archived from the original on 21 October 2014.
- ^ Kutchinsky, Serena (23 April 2014). "World thinkers 2014: The results". Prospect. Retrieved 19 June 2022.
- ^ "Digital Skills Committee – timeline". UK Parliament. Retrieved 17 March 2017.
- ^ "Team – Machine Intelligence Research Institute". Machine Intelligence Research Institute. Retrieved 17 March 2017.
- ^ "Team – Future of Life Institute". Future of Life Institute. Retrieved 17 March 2017.
- ^ McBain, Sophie (4 October 2014). "Apocalypse Soon: Meet The Scientists Preparing For the End Times". New Republic. Retrieved 17 March 2017.
- ^ a b Ladden-Hall, Dan (12 January 2023). "Top Oxford Philosopher Nick Bostrom Admits Writing 'Disgusting' N-Word Mass Email". The Daily Beast. Retrieved 12 January 2023.
- ^ a b Bostrom, Nick. "Apology for old email" (PDF). nickbostrom.com. Retrieved 17 January 2024.
- ^ Bilyard, Dylan (15 January 2023). "Investigation Launched into Oxford Don's Racist Email". The Oxford Blue.