Jebari, Karim and Lundborg, Joakim (2019) Artificial superintelligence and its limits: why AlphaZero cannot become a general agent. [Preprint]
|
Text
Artificial superintelligence and its limits.pdf - Submitted Version Available under License Creative Commons Attribution No Derivatives. Download (179kB) | Preview |
Abstract
An intelligent machine surpassing human intelligence across a wide set of skills has been proposed as a possible existential catastrophe (i.e., an event comparable in value to that of human extinction). Among those concerned about existential risk related to Artificial Intelligence (AI), it is common to assume that AI will not only be very intelligent, but also be a general agent (i.e., an agent capable of action in many different contexts).
This article explores the characteristics of machine agency, and what it would mean for a machine to become a general agent. In particular, it does so by articulating some important differences between belief and desire in the context of machine agency. One such difference is that while an agent can by itself acquire new beliefs through learning, desires need to be derived from preexisting desires or acquired with the help of an external influence. Such influence could be a human programmer or natural selection. We argue that to become a general agent, a machine needs productive desires, or desires that can direct behavior across multiple contexts. However, productive desires cannot sui generis be derived from non-productive desires. Thus, even though general agency in AI could in principle be created, it cannot be produced by an AI spontaneously through an endogenous process. In conclusion, we argue that a common AI scenario, where general agency suddenly emerges in a non-general agent AI, is not plausible.
| Export/Citation: | EndNote | BibTeX | Dublin Core | ASCII/Text Citation (Chicago) | HTML Citation | OpenURL |
| Social Networking: |
| Item Type: | Preprint | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Creators: |
|
|||||||||
| Keywords: | artificial intelligence, machine learning, super intelligence, general intelligence, agency, intentional action, | |||||||||
| Subjects: | Specific Sciences > Cognitive Science > Action Specific Sciences > Artificial Intelligence > AI and Ethics Specific Sciences > Cognitive Science > Concepts and Representations Specific Sciences > Artificial Intelligence > Machine Learning |
|||||||||
| Depositing User: | Dr. Karim jebari | |||||||||
| Date Deposited: | 06 Dec 2019 06:48 | |||||||||
| Last Modified: | 06 Dec 2019 06:48 | |||||||||
| Item ID: | 16683 | |||||||||
| Subjects: | Specific Sciences > Cognitive Science > Action Specific Sciences > Artificial Intelligence > AI and Ethics Specific Sciences > Cognitive Science > Concepts and Representations Specific Sciences > Artificial Intelligence > Machine Learning |
|||||||||
| Date: | December 2019 | |||||||||
| URI: | https://philsci-archive.pitt.edu/id/eprint/16683 |
Monthly Views for the past 3 years
Monthly Downloads for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](http://philsci-archive.pitt.edu/style/images/feed-icon-32x32.png){kind=icon}