Incompleteness of moral choice and evolution towards fully autonomous AI

The increasing integration of artificial intelligence (AI) into various aspects of life raises significant ethical questions, particularly concerning autonomous decision-making. The prevailing approach in AI development often assumes that ethical decision-making can be reduced to algorithmic processes, independent of the physical substrate or the nature of the decision-maker. This article challenges this assumption, arguing that if a task inherently involves moral authority when performed by humans, its automation requires the transfer of that moral competence to the AI system. This necessitates an empirical investigation into moral competence and a reevaluation of purely normative approaches in AI ethics. The central thesis is that the development of fully autonomous AI platforms, especially in areas like autonomous weapons systems (AWS), demands a deeper understanding of human moral decision-making to ensure responsible and ethical implementation. The complexity of moral dilemmas, influenced by factors like incomplete information and human biases, must be considered when designing AI systems capable of making ethically sound judgments, as the consequences of failures in this domain can be catastrophic.

The author reviews existing literature on artificial moral agents (AMAs), highlighting the functionalist perspective that views ethics as a programmable function realizable in any physical system. This perspective is contrasted with concerns raised by various researchers and organizations such as Human Rights Watch regarding the ethical challenges posed by increasingly autonomous AI platforms, particularly AWS. The literature emphasizes the autonomy of weapon systems, distinguishing between human-in-the-loop, human-on-the-loop, and human-out-of-the-loop systems. The article references existing research that points to the limitations of simply claiming a weapon system is 'autonomous' if it's merely capable of independent action within human-defined parameters. The existing literature also highlights concerns regarding the unpredictable nature of increasingly intelligent robots and the ethical implications arising from their capacity for independent and potentially unforeseen actions. The author notes the discussion in the literature on the conceptual and methodological challenges in applying existing ethical frameworks to AI, especially regarding the issues of accountability, human dignity and the fundamental differences between humans and machines regarding consciousness and free will.

The author employs a philosophical analysis of existing arguments in the debate surrounding autonomous weapons systems (AWS) and AI ethics. This involves critically examining the assumptions underlying common arguments against human-out-of-the-loop systems. The methodology involves identifying and dissecting three main argumentation strategies often used against AWS: 1) the undermining of human dignity argument; 2) the problem of accountability; and 3) the fundamental ontological difference between humans and machines. The author challenges these strategies by highlighting their implicit reliance on specific conceptions of ethics and what is expected from AI ethics. The methodology then shifts to address the conceptual and methodological problems within current AI ethics, particularly the reliance on established philosophical theories (Kantian deontology, utilitarianism, virtue ethics). The author argues that these theories, while providing valuable frameworks, may not be readily translatable into algorithmic form and that a focus on the ‘what actually’ is needed. The author introduces the concept of 'Kant's moral ambulance,' representing the use of abstract moral principles in emergency situations, contrasting it with a more naturalistic approach that emphasizes empirical research into actual moral decision-making processes. The methodology incorporates insights from various disciplines to create a more comprehensive understanding of the challenges. This includes drawing upon research in cognitive science, neuroscience, and evolutionary psychology to explore the nature of human moral decision-making, revealing the significant role of unconscious processes and emotions. The research suggests that moral principles are not inherently objective or subjective but rather emerge from our interactions with the environment, analogous to scientific theories.

The author identifies three common arguments against human-out-of-the-loop weapon systems (AWS): the undermining of human dignity, the problem of accountability, and the fundamental ontological difference between humans and machines. These arguments are critically examined, revealing underlying assumptions about ethics and the capabilities of AI. The paper argues against the simplistic view that complex algorithms automatically equate to ethical behavior. The analysis demonstrates that existing ethical theories, such as Kantian deontology and utilitarianism, may not easily translate into algorithmic form for AI systems, and suggests that a focus on the practical application of ethical principles is necessary. The author introduces a naturalistic approach to AI metaethics, proposing that moral choices should be viewed as empirical problems with empirical solutions. The author presents the concept of 'Kant's moral ambulance' to illustrate the limitations of applying abstract moral principles to all situations. The work highlights the role of unconscious processes, emotions, and evolutionary adaptations in human moral decision-making, as evidenced by research in cognitive science and neuroscience. This empirical perspective suggests that moral judgments are not simply reflections of objective truths but are shaped by the context and the individual's interaction with their environment. The author argues against the simplistic dichotomy of 'subjective' versus 'objective' values, suggesting that the degree of controversy about a value is not an indicator of its metaphysical status. The paper concludes that the field of AI ethics needs more targeted, empirically grounded approaches, moving beyond overly general and abstract ethical principles.

The findings challenge the prevalent assumption within AI ethics that complex algorithms automatically translate to ethical behavior. This challenges the simplistic view that ethical decision-making is simply a matter of creating complex enough programs. The paper’s emphasis on a naturalistic approach to AI metaethics highlights the importance of empirical research in understanding human moral decision-making processes and applying these insights to AI development. The author argues that a better understanding of the factors influencing human moral choices, including unconscious processes and emotional responses, is crucial for designing AI systems that behave ethically. This necessitates interdisciplinary collaboration, integrating insights from neuroscience, cognitive psychology, and other fields. The limitations of abstract moral principles, as exemplified by 'Kant's moral ambulance', are discussed, emphasizing the need for context-specific approaches to ethical dilemmas. The findings are relevant to the broader field of AI ethics, offering a critique of current approaches and proposing a more empirically grounded and interdisciplinary framework for future research.

The paper concludes that current AI ethics is hampered by a reliance on overly abstract and general ethical principles that do not readily translate into practical guidelines for AI systems. The author advocates for a shift toward a naturalistic metaethics, grounding AI ethical considerations in empirical research into human moral decision-making. This requires a move away from the simplistic equation of complex algorithms with ethical behavior and necessitates a multidisciplinary approach, incorporating insights from neuroscience, cognitive science, and other relevant fields. Future research should focus on developing context-specific, empirically informed approaches to ethical challenges in AI, moving beyond abstract principles toward practical and adaptable solutions.

The paper focuses primarily on the philosophical analysis of arguments surrounding autonomous weapons systems and AI ethics, limiting the scope to a specific area of AI application. While incorporating insights from cognitive science and neuroscience, the paper does not conduct its own empirical research, relying instead on existing findings in these fields. The exploration of various philosophical ethical theories is selective, focusing primarily on Kantianism and utilitarianism, potentially overlooking alternative perspectives that might offer different insights. The paper also does not delve into the technical aspects of AI development, focusing largely on the ethical and philosophical considerations.