Whose morality? Which rationality? Challenging artificial intelligence as a remedy for the lack of moral enhancement

The paper begins by defining machine ethics as distinct from computer ethics, focusing on ensuring ethically acceptable machine behavior towards humans and other machines. It introduces the crucial questions: "Whose morality?" and "Which rationality?" These questions highlight the challenge of creating AI that not only possesses intelligence but also exhibits moral autonomy. The paper clarifies the distinction between Moor's implicit ethical agents (programmed to behave ethically) and explicit ethical agents (capable of ethical calculation). The more ambitious goal is creating explicit ethical agents, which demands achieving a human level of both cognition and morality. However, this is complicated by the fact that humans possess consciousness, intentionality, and free will, aspects not yet replicable in machines. The paper proposes examining the difficulties in cultivating autonomous moral concern in AI based on self-developing moral reasoning, exploring whether the distinction between strong and weak AI scenarios can be extrapolated to 'moral' AI scenarios (strong: explicit ethical agents; weak: implicit ethical agents).

The paper reviews existing literature on machine ethics, drawing on works by Anderson and Anderson (2007) on ethical intelligent agents, Powers (2006) on Kantian machines, Anderson and Anderson's reinterpretation of act utilitarianism, and Howard and Muntean's (2017) virtue ethical approach to moral machines. It also references works on algorithmic bias, the Moral Lag Problem, and the challenges of formalizing moral reasoning within computational frameworks. The review establishes the existing discourse and identifies key challenges that need to be addressed in the pursuit of creating moral AI.

The paper employs a comparative analysis of three prominent first-order normative theories in machine ethics: Kantianism, act utilitarianism, and virtue ethics. Each theory's implications for building moral machines are explored, focusing on the challenges of computation and the irreducibility of human moral reasoning to purely computational processes. The analysis examines the computational and moral challenges arising from the application of each theory. For Kantian machines, the paper explores the limitations of nonmonotonic logic, the problem of semidecidability, and the lack of moral feelings. For utilitarian machines, the challenges of computing utility, the problem of moral feelings, and the risk of reducing morality to a cost-benefit analysis are highlighted. Finally, the paper analyzes Howard and Muntean's virtue ethical model, focusing on the concept of active moral learning, the use of soft computation, and the limitations of relying solely on behavioral replicability. The paper uses these case studies to exemplify broader challenges in machine ethics.

The analysis reveals significant challenges in building moral machines. The core difficulty lies in the discrepancy between epistemological verification and normative validity in morality as projected into machine ethics. The "ethos of reducing intelligence to a numeric value" is identified as a key problem, raising concerns about the ability of AI to generate persuasive moral arguments using algorithms. The Moral Lag Problem is discussed, highlighting the difficulty of avoiding human biases when building moral machines and questioning the possibility of truly autonomous moral machines. The paper demonstrates that despite the differences between Kantian, utilitarian, and virtue ethical approaches, all three face difficulties in relating computational processes to moral estimation. The issue of semidecidability—the inability to definitively determine whether a given element belongs to a set—is highlighted as a significant constraint, impacting the capacity for moral self-update in AI systems. The role of biases in both the design and application of algorithms is emphasized, leading to the risk of intermingling computational errors with moral mistakes. The paper also explores the challenges of incorporating moral feelings and motivations into AI, and the difficulties of ensuring that maximizing utility does not lead to morally questionable outcomes. The virtue ethics approach, while promising, presents the risk of moral relativism and uncontrolled evolution of AI 'robo-virtues'. Finally, the paper argues that even with methodological limitations, the process of 'reading moral behavior from data' is inherently flawed, influenced by programmer biases, and risks producing morally deficient AI systems.

The findings highlight the limitations of current approaches to building moral AI. The paper argues that building functional AI systems that serve moral purposes does not guarantee the creation of moral machines. The reduction of intelligence to numerical values in computation processes, the influence of human biases, and the irreducibility of human moral reasoning to computation are all identified as major obstacles. The comparison of three distinct ethical frameworks reveals the pervasive challenge of bridging the gap between computational processes and moral estimation. The inherent limitations of algorithmic approaches to moral reasoning call into question the feasibility of both 'strong' and 'weak' moral AI scenarios. The paper suggests that even the most sophisticated AI may only be capable of simulating ethical deliberation, rather than genuinely engaging in moral judgment.

The paper concludes that creating truly moral AI remains a significant challenge. The inherent complexities of human morality, the influence of human biases in AI development, and the limitations of current computational approaches to moral reasoning all contribute to this difficulty. While the pursuit of ethical AI is worthwhile, the paper emphasizes the need for a more nuanced and critical approach, acknowledging the limitations of reducing complex human morality to algorithms and calculations. Future research should focus on addressing the identified limitations, particularly exploring methods to minimize biases and enhance the capacity for AI systems to understand and respond to the complexities of human moral experience.

The paper primarily focuses on a theoretical analysis of three prominent ethical frameworks, limiting the scope of empirical evidence. The discussion of biases is largely conceptual, and further empirical research would be needed to fully explore their influence on the development and deployment of moral AI. The paper's critique of existing approaches to moral AI does not offer concrete solutions but rather highlights the inherent challenges and limitations.