The ironies of autonomy

The article interrogates how autonomous vehicles (AVs) reshape human subjectivity and human–machine relations when automation is implemented via big data and machine learning. Using recent AV crashes as context, it highlights recurring issues of faulty handover between human and machine and failures in computer vision that reveal an ontological gap between the world and its algorithmic representation. The author sets two core aims: (1) to show that automation displaces rather than replaces humans, distributing their roles across micro-work that supports AI systems and the vigilant oversight expected of AV operators; and (2) to examine how, because AVs are not only cars but data infrastructures, humans occupy unpredictable subject positions (operator, data-labeled subject, micro-worker) shaped by statistical systems and surveillance. The paper frames these questions within postphenomenology, focusing on embodiment and instrumentation, and critiques legal-accountability assumptions (e.g., the “human in the loop”) inherited from 20th century safety engineering as inadequate for distributed, data-driven AV systems.

The paper surveys multiple literatures that construct AV autonomy: (a) policy and transport studies emphasizing safety, regulation, and efficiency; (b) human factors and automation research from aviation and industrial contexts (e.g., Fitts List/MABA-MABA, Rasmussen’s SRK model, and ‘human in the loop’), which underpin SAE J3016 levels yet risk misleading linear notions of ‘full’ autonomy; (c) critiques of metrics and metaphors—autonowashing, anthropomorphism, and the ‘brain’ trope—that shape expectations; (d) media and cultural studies on automobility and embodiment; (e) scholarship on algorithmic bias and computer vision inequities (e.g., misidentification of darker skin tones and female phenotypes) and the ‘objectifying force of statistics’; (f) platform/gig work and heteromation studies on human labor supporting AI; and (g) surveillance/measurement literatures on affective computing and data-driven governance. These streams collectively show autonomy as an effect of discourses, standards, and infrastructures rather than an inherent machine attribute.

Qualitative, interpretive study combining desk research with empirical fieldwork (2016–2019): 20 unstructured, in-depth interviews with academic, policy, and industry experts in law, computer science, design, mapping, robotics, and automotive engineering (Germany, India, USA); two interviews with Tesla owners in North America and one Tesla test drive; two workshops on ethics and future technologies with engineers/technologists in Germany and the USA; and a two-year professional association with a futurologist at a German automaker. Excerpts from these interactions are woven into the argument to illuminate practices of autonomy, embodiment, accountability, and labor in AV contexts. Data from interviews are not publicly shared due to participant privacy.

• Automation displaces rather than replaces humans: people become distributed across roles such as micro-workers annotating images for computer vision (via specialized platforms and reCAPTCHAs) and drivers tasked with vigilant oversight during Auto-pilot.
• Ironies of autonomy: despite claims of machine superiority, humans remain responsible and are placed in ‘moral crumple zones’ when automated systems fail. Legal and accountability frameworks still presume a controllable human-in-the-loop even as ML-driven processes resist meaningful human intervention.
• Empirical crash cases reveal recurrent failure points: problematic handovers during Auto-pilot and computer vision misrecognitions (e.g., Florida Tesla incident mistaking a white truck for sky; Arizona Uber crash misidentifying a pedestrian). NTSB data show the Arizona safety driver looked at a phone 34% of the time, glanced at it 23 times in the three minutes before impact, and looked up one second before collision.
• Computer vision inequity: object detection performs worse for darker skin tones (Fitzpatrick 4–6), implying uneven risk distribution for pedestrians across demographic groups.
• Embodiment and discipline: AV operation reshapes driver bodies and attention through constant engagement/disengagement beeps, hands-on-wheel requirements, and instruction to ‘let Auto-pilot do its job,’ producing new hybrid human–machine subjectivities.
• Surveillance and affective computing: driver-facing cameras and analytics monitor attention and affect (e.g., fatigue, road rage), disciplining operators and potentially shifting liability while raising concerns about validity of emotion inference.
• Metrics and standards (e.g., SAE levels, disengagement reports) actively produce realities of ‘autonomy’ but can be misleading, obscuring context and normalizing linear progress narratives.
• Heteromation’s economic logic: low-paid, global micro-workers are integral to AV perception pipelines; humans function as ‘computational components,’ generating value for AI companies while remaining marginal and minimally protected.

Findings demonstrate that AV ‘autonomy’ is not a machine property but an emergent effect of socio-technical relations, standards, data infrastructures, and human labor. The assumed dyad of driver and vehicle (e.g., human-in-the-loop) fails to capture distributed human involvement and limited avenues for timely intervention in ML-driven perception and decision pipelines. This misfit produces accountability gaps wherein humans are surveilled, disciplined, and blamed despite diminished control, while biases in datasets and algorithms unevenly distribute risk (e.g., to darker-skinned pedestrians). Embodiment processes show operators learning to ‘team’ with automation under ambiguous control, aligning with postphenomenological insights that technologies mediate and co-constitute subjectivity. Consequently, policy and legal regimes must move from loop metaphors to ‘policy knots’ tying together design, implementation, datasets, labor practices, and local contexts to address safety, equity, and responsibility in AV deployments.

The paper contributes a cultural-philosophical and empirical account of AV autonomy as a relational, infrastructural phenomenon that entangles humans through embodiment, heteromation, surveillance, and liability. It cautions against linear automation narratives (e.g., SAE levels) and calls for rethinking accountability beyond the human-in-the-loop toward contextually grounded ‘policy knots’ spanning design, testing, data practices, and governance. Future directions include: creating new social protections and insurance for safety drivers and future AV users; enforcing accountability for computer vision development, benchmarking, and deployment to mitigate demographic harms; recognizing AVs as commercial data platforms subject to labor, data protection, and AI governance; ensuring community protections where testing occurs; and aligning legal frameworks with distributed agency and limited human intervention capacity in ML systems.

• Scope: The article is inspired by, but does not draw on or contribute to, broader gig/platform labor literatures in depth, focusing instead on AV-specific heteromation and embodiment.
• Data access: Interview datasets are not publicly available due to interviewee privacy.
• Methodological: Qualitative, interpretive approach with a limited number of interviews and illustrative field excerpts; not designed for statistical generalization.