Metacognitive sensitivity: The key to calibrating trust and optimal decision making with AI

This perspective addresses how humans can achieve optimal decision making when collaborating with AI systems under uncertainty. The central thesis is that reports of metacognitive sensitivity—how well confidence judgments track accuracy—are essential both for calibrating human trust in AI and for optimally weighting AI advice in joint human-AI decisions. The authors situate the problem in the context of AI systems that can be highly capable yet still error-prone, noting that trust can be miscalibrated when users lack insight into AI reliability. They propose that AI should provide not only type 1 outputs (choices/accuracy) but also type 2 information (confidence and metacognitive sensitivity) to guide users’ decisions.

The paper reviews evidence across domains showing benefits and limits of human-AI collaboration, with improved outcomes in medicine (e.g., radiology and dermatology) and education, but not uniformly across tasks. Trust is influenced by AI representation, transparency, and explanations; however, explanations and confidence indicators can increase trust even without improving accuracy, demonstrating a dissociation between trust and correctness. In metacognition research, confidence judgments yield metacognitive bias and metacognitive sensitivity, the latter quantifiable via signal detection theoretic measures such as meta-d' and metacognitive efficiency (M-ratio = meta-d'/d'). Prior work on dyadic human decision making (Bahrami et al., Science 2010) shows that sharing confidence enables collective benefits when observers have similar perceptual sensitivities, with weighted confidence sharing outperforming coin-flip or limited feedback models. Additional studies indicate that collective benefits depend on adequate metacognitive sensitivity, and that differences in metacognition and performance levels affect optimal integration. The authors also review challenges in extending metacognition metrics to continuous, naturalistic tasks (e.g., navigation), and survey emerging work on LLM metacognitive behaviors, including overconfidence, varying sensitivity across task types, and methods to align AI confidence with accuracy.

This is a Perspective article; it does not report original empirical data. The authors outline a conceptual and experimental framework for evaluating how AI-reported information influences trust calibration and joint decision making. They propose four classes of AI reports: (1) type 1 decision outcomes and long-run accuracy; (2) trial-level confidence ratings (type 2 reports); (3) summaries of metacognitive sensitivity (e.g., meta-d' and M-ratio); and (4) introspections explaining type 1 and type 2 decisions. Measurement approaches draw on signal detection theory: d' for type 1 performance, c for response bias, meta-d' for metacognitive sensitivity, and M-ratio as performance-adjusted efficiency (optimal benchmark = 1). They suggest paradigms where users receive different combinations of these reports to test effects on trust calibration and joint performance. The article discusses adapting metacognitive assessments to continuous, naturalistic domains and proposes user training and visualization strategies to improve comprehension and use of metacognitive metrics.

- Confidence indicators can increase users’ trust in AI even without improving accuracy, underscoring the need for trust calibration beyond raw confidence. - Metacognitive sensitivity—capturing the confidence-accuracy correspondence via measures like meta-d' and M-ratio—is crucial for knowing when to trust AI and for enabling optimal human-AI decision fusion. - In dyadic decisions, weighted confidence sharing yields collective benefits when observers have similar sensitivities; metacognitive sensitivity within dyads correlates with performance gains, indicating that confidence must be informative (not merely present) to improve joint outcomes. - Reporting metacognitive efficiency (M-ratio) may be especially interpretable to users because it normalizes sensitivity to performance with an optimal value of 1. - There are measurement gaps for continuous type 1 decisions in naturalistic tasks; existing metacognition metrics handle continuous confidence better than continuous primary judgments. - LLMs display overconfidence and variable metacognitive sensitivity depending on task type; aligning model confidence with accuracy via training, synthetic data, and prompt engineering can improve collaboration. Reporting internal model confidence may further facilitate metacognitive alignment with users. - Practical presentation matters: visual scales and brief training/demonstrations may help users understand and use metacognitive sensitivity reports. External monitoring of metacognition may be needed if AI systems exhibit biased confidence.

The article argues that metacognitive sensitivity provides the missing link between AI confidence signals and human trust, enabling users to appropriately weight AI advice in joint decisions. Trust and optimal integration are distinct: humans may trust AI yet still combine information suboptimally. By reporting metacognitive sensitivity and efficiency, AI systems can inform users about when confidence is reliable, supporting rules akin to weighted reliability fusion observed in multisensory and dyadic decision-making research. The authors highlight domain-specific factors—task demands, information exchange, feedback—that modulate outcomes, and emphasize that collective benefits require meaningful metacognitive sensitivity. They discuss implications for healthcare (e.g., medical imaging), navigation, education, and other areas where decisions are complex and continuous, and propose that combining quantitative metacognitive metrics with qualitative introspections (e.g., LLM-based explanations) can improve transparency and appropriate reliance.

The Perspective contends that AI reports of metacognitive sensitivity are central to calibrating human trust and achieving optimal joint decisions. It calls for systematic research varying performance, confidence, metacognitive sensitivity, and decision domains to evaluate when and how these reports improve outcomes. Future work should: (1) develop and validate metacognition measures for continuous, naturalistic tasks; (2) design effective visualizations and user training for interpreting metacognitive metrics; (3) explore external monitoring to ensure trustworthy metacognition reporting; and (4) test training methods (e.g., synthetic data, prompt engineering) that improve AI metacognitive alignment. As AI is increasingly embedded in high-stakes domains, measuring and reporting both type 1 and type 2 sensitivities will be critical to making the best use of AI-provided information.

No original empirical data are presented; conclusions derive from prior literature and conceptual arguments. Current metacognition metrics (e.g., meta-d', M-ratio) are primarily validated in discrete type 1 tasks and are less developed for continuous, naturalistic decisions. Metacognitive measures may be difficult for end-users to interpret without training. AI systems can exhibit biased confidence and may be trained on data with misinformation, complicating metacognitive calibration. Even with accurate metacognitive signals, users may not integrate AI advice optimally, and the generalizability of dyadic models (e.g., weighted confidence sharing) to diverse human-AI settings requires empirical testing.