Metacognitive sensitivity: The key to calibrating trust and optimal decision making with AI

This perspective addresses how humans can achieve optimal decision making when collaborating with AI systems, given persistent uncertainty and occasional AI errors. The central hypothesis is that AI-provided reports of metacognitive sensitivity—capturing how well confidence tracks accuracy—are essential both to calibrate human trust in AI and to guide the optimal integration of AI advice into joint (human–AI) decisions. Drawing on perceptual metacognition, the authors argue AI should report both type 1 performance (choices/accuracy) and type 2 information (confidence, metacognitive sensitivity), and propose a framework to test how these reports affect trust calibration and decision outcomes.

The paper synthesizes research on human–AI collaboration and trust, showing that hybrid decisions can outperform either human or AI alone in domains such as medical imaging and radiology, while benefits depend on comparable accuracy between partners. Trust is influenced by AI embodiment, perceived intelligence, transparency, and explanations; however, increases in trust do not always improve accuracy, demonstrating dissociations between trust and performance. Confidence indicators from AI can change user trust even without accuracy gains. Metacognition research provides measurement tools: metacognitive sensitivity (e.g., meta-d') reflects the confidence–accuracy relationship, and metacognitive efficiency (M-ratio = meta-d'/d') contextualizes sensitivity relative to task difficulty. Signal detection theory underpins both type 1 (d', c) and type 2 (meta-d') metrics. Seminal dyadic decision research (Bahrami et al.) shows weighted confidence sharing yields collective benefits when participants have similar sensitivities; metacognitive sensitivity in a dyad predicts the benefit of joint decisions. Emerging work on LLMs reveals overconfidence, variable metacognitive sensitivity across task types, and opportunities to align confidence with accuracy via metacognitive training, synthetic data, and prompt engineering.

• Metacognitive sensitivity (the correspondence between confidence and accuracy) is a critical, missing ingredient in most human–AI collaboration frameworks for calibrating trust and enabling optimal joint decisions. • AI should report: (1) type 1 decisions and long-run accuracy, (2) trial-level confidence, (3) long-run metacognitive sensitivity (e.g., meta-d', M-ratio), and (4) qualitative introspections about decision processes; these reports can be empirically tested for their impact on trust calibration and decision outcomes. • Signal detection theory provides a principled framework: type 1 (d', c) for task performance and type 2 (meta-d') for metacognitive sensitivity; M-ratio facilitates interpretation relative to an optimal benchmark (1.0). • Evidence from dyadic perceptual decision-making (weighted confidence sharing) shows joint decisions improve when partners have similar metacognitive sensitivity; mean dyad sensitivity correlates with collective benefit. • Confidence reports alone can increase trust without improving accuracy; meaningful metacognitive sensitivity is necessary for confidence to aid optimal integration. • Practical proposals include visualizations (e.g., sliding scale) and brief training exemplars to help users interpret metacognitive sensitivity, plus potential external monitoring to mitigate AI confidence biases. • Early LLM studies indicate overconfidence and metacognitive myopia; aligning confidence with accuracy and reporting internal model confidence can improve human–AI collaboration. • Example application: medical imaging for prostate cancer—metacognitive sensitivity would help clinicians decide when to weight AI confidence appropriately and recognize systematic high-confidence errors.

The authors argue that reporting metacognitive sensitivity enables users to discern when and how to trust AI outputs and to weight AI advice optimally in joint decisions. Insights from dyadic decision-making show that confidence sharing improves outcomes only when confidence is meaningfully linked to accuracy, implying AI confidence must exhibit adequate metacognitive sensitivity. Trust calibration and optimal integration are distinct: AI explanations and confidence can raise trust without improving accuracy, so sensitivity metrics (e.g., meta-d', M-ratio) should inform decision weights. Extending metacognition measures beyond discrete lab tasks to continuous, naturalistic domains (e.g., navigation) is necessary. For LLMs, metacognitive training and alignment of confidence with accuracy can bridge calibration gaps, improving collaboration and safety. Combined quantitative (metrics) and qualitative (process introspections) reports may be required for real-world decisions.

The paper proposes that metacognitive sensitivity reporting by AI is central to calibrating human trust and achieving optimal human–AI decision making. It outlines a testable framework for integrating type 1 performance, trial-level confidence, and type 2 sensitivity metrics, supported by dyadic decision-making evidence. Future work should: (1) develop interpretable sensitivity displays and user training; (2) expand metacognition measures to complex, continuous tasks; (3) study how sensitivity communication affects decisions when performance or sensitivity differs across agents; (4) explore external monitoring to mitigate AI confidence biases; and (5) refine LLM metacognitive training (e.g., synthetic data, prompt engineering) to align confidence with accuracy in high-stakes domains such as medicine, finance, and education.

This is a perspective synthesis without new empirical data. Proposed benefits of metacognitive sensitivity reporting remain to be validated experimentally across domains. Current measures (e.g., meta-d') are best established for discrete tasks and may not generalize to continuous, naturalistic decision environments. Trust calibration does not guarantee optimal integration; human factors (biases, anthropomorphism, explanations) can dissociate trust from accuracy. AI confidence may be biased, raising questions about the reliability of self-reported metacognitive sensitivity and the need for external monitoring.