A Generalized Explanation Framework for Visualization of Deep Learning Model Predictions

Deep learning models in vision are powerful but hard to trust due to their black-box nature. Standard attribution-based saliency maps often suffice for coarse-grained tasks but fail in fine-grained settings (e.g., distinguishing similar bird species), where users ask: why was class A chosen and why not class B? The paper proposes GALORE, a unified framework to generate three complementary visual explanation types: attributive (what supports the prediction), deliberative (why: which regions are ambiguous and between which classes), and counterfactual (why not: what needs to change to obtain an alternative class). The research goal is to create accurate, efficient, and human-intuitive explanations that scale from naive to expert users and support both post-hoc analysis and interactive applications like machine teaching. The work also aims to leverage model self-awareness via confidence scores to reason about difficulty, ambiguity, and discrimination at the regional level and to propose a replicable quantitative evaluation protocol for explanations using datasets with part and attribute annotations.

The paper surveys XAI methods for computer vision, including concept-, example-, transformation-, and language-based explanations, with saliency/attribution maps being most widely used. It reviews post-hoc attribution techniques (gradient-based, Grad-CAM/score-CAM, SHAP, LIME, RISE) and notes GALORE is compatible with any attribution function. It discusses contrastive/counterfactual explanations (adversarial perturbations, generative approaches, and feature-replacement/search), highlighting limitations in realism, data demands, and computational cost. Evaluation strategies include human-in-the-loop and automated proxy tasks (erasure/addition and localization), as well as robustness sanity checks. The role of self-awareness (confidence estimation, OOD detection, open-set recognition) is reviewed, noting typical entropy-based approaches are insufficient for precise regional ambiguity/discrimination needed by deliberative and counterfactual explanations; connections are drawn to realistic classification with difficulty prediction.

GALORE defines all explanations as combinations (products) of attribution maps with respect to class predictions and a confidence score, forming a heatmap that highlights locations where all factors agree (sharp, specific regions). The general heatmap M(x) multiplies: (i) attribution for the predicted class, (ii) attributions for other relevant class predictions, and (iii) attribution of a confidence score. Attributions are computed on chosen feature layers using functions such as vanilla gradients, Integrated Gradients, Grad-CAM, score-CAM, or SHAP (all normalized). - Attributive explanations: standard attribution A(x,y*) using a(h_y*(x)); self-aware variant multiplies by a(s(x)) to emphasize confident, class-specific regions. - Deliberative explanations: produce insecurities as triplets (region mask r, class pair a,b). For top-E probable classes, all pairs form ambiguity candidates. For each pair, compute an ambiguity map I(x,{a,b}) by multiplying attributions to both classes and the attribution to 1−s(x) (difficulty), then threshold to segment r. Multi-class deliberative uses V-tuples of classes, multiplying attributions across all V classes and difficulty. - Counterfactual explanations: define discriminant maps D(x,y*,y_c) that highlight regions informative for y* but not y_c and with high confidence: multiply a(h_y*(x)), the complement of a(h_yc(x)) (max−value), and a(s(x)). A counterfactual explanation pairs D(x,y*,y_c) on the query image with D(x_c,y_c,y*) on a randomly sampled image from the counterfactual class. Multi-class counterfactual extends by contrasting y* against a set C of counter classes, and for each counter image y_v contrasts against C_v. - Explanation strength: define scalar scores as average heat inside the segmented regions to rank insecurities or counterfactuals. - Confidence scores: three variants are supported: max softmax (self-referential), certainty (1−normalized entropy; self-referential), and easiness (1−hardness) predicted by a separate network trained jointly (non-self-referential). - Evaluation protocol: Quantitative proxy tasks using CUB200 (parts labeled as points with attribute distributions) and ADE20K (scene objects as parts with occurrence probabilities). Ground-truth for deliberative: top ambiguous parts across class pairs/tuples. Ground-truth for counterfactual/discriminant: parts with smallest similarity (class-specific). Metrics: precision–recall (points), IoU (masks), and part-IoU (PIoU) for semantic consistency across paired regions; also human studies (MTurk) for deliberative intuitiveness and machine teaching user study for counterfactual utility. Robustness sanity checks include input translations and parameter randomization.

- Self-awareness improves explanations across strategies; the easiness confidence score generally outperforms softmax max and certainty for attributive, deliberative, and counterfactual explanations (larger gains for expert-like counterfactuals where classes are similar). For deliberative explanations, only easiness reliably improves over baseline. - Attribution functions: advanced methods (Integrated Gradients, Grad-CAM, score-CAM, SHAP) slightly outperform plain gradients, but differences are small; no single method dominates consistently. - Architectures: On CUB200, ResNet-50 produces more accurate segments than VGG16 despite similar classification accuracy, suggesting more human-like deliberations; on ADE20K differences are minor due to task difficulty. - Multi-class deliberative explanations can achieve higher precision at a given recall than binary, likely because combining multiple class attributions better isolates shared attributes. - Segment strength correlates strongly and positively with segment quality (precision), validating its use for ranking (significant Pearson correlations reported). - Robustness: Explanations are stable to small input translations (average IoU usually >75% across thresholds). Sanity checks confirm sensitivity to model parameters (pretrained >> random weights), especially for score-CAM and more so in counterfactuals. - State-of-the-art comparison: GALORE outperforms feature-search counterfactuals and CounteRGAN on localization metrics and PIoU while being dramatically faster (roughly 50×–1000× speedups depending on architecture and region size), since it avoids exhaustive matching and generative synthesis. PIoU improves up to ~0.5 with larger region sizes. - Human MTurk (deliberative): Turkers agreed among themselves on algorithm-identified ambiguities for 59.4% of insecurities vs 33.7% for random crops; agreement with the algorithm was 51.9% vs 26.3% for random crops, indicating insecurities are intuitive and align with human perception. - Machine teaching (counterfactual): After training with GALORE counterfactuals, learners achieved 95% accuracy distinguishing two challenging bird species, versus 60% with random highlighted regions and 77% with full-image highlighting, confirming practical utility for human learning.

The study addresses the need for richer, fine-grained explanations by unifying three complementary types within a single, efficient framework leveraging attribution maps and confidence attributions. Deliberative explanations reveal where and between which classes the model is uncertain, providing transparency into decision ambiguity—often aligning with human reasoning. Counterfactual explanations, built from discriminant attributions, efficiently answer why-not queries and support interactive settings like machine teaching. Results show self-awareness (particularly easiness) is crucial for sharpening and specializing explanations. Robustness and sanity checks indicate GALORE produces stable and parameter-sensitive maps. The framework’s efficiency enables real-time or interactive use, overcoming computational barriers of prior counterfactual approaches. Overall, the findings validate that GALORE improves explanation quality, intuitiveness, and practical impact in expert domains requiring fine-grained distinctions.

GALORE introduces a unified, efficient framework for visualization-based explanations—attributive, deliberative, and counterfactual—by composing attribution maps for class predictions and confidence. It proposes new deliberative explanations that expose insecurities and redefines counterfactuals via discriminant attributions, yielding significant computational speedups. A replicable evaluation protocol using part/attribute annotations on CUB200 and ADE20K demonstrates improved accuracy, robustness, and human alignment. Human studies confirm that deliberative insecurities are intuitive and that counterfactuals improve human learning. Future directions include optimizing thresholding strategies for segmentation in diverse object-size scenarios, exploring broader datasets and domains (e.g., medical imaging), improving confidence estimation, and extending to other modalities or multi-task settings.

- Evaluation relies on datasets with part and attribute annotations to construct proxy ground-truth; such annotations may not be available in many domains. - The heatmap composition assumes a form of independence between attribution maps; this is an approximation and may not always hold. - Segmentation thresholds are tuned to equalize region sizes for paired explanations; optimal strategies may vary with object scales and are left for future work. - Confidence scores are critical; self-referential measures can be unreliable for some tasks, and non-self-referential easiness requires an auxiliary predictor. - Visualizations depend on chosen network layers (typically last conv layer), which may limit spatial granularity or miss deeper semantic cues. - Generative counterfactuals are not attempted; while GALORE is efficient, it does not synthesize images and thus cannot illustrate full image transformations. - Results are primarily on CUB200 and ADE20K; generalization to other expert domains is promising but untested in this work.