Evaluating explainability for graph neural networks

As explanations are increasingly used to understand the behavior of graph neural networks (GNNs), evaluating the quality and reliability of GNN explanations is crucial. However, assessment is challenging because existing graph datasets often lack ground-truth explanations or contain unreliable ones. The authors introduce SHAPEGGEN, a synthetic graph data generator capable of producing diverse benchmark datasets (e.g., varying graph sizes, degree distributions, homophilic vs. heterophilic graphs) accompanied by reliable ground-truth explanations. SHAPEGGEN’s flexibility enables mimicking data properties from various real-world domains while avoiding pitfalls such as redundant explanations, weak GNN predictors, and trivial explanations. They incorporate SHAPEGGEN and several real-world datasets into GRAPHXAI, a library that provides datasets with ground-truth explanations, data loaders, processing functions, visualization tools, GNN model implementations, and evaluation metrics to benchmark GNN explainability methods.

Chirag Agarwal, Owen Queen, Himabindu Lakkaraju, Marinka Zitnik