Skip to content

Evaluator [source]

The Evaluator module provides several metrics to evaluate the result explainations generated by explainers. Based on Co-12 properties [1], correctness, continuity, compactness, and completeness are chosen. Currently one metrics for each properties are implemented except completeness.

Properties of evaluation

Property [1] Explanation Corresponding Metrics Reference
Correctness Evaluates the truth/reliability of the explanation of the prediction model (AI model). In other words, it indicates how truthful the explanation is compared to the behavior of the black box model. Moreover, it evaluates the degree to which a prediction model (AI model) is explained. MuFidelity, Area between Perturbation Curves [2], [4]
Continuity Continuity assesses how continuous (i.e. smooth) the description is. High-continuity explanation functions ensure that small changes in the input do not lead to large changes in the explanation. Sensitivity [3]
Compactness Evaluates the size/amount of explanation. Ensure that you do not present complex and redundant explanations that are difficult to understand. Complexity [2]

Usage

import torch
from torch.utils.data import DataLoader

from pnpxai.utils import set_seed
from pnpxai.explainers import LRP, ExplainerWArgs
from pnpxai.evaluator import Complexity

from helpers import get_imagenet_dataset, get_torchvision_model

set_seed(seed=0)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# load model, dataset, and explainer
model, transform = get_torchvision_model("resnet18")
model = model.to(device)
explainer = ExplainerWArgs(
    explainer=LRP(model=model),
    kwargs={"epsilon": 1e-6, "n_classes": 1000},
)

dataset = get_imagenet_dataset(transform=transform, subset_size=8)
loader = DataLoader(dataset, batch_size=8)
inputs, targets = next(iter(loader))
inputs, targets = inputs.to(device), targets.to(device)

# make explanation
attrs = explainer.attribute(inputs, targets)

# test evaluator
metric = Complexity()
evaluations = metric(attributions=attrs)
print(evaluations)

Reference

[1] M. Nauta, J. Trienes, S. Pathak, E. Nguyen, M. Peters, Y. Schmitt, J. Schlötterer, M. V. Keulen, C. Seifert. From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI. ACM Comput. Surv. 55(13s): 295:1-295:42 (2023).

[2] U. Bhatt, A. Weller, and J. M. F. Moura. Evaluating and aggregating feature-based model explanations. In Proceedings of the IJCAI (2020).

[3] C.-K. Yeh, C.-Y. Hsieh, A.S. Suggala, D.I. Inouye, and P. Ravikumar. On the (in)fidelity and sensitivity of explanations. In Proceedings of the NeurIPS (2019).

[4] X. Han, Z. Jiang, H. Jin, Z. Liu, N. Zou, Q. Wang, and X. Hu, Retiring $\Delta $ DP: New Distribution-Level Metrics for Demographic Parity. arXiv preprint arXiv:2301.13443 (2023).