Auto Explanation

pnpxai.core.experiment.auto_explanation

METRICS_BASELINE_FN_REQUIRED = PIXEL_FLIPPING_METRICS module-attribute

METRICS_CHANNEL_DIM_REQUIRED = PIXEL_FLIPPING_METRICS module-attribute

DEFAULT_METRICS_FOR_TEXT = [MoRF, LeRF, AbPC] module-attribute

DEFAULT_METRICS = [MuFidelity, AbPC, Sensitivity, Complexity] module-attribute

AutoExplanation

Bases: Experiment

An extension of Experiment class with automatic explainers and parameters recommendation.

Parameters:

Name	Type	Description	Default
model	Model	The machine learning model to be analyzed.	required
data	DataSource	The data source used for the experiment.	required
modality	Modality	An object to specify modality-specific workflow.	required
input_extractor	Optional[Callable]	Custom function to extract input features.	None
label_extractor	Optional[Callable]	Custom function to extract labels features.	None
target_extractor	Optional[Callable]	Custom function to extract target features.	None
input_visualizer	Optional[Callable]	Custom function for visualizing input features.	required
target_labels	Optional[bool]	Whether to use target labels.	False

Attributes:

Name	Type	Description
recommended	RecommenderOutput	A data object, containing recommended explainers.

recommended = XaiRecommender().recommend(modality=modality, model=model) instance-attribute

modality = modality instance-attribute

model = model instance-attribute

model_device = next(self.model.parameters()).device instance-attribute

manager = ExperimentManager(data=data, cache_device=cache_device) instance-attribute

input_extractor = input_extractor if input_extractor is not None else default_input_extractor instance-attribute

label_extractor = label_extractor if label_extractor is not None else default_target_extractor instance-attribute

target_extractor = target_extractor if target_extractor is not None else default_target_extractor instance-attribute

target_labels = target_labels instance-attribute

errors property

has_explanations property

__init__(model: Model, data: DataSource, modality: Modality, input_extractor: Optional[Callable] = None, label_extractor: Optional[Callable] = None, target_extractor: Optional[Callable] = None, target_labels: bool = False)

subscribe(callback)

fire(event)

reset_errors()

to_device(x)

run_batch(explainer_id: int, postprocessor_id: int, metric_id: int, data_ids: Optional[Sequence[int]] = None) -> dict

Runs the experiment for selected batch of data, explainer, postprocessor and metric.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	None
explainer_id	int	ID of explainer to use for the run.	required
postprocessor_id	int	ID of postprocessor to use for the run.	required
metrics_id	int	ID of metric to use for the run.	required

Returns:

Type	Description
dict	The dictionary of inputs, labels, outputs, targets, explainer, explanation, postprocessor, postprocessed, metric, and evaluation.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then caches the results in the manager, and returns back to the user.

Note: The input parameters allow for flexibility in specifying subset of data, explainer, postprocessor and metric to process.

predict_batch(data_ids: Sequence[int])

Predicts results of the experiment for selected batch of data.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model outputs corresponding to data ids.

This method orchestrates the experiment by configuring the manager and processing data. It then caches the results in the manager, and returns back to the user.

explain_batch(data_ids: Sequence[int], explainer_id: int)

Explains selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

postprocess_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int)

Postprocesses selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required

Returns:

Type	Description
	Batched postprocessed model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

evaluate_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int, metric_id: int)

Evaluates selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required
metric_id	int	A metric ID to evaluate the model explanations.	required

Returns:

Type	Description
	Batched model evaluations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, generating explanations, and evaluating results. It then caches the results in the manager, and returns back to the user.

optimize(data_ids: Union[int, Sequence[int]], explainer_id: int, metric_id: int, direction: Literal['minimize', 'maximize'] = 'maximize', sampler: Literal['grid', 'random', 'tpe'] = 'tpe', n_trials: Optional[int] = None, timeout: Optional[float] = None, **kwargs)

Optimize experiment hyperparameters by processing data, generating explanations, evaluating with metrics, caching and retrieving the data.

Parameters:

Name	Type	Description	Default
data_ids	Union[int, Sequence[int]]	A single data ID or sequence of data IDs to specify the subset of data to process.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
metric_id	int	A metric ID to evaluate optimizer decisions.	required
direction	Literal['minimize', 'maximize']	A string to specify the direction of optimization.	'maximize'
sampler	Literal['grid', 'random', 'tpe']	A string to specify the sampler to use for optimization.	'tpe'
n_trials	Optional[int]	An integer to specify the number of trials for optimization. If none passed, the number of trials is inferred from timeout.	None
timeout	Optional[float]	A float to specify the timeout for optimization. Ignored, if n_trials is specified.	None

Returns:

Type	Description
	The Experiment instance with updated results and state.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then saves the results in the manager.

Note: The input parameters allow for flexibility in specifying subsets of data, explainers, and metrics to process. If not provided, the method processes all available data, explainers, postprocessors, and metrics.

get_inputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten last run input data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened input data.

This method retrieves input data using the input extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_all_inputs_flattened() -> Sequence[Tensor]

Retrieve and flatten all input data.

Returns:

Type	Description
Sequence[Tensor]	Flattened input data from all available data.

This method retrieves input data from all available data points using the input extractor and flattens it.

get_labels_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten labels data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened labels data.

This method retrieves label data using the label extractor and flattens it for further processing.

get_targets_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten target data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened target data.

This method retrieves target data using the target extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_outputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten model outputs.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened model outputs.

This method retrieves flattened model outputs using the manager's get_flat_outputs method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explanations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Tensor]]

Retrieve and flatten explanations from all explainers.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Tensor]]	Flattened explanations from all explainers.

This method retrieves flattened explanations for each explainer using the manager's get_flat_explanations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_evaluations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Sequence[Tensor]]]

Retrieve and flatten evaluations for all explainers and metrics.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Sequence[Tensor]]]	Flattened evaluations for all explainers and metrics.

This method retrieves flattened evaluations for each explainer and metric using the manager's get_flat_evaluations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explainers_ranks() -> Optional[Sequence[Sequence[int]]]

Calculate and return rankings for explainers based on evaluations.

Returns:

Type	Description
Optional[Sequence[Sequence[int]]]	Rankings of explainers. Returns None if rankings cannot be calculated.

This method calculates rankings for explainers based on evaluations and metric scores. It considers metric priorities and sorting preferences to produce rankings.

AutoExplanationForImageClassification

Bases: AutoExplanation

An extension of AutoExplanation class with modality set to the ImageModality.

Parameters:

Name	Type	Description	Default
model	Model	The machine learning model to be analyzed.	required
data	DataSource	The data source used for the experiment.	required
input_extractor	Optional[Callable]	Custom function to extract input features.	None
label_extractor	Optional[Callable]	Custom function to extract labels features.	None
target_extractor	Optional[Callable]	Custom function to extract target features.	None
target_labels	Optional[bool]	Whether to use target labels.	False
channel_dim	int	Channel dimension.	1

Attributes:

Name	Type	Description
modality	ImageModality	An object to specify modality-specific workflow.
recommended	RecommenderOutput	A data object, containing recommended explainers.

model = model instance-attribute

model_device = next(self.model.parameters()).device instance-attribute

manager = ExperimentManager(data=data, cache_device=cache_device) instance-attribute

input_extractor = input_extractor if input_extractor is not None else default_input_extractor instance-attribute

label_extractor = label_extractor if label_extractor is not None else default_target_extractor instance-attribute

target_extractor = target_extractor if target_extractor is not None else default_target_extractor instance-attribute

target_labels = target_labels instance-attribute

modality = modality instance-attribute

errors property

has_explanations property

recommended = XaiRecommender().recommend(modality=modality, model=model) instance-attribute

__init__(model: Module, data: DataLoader, input_extractor: Optional[Callable] = None, label_extractor: Optional[Callable] = None, target_extractor: Optional[Callable] = None, target_labels: bool = False, channel_dim: int = 1)

subscribe(callback)

fire(event)

reset_errors()

to_device(x)

run_batch(explainer_id: int, postprocessor_id: int, metric_id: int, data_ids: Optional[Sequence[int]] = None) -> dict

Runs the experiment for selected batch of data, explainer, postprocessor and metric.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	None
explainer_id	int	ID of explainer to use for the run.	required
postprocessor_id	int	ID of postprocessor to use for the run.	required
metrics_id	int	ID of metric to use for the run.	required

Returns:

Type	Description
dict	The dictionary of inputs, labels, outputs, targets, explainer, explanation, postprocessor, postprocessed, metric, and evaluation.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then caches the results in the manager, and returns back to the user.

Note: The input parameters allow for flexibility in specifying subset of data, explainer, postprocessor and metric to process.

predict_batch(data_ids: Sequence[int])

Predicts results of the experiment for selected batch of data.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model outputs corresponding to data ids.

This method orchestrates the experiment by configuring the manager and processing data. It then caches the results in the manager, and returns back to the user.

explain_batch(data_ids: Sequence[int], explainer_id: int)

Explains selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

postprocess_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int)

Postprocesses selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required

Returns:

Type	Description
	Batched postprocessed model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

evaluate_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int, metric_id: int)

Evaluates selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required
metric_id	int	A metric ID to evaluate the model explanations.	required

Returns:

Type	Description
	Batched model evaluations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, generating explanations, and evaluating results. It then caches the results in the manager, and returns back to the user.

optimize(data_ids: Union[int, Sequence[int]], explainer_id: int, metric_id: int, direction: Literal['minimize', 'maximize'] = 'maximize', sampler: Literal['grid', 'random', 'tpe'] = 'tpe', n_trials: Optional[int] = None, timeout: Optional[float] = None, **kwargs)

Optimize experiment hyperparameters by processing data, generating explanations, evaluating with metrics, caching and retrieving the data.

Parameters:

Name	Type	Description	Default
data_ids	Union[int, Sequence[int]]	A single data ID or sequence of data IDs to specify the subset of data to process.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
metric_id	int	A metric ID to evaluate optimizer decisions.	required
direction	Literal['minimize', 'maximize']	A string to specify the direction of optimization.	'maximize'
sampler	Literal['grid', 'random', 'tpe']	A string to specify the sampler to use for optimization.	'tpe'
n_trials	Optional[int]	An integer to specify the number of trials for optimization. If none passed, the number of trials is inferred from timeout.	None
timeout	Optional[float]	A float to specify the timeout for optimization. Ignored, if n_trials is specified.	None

Returns:

Type	Description
	The Experiment instance with updated results and state.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then saves the results in the manager.

Note: The input parameters allow for flexibility in specifying subsets of data, explainers, and metrics to process. If not provided, the method processes all available data, explainers, postprocessors, and metrics.

get_inputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten last run input data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened input data.

This method retrieves input data using the input extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_all_inputs_flattened() -> Sequence[Tensor]

Retrieve and flatten all input data.

Returns:

Type	Description
Sequence[Tensor]	Flattened input data from all available data.

This method retrieves input data from all available data points using the input extractor and flattens it.

get_labels_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten labels data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened labels data.

This method retrieves label data using the label extractor and flattens it for further processing.

get_targets_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten target data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened target data.

This method retrieves target data using the target extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_outputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten model outputs.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened model outputs.

This method retrieves flattened model outputs using the manager's get_flat_outputs method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explanations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Tensor]]

Retrieve and flatten explanations from all explainers.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Tensor]]	Flattened explanations from all explainers.

This method retrieves flattened explanations for each explainer using the manager's get_flat_explanations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_evaluations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Sequence[Tensor]]]

Retrieve and flatten evaluations for all explainers and metrics.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Sequence[Tensor]]]	Flattened evaluations for all explainers and metrics.

This method retrieves flattened evaluations for each explainer and metric using the manager's get_flat_evaluations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explainers_ranks() -> Optional[Sequence[Sequence[int]]]

Calculate and return rankings for explainers based on evaluations.

Returns:

Type	Description
Optional[Sequence[Sequence[int]]]	Rankings of explainers. Returns None if rankings cannot be calculated.

This method calculates rankings for explainers based on evaluations and metric scores. It considers metric priorities and sorting preferences to produce rankings.

AutoExplanationForTextClassification

Bases: AutoExplanation

An extension of AutoExplanation class with modality set to the TextModality.

Parameters:

Name	Type	Description	Default
model	Model	The machine learning model to be analyzed.	required
data	DataSource	The data source used for the experiment.	required
layer	TargetLayer	A Module or its string representation to select a target layer for analysis.	required
mask_token_id	int	A mask token id.	required
input_extractor	Optional[Callable]	Custom function to extract input features.	None
forward_arg_extractor	Optional[Callable]	Custom function to extract forward arguments.	None
additional_forward_arg_extractor	Optional[Callable]	Custom function to extract additional forward arguments.	None
label_extractor	Optional[Callable]	Custom function to extract labels features.	None
target_extractor	Optional[Callable]	Custom function to extract target features.	None
target_labels	Optional[bool]	Whether to use target labels.	False
channel_dim	int	Channel dimension.	-1

Attributes:

Name	Type	Description
modality	ImageModality	An object to specify modality-specific workflow.
recommended	RecommenderOutput	A data object, containing recommended explainers.

layer = layer instance-attribute

mask_token_id = mask_token_id instance-attribute

forward_arg_extractor = forward_arg_extractor instance-attribute

additional_forward_arg_extractor = additional_forward_arg_extractor instance-attribute

model = model instance-attribute

model_device = next(self.model.parameters()).device instance-attribute

manager = ExperimentManager(data=data, cache_device=cache_device) instance-attribute

input_extractor = input_extractor if input_extractor is not None else default_input_extractor instance-attribute

label_extractor = label_extractor if label_extractor is not None else default_target_extractor instance-attribute

target_extractor = target_extractor if target_extractor is not None else default_target_extractor instance-attribute

target_labels = target_labels instance-attribute

modality = modality instance-attribute

errors property

has_explanations property

recommended = XaiRecommender().recommend(modality=modality, model=model) instance-attribute

__init__(model: Module, data: DataLoader, layer: TargetLayer, mask_token_id: int, input_extractor: Optional[Callable] = None, forward_arg_extractor: Optional[Callable] = None, additional_forward_arg_extractor: Optional[Callable] = None, label_extractor: Optional[Callable] = None, target_extractor: Optional[Callable] = None, target_labels: bool = False, channel_dim: int = -1)

subscribe(callback)

fire(event)

reset_errors()

to_device(x)

run_batch(explainer_id: int, postprocessor_id: int, metric_id: int, data_ids: Optional[Sequence[int]] = None) -> dict

Runs the experiment for selected batch of data, explainer, postprocessor and metric.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	None
explainer_id	int	ID of explainer to use for the run.	required
postprocessor_id	int	ID of postprocessor to use for the run.	required
metrics_id	int	ID of metric to use for the run.	required

Returns:

Type	Description
dict	The dictionary of inputs, labels, outputs, targets, explainer, explanation, postprocessor, postprocessed, metric, and evaluation.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then caches the results in the manager, and returns back to the user.

Note: The input parameters allow for flexibility in specifying subset of data, explainer, postprocessor and metric to process.

predict_batch(data_ids: Sequence[int])

Predicts results of the experiment for selected batch of data.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model outputs corresponding to data ids.

This method orchestrates the experiment by configuring the manager and processing data. It then caches the results in the manager, and returns back to the user.

explain_batch(data_ids: Sequence[int], explainer_id: int)

Explains selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

postprocess_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int)

Postprocesses selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required

Returns:

Type	Description
	Batched postprocessed model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

evaluate_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int, metric_id: int)

Evaluates selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required
metric_id	int	A metric ID to evaluate the model explanations.	required

Returns:

Type	Description
	Batched model evaluations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, generating explanations, and evaluating results. It then caches the results in the manager, and returns back to the user.

optimize(data_ids: Union[int, Sequence[int]], explainer_id: int, metric_id: int, direction: Literal['minimize', 'maximize'] = 'maximize', sampler: Literal['grid', 'random', 'tpe'] = 'tpe', n_trials: Optional[int] = None, timeout: Optional[float] = None, **kwargs)

Optimize experiment hyperparameters by processing data, generating explanations, evaluating with metrics, caching and retrieving the data.

Parameters:

Name	Type	Description	Default
data_ids	Union[int, Sequence[int]]	A single data ID or sequence of data IDs to specify the subset of data to process.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
metric_id	int	A metric ID to evaluate optimizer decisions.	required
direction	Literal['minimize', 'maximize']	A string to specify the direction of optimization.	'maximize'
sampler	Literal['grid', 'random', 'tpe']	A string to specify the sampler to use for optimization.	'tpe'
n_trials	Optional[int]	An integer to specify the number of trials for optimization. If none passed, the number of trials is inferred from timeout.	None
timeout	Optional[float]	A float to specify the timeout for optimization. Ignored, if n_trials is specified.	None

Returns:

Type	Description
	The Experiment instance with updated results and state.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then saves the results in the manager.

Note: The input parameters allow for flexibility in specifying subsets of data, explainers, and metrics to process. If not provided, the method processes all available data, explainers, postprocessors, and metrics.

get_inputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten last run input data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened input data.

This method retrieves input data using the input extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_all_inputs_flattened() -> Sequence[Tensor]

Retrieve and flatten all input data.

Returns:

Type	Description
Sequence[Tensor]	Flattened input data from all available data.

This method retrieves input data from all available data points using the input extractor and flattens it.

get_labels_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten labels data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened labels data.

This method retrieves label data using the label extractor and flattens it for further processing.

get_targets_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten target data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened target data.

This method retrieves target data using the target extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_outputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten model outputs.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened model outputs.

This method retrieves flattened model outputs using the manager's get_flat_outputs method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explanations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Tensor]]

Retrieve and flatten explanations from all explainers.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Tensor]]	Flattened explanations from all explainers.

This method retrieves flattened explanations for each explainer using the manager's get_flat_explanations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_evaluations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Sequence[Tensor]]]

Retrieve and flatten evaluations for all explainers and metrics.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Sequence[Tensor]]]	Flattened evaluations for all explainers and metrics.

This method retrieves flattened evaluations for each explainer and metric using the manager's get_flat_evaluations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explainers_ranks() -> Optional[Sequence[Sequence[int]]]

Calculate and return rankings for explainers based on evaluations.

Returns:

Type	Description
Optional[Sequence[Sequence[int]]]	Rankings of explainers. Returns None if rankings cannot be calculated.

This method calculates rankings for explainers based on evaluations and metric scores. It considers metric priorities and sorting preferences to produce rankings.

AutoExplanationForVisualQuestionAnswering

Bases: AutoExplanation

An extension of AutoExplanation class with multiple modalities, namely ImageModality and TextModality.

Parameters:

Name	Type	Description	Default
model	Model	The machine learning model to be analyzed.	required
data	DataSource	The data source used for the experiment.	required
layer	TargetLayer	A Module or its string representation to select a target layer for analysis.	required
mask_token_id	int	A mask token id.	required
modality	Modality	An object to specify modality-specific workflow.	required
input_extractor	Optional[Callable]	Custom function to extract input features.	None
forward_arg_extractor	Optional[Callable]	Custom function to extract forward arguments.	None
additional_forward_arg_extractor	Optional[Callable]	Custom function to extract additional forward arguments.	None
label_extractor	Optional[Callable]	Custom function to extract labels features.	None
target_extractor	Optional[Callable]	Custom function to extract target features.	None
target_labels	Optional[bool]	Whether to use target labels.	False
channel_dim	Tuple[int]	Channel dimension. Requires a tuple channel dimensions for image and text modalities.	(1, -1)

Attributes:

Name	Type	Description
modality	Tuple[ImageModality, TextModality]	A tuple of objects to specify modality-specific workflow.
recommended	RecommenderOutput	A data object, containing recommended explainers.

layer = layer instance-attribute

mask_token_id = mask_token_id instance-attribute

forward_arg_extractor = forward_arg_extractor instance-attribute

additional_forward_arg_extractor = additional_forward_arg_extractor instance-attribute

model = model instance-attribute

model_device = next(self.model.parameters()).device instance-attribute

manager = ExperimentManager(data=data, cache_device=cache_device) instance-attribute

input_extractor = input_extractor if input_extractor is not None else default_input_extractor instance-attribute

label_extractor = label_extractor if label_extractor is not None else default_target_extractor instance-attribute

target_extractor = target_extractor if target_extractor is not None else default_target_extractor instance-attribute

target_labels = target_labels instance-attribute

modality = modality instance-attribute

errors property

has_explanations property

recommended = XaiRecommender().recommend(modality=modality, model=model) instance-attribute

__init__(model: Module, data: DataLoader, layer: List[TargetLayer], mask_token_id: int, input_extractor: Optional[Callable] = None, forward_arg_extractor: Optional[Callable] = None, additional_forward_arg_extractor: Optional[Callable] = None, label_extractor: Optional[Callable] = None, target_extractor: Optional[Callable] = None, target_labels: bool = False, channel_dim: Tuple[int] = (1, -1))

subscribe(callback)

fire(event)

reset_errors()

to_device(x)

run_batch(explainer_id: int, postprocessor_id: int, metric_id: int, data_ids: Optional[Sequence[int]] = None) -> dict

Runs the experiment for selected batch of data, explainer, postprocessor and metric.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	None
explainer_id	int	ID of explainer to use for the run.	required
postprocessor_id	int	ID of postprocessor to use for the run.	required
metrics_id	int	ID of metric to use for the run.	required

Returns:

Type	Description
dict	The dictionary of inputs, labels, outputs, targets, explainer, explanation, postprocessor, postprocessed, metric, and evaluation.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then caches the results in the manager, and returns back to the user.

Note: The input parameters allow for flexibility in specifying subset of data, explainer, postprocessor and metric to process.

predict_batch(data_ids: Sequence[int])

Predicts results of the experiment for selected batch of data.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model outputs corresponding to data ids.

This method orchestrates the experiment by configuring the manager and processing data. It then caches the results in the manager, and returns back to the user.

explain_batch(data_ids: Sequence[int], explainer_id: int)

Explains selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

postprocess_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int)

Postprocesses selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required

Returns:

Type	Description
	Batched postprocessed model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

evaluate_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int, metric_id: int)

Evaluates selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required
metric_id	int	A metric ID to evaluate the model explanations.	required

Returns:

Type	Description
	Batched model evaluations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, generating explanations, and evaluating results. It then caches the results in the manager, and returns back to the user.

optimize(data_ids: Union[int, Sequence[int]], explainer_id: int, metric_id: int, direction: Literal['minimize', 'maximize'] = 'maximize', sampler: Literal['grid', 'random', 'tpe'] = 'tpe', n_trials: Optional[int] = None, timeout: Optional[float] = None, **kwargs)

Optimize experiment hyperparameters by processing data, generating explanations, evaluating with metrics, caching and retrieving the data.

Parameters:

Name	Type	Description	Default
data_ids	Union[int, Sequence[int]]	A single data ID or sequence of data IDs to specify the subset of data to process.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
metric_id	int	A metric ID to evaluate optimizer decisions.	required
direction	Literal['minimize', 'maximize']	A string to specify the direction of optimization.	'maximize'
sampler	Literal['grid', 'random', 'tpe']	A string to specify the sampler to use for optimization.	'tpe'
n_trials	Optional[int]	An integer to specify the number of trials for optimization. If none passed, the number of trials is inferred from timeout.	None
timeout	Optional[float]	A float to specify the timeout for optimization. Ignored, if n_trials is specified.	None

Returns:

Type	Description
	The Experiment instance with updated results and state.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then saves the results in the manager.

Note: The input parameters allow for flexibility in specifying subsets of data, explainers, and metrics to process. If not provided, the method processes all available data, explainers, postprocessors, and metrics.

get_inputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten last run input data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened input data.

This method retrieves input data using the input extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_all_inputs_flattened() -> Sequence[Tensor]

Retrieve and flatten all input data.

Returns:

Type	Description
Sequence[Tensor]	Flattened input data from all available data.

This method retrieves input data from all available data points using the input extractor and flattens it.

get_labels_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten labels data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened labels data.

This method retrieves label data using the label extractor and flattens it for further processing.

get_targets_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten target data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened target data.

This method retrieves target data using the target extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_outputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten model outputs.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened model outputs.

This method retrieves flattened model outputs using the manager's get_flat_outputs method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explanations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Tensor]]

Retrieve and flatten explanations from all explainers.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Tensor]]	Flattened explanations from all explainers.

This method retrieves flattened explanations for each explainer using the manager's get_flat_explanations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_evaluations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Sequence[Tensor]]]

Retrieve and flatten evaluations for all explainers and metrics.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Sequence[Tensor]]]	Flattened evaluations for all explainers and metrics.

This method retrieves flattened evaluations for each explainer and metric using the manager's get_flat_evaluations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explainers_ranks() -> Optional[Sequence[Sequence[int]]]

Calculate and return rankings for explainers based on evaluations.

Returns:

Type	Description
Optional[Sequence[Sequence[int]]]	Rankings of explainers. Returns None if rankings cannot be calculated.

This method calculates rankings for explainers based on evaluations and metric scores. It considers metric priorities and sorting preferences to produce rankings.

AutoExplanationForTSClassification

Bases: AutoExplanation

An extension of AutoExplanation class with modality set to the TimeSeriesModality.

Parameters:

Name	Type	Description	Default
model	Model	The machine learning model to be analyzed.	required
data	DataSource	The data source used for the experiment.	required
input_extractor	Optional[Callable]	Custom function to extract input features.	None
label_extractor	Optional[Callable]	Custom function to extract labels features.	None
target_extractor	Optional[Callable]	Custom function to extract target features.	None
target_labels	Optional[bool]	Whether to use target labels.	False
sequence_dim	Tuple[int]	Sequence dimension.	-1
mask_agg_dim	Tuple[int]	A dimension for aggregating mask values. Usually, a channel dimension.	-2

Attributes:

Name	Type	Description
modality	TimeSeriesModality	An object to specify modality-specific workflow.
recommended	RecommenderOutput	A data object, containing recommended explainers.

mask_agg_dim = mask_agg_dim instance-attribute

model = model instance-attribute

model_device = next(self.model.parameters()).device instance-attribute

manager = ExperimentManager(data=data, cache_device=cache_device) instance-attribute

input_extractor = input_extractor if input_extractor is not None else default_input_extractor instance-attribute

label_extractor = label_extractor if label_extractor is not None else default_target_extractor instance-attribute

target_extractor = target_extractor if target_extractor is not None else default_target_extractor instance-attribute

target_labels = target_labels instance-attribute

modality = modality instance-attribute

errors property

has_explanations property

recommended = XaiRecommender().recommend(modality=modality, model=model) instance-attribute

__init__(model: Module, data: DataLoader, input_extractor: Optional[Callable] = None, label_extractor: Optional[Callable] = None, target_extractor: Optional[Callable] = None, target_labels: bool = False, sequence_dim: int = -1, mask_agg_dim: int = -2)

subscribe(callback)

fire(event)

reset_errors()

to_device(x)

run_batch(explainer_id: int, postprocessor_id: int, metric_id: int, data_ids: Optional[Sequence[int]] = None) -> dict

Runs the experiment for selected batch of data, explainer, postprocessor and metric.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	None
explainer_id	int	ID of explainer to use for the run.	required
postprocessor_id	int	ID of postprocessor to use for the run.	required
metrics_id	int	ID of metric to use for the run.	required

Returns:

Type	Description
dict	The dictionary of inputs, labels, outputs, targets, explainer, explanation, postprocessor, postprocessed, metric, and evaluation.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then caches the results in the manager, and returns back to the user.

Note: The input parameters allow for flexibility in specifying subset of data, explainer, postprocessor and metric to process.

predict_batch(data_ids: Sequence[int])

Predicts results of the experiment for selected batch of data.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model outputs corresponding to data ids.

This method orchestrates the experiment by configuring the manager and processing data. It then caches the results in the manager, and returns back to the user.

explain_batch(data_ids: Sequence[int], explainer_id: int)

Explains selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to process.	required

Returns:

Type	Description
	Batched model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

postprocess_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int)

Postprocesses selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required

Returns:

Type	Description
	Batched postprocessed model explanations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, and generating explanations. It then caches the results in the manager, and returns back to the user.

evaluate_batch(data_ids: List[int], explainer_id: int, postprocessor_id: int, metric_id: int)

Evaluates selected batch of data within experiment.

Parameters:

Name	Type	Description	Default
data_ids	Sequence[int]	A sequence of data IDs to specify the subset of data to postprocess.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
postprocessor_id	int	A postprocessor ID to specify the postprocessor to use.	required
metric_id	int	A metric ID to evaluate the model explanations.	required

Returns:

Type	Description
	Batched model evaluations corresponding to data ids.

This method orchestrates the experiment by configuring the manager, obtaining explainer instance, processing data, generating explanations, and evaluating results. It then caches the results in the manager, and returns back to the user.

optimize(data_ids: Union[int, Sequence[int]], explainer_id: int, metric_id: int, direction: Literal['minimize', 'maximize'] = 'maximize', sampler: Literal['grid', 'random', 'tpe'] = 'tpe', n_trials: Optional[int] = None, timeout: Optional[float] = None, **kwargs)

Optimize experiment hyperparameters by processing data, generating explanations, evaluating with metrics, caching and retrieving the data.

Parameters:

Name	Type	Description	Default
data_ids	Union[int, Sequence[int]]	A single data ID or sequence of data IDs to specify the subset of data to process.	required
explainer_id	int	An explainer ID to specify the explainer to use.	required
metric_id	int	A metric ID to evaluate optimizer decisions.	required
direction	Literal['minimize', 'maximize']	A string to specify the direction of optimization.	'maximize'
sampler	Literal['grid', 'random', 'tpe']	A string to specify the sampler to use for optimization.	'tpe'
n_trials	Optional[int]	An integer to specify the number of trials for optimization. If none passed, the number of trials is inferred from timeout.	None
timeout	Optional[float]	A float to specify the timeout for optimization. Ignored, if n_trials is specified.	None

Returns:

Type	Description
	The Experiment instance with updated results and state.

This method orchestrates the experiment by configuring the manager, obtaining explainer and metric instances, processing data, generating explanations, and evaluating metrics. It then saves the results in the manager.

Note: The input parameters allow for flexibility in specifying subsets of data, explainers, and metrics to process. If not provided, the method processes all available data, explainers, postprocessors, and metrics.

get_inputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten last run input data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened input data.

This method retrieves input data using the input extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_all_inputs_flattened() -> Sequence[Tensor]

Retrieve and flatten all input data.

Returns:

Type	Description
Sequence[Tensor]	Flattened input data from all available data.

This method retrieves input data from all available data points using the input extractor and flattens it.

get_labels_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten labels data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened labels data.

This method retrieves label data using the label extractor and flattens it for further processing.

get_targets_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten target data.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened target data.

This method retrieves target data using the target extractor and flattens it for further processing.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_outputs_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Tensor]

Retrieve and flatten model outputs.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Tensor]	Flattened model outputs.

This method retrieves flattened model outputs using the manager's get_flat_outputs method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explanations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Tensor]]

Retrieve and flatten explanations from all explainers.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Tensor]]	Flattened explanations from all explainers.

This method retrieves flattened explanations for each explainer using the manager's get_flat_explanations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_evaluations_flattened(data_ids: Optional[Sequence[int]] = None) -> Sequence[Sequence[Sequence[Tensor]]]

Retrieve and flatten evaluations for all explainers and metrics.

Parameters:

Name	Type	Description	Default
data_ids	Optional[Sequence[int]]	A sequence of data IDs to specify the subset of data to process.	None

Returns:

Type	Description
Sequence[Sequence[Sequence[Tensor]]]	Flattened evaluations for all explainers and metrics.

This method retrieves flattened evaluations for each explainer and metric using the manager's get_flat_evaluations method.

Note: The input parameters allow for flexibility in specifying subsets of data to process. If not provided, the method processes all available data.

get_explainers_ranks() -> Optional[Sequence[Sequence[int]]]

Calculate and return rankings for explainers based on evaluations.

Returns:

Type	Description
Optional[Sequence[Sequence[int]]]	Rankings of explainers. Returns None if rankings cannot be calculated.

This method calculates rankings for explainers based on evaluations and metric scores. It considers metric priorities and sorting preferences to produce rankings.