Optimizer

pnpxai.evaluator.optimizer.objectives

Objective

A class that encapsulates the logic for evaluating a model's performance using a specified explainer, postprocessor, and metric within a given modality. The Objective class is designed to be callable and can be used within an optimization framework like Optuna to evaluate different configurations of explainers and postprocessors.

Parameters:

Name	Type	Description	Default
explainer	Explainer	The explainer used to generate attributions for the model.	required
postprocessor	PostProcessor	The postprocessor applied to the attributions generated by the explainer.	required
metric	Metric	The metric used to evaluate the effectiveness of the postprocessed attributions.	required
modality	Modality	The data modality (e.g., image, text) the model and explainer are operating on.	required
inputs	Optional[TensorOrTupleOfTensors]	The input data to the model. Defaults to None.	None
targets	Optional[Tensor]	The target labels for the input data. Defaults to None.	None

EXPLAINER_KEY = 'explainer' class-attribute instance-attribute

POSTPROCESSOR_KEY = 'postprocessor' class-attribute instance-attribute

explainer = explainer instance-attribute

postprocessor = postprocessor instance-attribute

metric = metric instance-attribute

modality = modality instance-attribute

inputs = inputs instance-attribute

targets = targets instance-attribute

__init__(explainer: Explainer, postprocessor: PostProcessor, metric: Metric, modality: Modality, inputs: Optional[TensorOrTupleOfTensors] = None, targets: Optional[Tensor] = None)

set_inputs(inputs)

Sets the input data for the objective.

Parameters:

Name	Type	Description	Default
inputs	TensorOrTupleOfTensors	The input data to be used by the explainer.	required

Returns:

Name	Type	Description
Objective		The updated Objective instance.

set_targets(targets)

Sets the target labels for the objective.

Parameters:

Name	Type	Description	Default
targets	Tensor	The target labels corresponding to the input data.	required

Returns:

Name	Type	Description
Objective		The updated Objective instance.

set_data(inputs, targets)

Sets both the input data and target labels for the objective.

Parameters:

Name	Type	Description	Default
inputs	TensorOrTupleOfTensors	The input data to be used by the explainer.	required
targets	Tensor	The target labels corresponding to the input data.	required

Returns:

Name	Type	Description
Objective		The updated Objective instance.

__call__(trial: Trial) -> float

Executes the objective function within an optimization trial. This method is responsible for selecting the explainer and postprocessor based on the trial suggestions, performing the explanation and postprocessing, and evaluating the results using the specified metric.

Parameters:

Name	Type	Description	Default
trial	Trial	The trial object from an optimization framework like Optuna.	required

Returns:

Name	Type	Description
float	float	The evaluation result of the metric after applying the explainer and
	float	postprocessor to the model's predictions. Returns nan if the postprocessed
	float	results contain non-countable values like nan or inf.

pnpxai.evaluator.optimizer.suggestor

map_suggest_method(trial: Trial, method_type: Type[Any])

suggest(trial: Trial, obj: Any, modality: Union[Modality, Tuple[Modality]], key: Optional[str] = None, force_params: Optional[Dict[str, Any]] = None)

A utility function that suggests parameters for a given object based on an optimization trial. The function recursively tunes the parameters of the object according to the modality (or modalities) provided.

Parameters:

Name	Type	Description	Default
trial	Trial	The trial object from an optimization framework like Optuna, used to suggest parameters for tuning.	required
obj	Any	The object whose parameters are being tuned. This object must implement get_tunables() and set_kwargs() methods.	required
modality	Union[Modality, Tuple[Modality]]	The modality (e.g., image, text) or tuple of modalities the object is operating on. If multiple modalities are provided, the function handles multi-modal tuning.	required
key	Optional[str]	An optional key to uniquely identify the set of parameters being tuned, useful for differentiating parameters in multi-modal scenarios. Defaults to None.	None

Returns:

Name	Type	Description
Any		The object with its parameters set according to the trial suggestions.

Notes

• The function uses map_suggest_method to map the tuning method based on the method type provided in the tunables.
• It supports multi-modal tuning, where different modalities may require different parameters to be tuned.
• For utility functions (UtilFunction), the function further tunes parameters based on the selected function from the modality.

Example

Assuming trial is an instance of optuna.trial.Trial, and explainer: Explainer is an object with tunable parameters, you can tune it as follows:

tuned_explainer = suggest(trial, explainer, modality)

pnpxai.evaluator.optimizer.types

OptimizationOutput dataclass

A dataclass used to store the results of an optimization process, including the chosen explainer, postprocessor, and the Optuna study object that was used during the optimization.

Attributes:

Name	Type	Description
explainer	Explainer	The explainer object selected during the optimization process. This object is responsible for generating explanations or attributions for a model.
postprocessor	PostProcessor	The postprocessor object selected during the optimization process. This object is used to process the attributions generated by the explainer, often involving normalization or other transformations.
study	Study	The Optuna study object that was used to conduct the optimization. This object contains information about the optimization trials and results.

explainer: Explainer instance-attribute

postprocessor: PostProcessor instance-attribute

study: optuna.study.Study instance-attribute

__init__(explainer: Explainer, postprocessor: PostProcessor, study: optuna.study.Study) -> None