Metrics
pnpxai.evaluator.metrics.complexity
Complexity
Bases: Metric
Computes the complexity of attributions.
Given attributions
, calculates a fractional contribution distribution prob_mass
,
prob_mass[i] = hist[i] / sum(hist)
. where hist[i] = histogram(attributions[i])
.
The complexity is defined by the entropy,
evaluation = -sum(hist * ln(hist))
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model
|
Model
|
The model used for evaluation |
required |
explainer
|
Optional[Explainer]
|
The explainer used for evaluation. |
None
|
n_bins
|
int
|
The number of bins for histogram computation. |
10
|
Reference
U. Bhatt, A. Weller, and J. M. F. Moura. Evaluating and aggregating feature-based model attributions. In Proceedings of the IJCAI (2020).
n_bins = n_bins
instance-attribute
SUPPORTED_EXPLANATION_TYPE: ExplanationType = 'attribution'
class-attribute
instance-attribute
model = model.eval()
instance-attribute
explainer = explainer
instance-attribute
device = next(model.parameters()).device
instance-attribute
__init__(model: Model, explainer: Optional[Explainer] = None, n_bins: int = 10)
evaluate(inputs: torch.Tensor, targets: torch.Tensor, attributions: Optional[torch.Tensor] = None) -> torch.Tensor
Evaluate the explainer's complexity based on their probability masses.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
inputs
|
Optional[Tensor]
|
The input tensors to the model. |
required |
targets
|
Optional[Tensor]
|
The target labels for the inputs. |
required |
attributions
|
Optional[Tensor]
|
The attributions for the inputs. |
None
|
Returns:
Name | Type | Description |
---|---|---|
Tensor |
Tensor
|
A tensor of the complexity evaluations. |
__repr__()
Provides a string representation of the Metric object, displaying its attributes.
Returns:
Name | Type | Description |
---|---|---|
str |
A string representation of the Metric object. |
copy()
Creates a shallow copy of the Metric object.
Returns:
Name | Type | Description |
---|---|---|
Metric |
A copy of the Metric object. |
set_explainer(explainer: Explainer)
Sets the explainer for the metric, ensuring it is associated with the same model.
set_kwargs(**kwargs)
Sets additional attributes for the Metric object using keyword arguments.
pnpxai.evaluator.metrics.mu_fidelity
MuFidelity
Bases: Metric
Computes the MuFidelity metric for attributions.
Given a model
and inputs
, mufidelity of model
to an explainer at inputs
is calculated by
a correlation between difference of predictions and attributions of maked inputs,
evaluation = corr(pred_diff, masked_attr)
.
The masked inputs are generated by masking subset_mask
to noised inputs
,
masked = perturbed * subset_mask + (1.0 - subset_mask) * baseline
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model
|
Model
|
The model to evaluate. |
required |
explainer
|
Optional[Explainer]
|
The explainer to evaluate. |
None
|
n_perturb
|
int
|
Number of perturbations to generate. |
150
|
noise_scale
|
int
|
Scale factor for Gaussian random noise. |
0.2
|
batch_size
|
int
|
Batch size for model evaluation. |
32
|
grid_size
|
int
|
Size of the grid for creating subsets. |
9
|
baseline
|
Union[float, Tensor]
|
Baseline value for masked subsets. |
0.0
|
mask_agg_dim
|
Optional[int]
|
Dimension to aggregate masks. |
None
|
**kwargs
|
Additional kwargs to compute metric in an evaluator. Not required for single usage. |
required |
Reference
U. Bhatt, A. Weller, and J. M. F. Moura. Evaluating and aggregating feature-based model attributions. In Proceedings of the IJCAI (2020).
n_perturb = n_perturb
instance-attribute
noise_scale = noise_scale
instance-attribute
batch_size = batch_size
instance-attribute
grid_size = grid_size
instance-attribute
baseline = baseline
instance-attribute
mask_agg_dim = mask_agg_dim
instance-attribute
SUPPORTED_EXPLANATION_TYPE: ExplanationType = 'attribution'
class-attribute
instance-attribute
model = model.eval()
instance-attribute
explainer = explainer
instance-attribute
device = next(model.parameters()).device
instance-attribute
__init__(model: Model, explainer: Optional[Explainer] = None, n_perturb: int = 150, noise_scale: float = 0.2, batch_size: int = 32, grid_size: int = 9, baseline: float = 0.0, mask_agg_dim: Optional[int] = None)
evaluate(inputs: torch.Tensor, targets: torch.Tensor, attributions: Optional[torch.Tensor] = None) -> torch.Tensor
Parameters:
Name | Type | Description | Default |
---|---|---|---|
inputs
|
Tensor
|
The input data (N x C x H x W). |
required |
targets
|
Tensor
|
The target labels for the inputs (N x 1). |
required |
attributions
|
Tensor
|
The attributions of the inputs. |
None
|
Returns:
Type | Description |
---|---|
Tensor
|
torch.Tensor: The result of the metric evaluation. |
__repr__()
Provides a string representation of the Metric object, displaying its attributes.
Returns:
Name | Type | Description |
---|---|---|
str |
A string representation of the Metric object. |
copy()
Creates a shallow copy of the Metric object.
Returns:
Name | Type | Description |
---|---|---|
Metric |
A copy of the Metric object. |
set_explainer(explainer: Explainer)
Sets the explainer for the metric, ensuring it is associated with the same model.
set_kwargs(**kwargs)
Sets additional attributes for the Metric object using keyword arguments.
pnpxai.evaluator.metrics.sensitivity
Sensitivity
Bases: Metric
Computes the complexity of attributions.
Given attributions
, calculates a fractional contribution distribution prob_mass
,
prob_mass[i] = hist[i] / sum(hist)
. where hist[i] = histogram(attributions[i])
.
The complexity is defined by the entropy,
evaluation = -sum(hist * ln(hist))
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model
|
Model
|
The model used for evaluation |
required |
explainer
|
Optional[Union[Explainer, Callable]]
|
The explainer used for evaluation. It can be an instance of |
None
|
n_iter
|
Optional[int]
|
The number of iterations for perturbation. |
8
|
epsilon
|
Optional[float]
|
The magnitude of random uniform noise. |
0.2
|
n_iter = n_iter
instance-attribute
epsilon = epsilon
instance-attribute
SUPPORTED_EXPLANATION_TYPE: ExplanationType = 'attribution'
class-attribute
instance-attribute
model = model.eval()
instance-attribute
explainer = explainer
instance-attribute
device = next(model.parameters()).device
instance-attribute
__init__(model: Model, explainer: Optional[Union[Explainer, Callable]] = None, n_iter: Optional[int] = 8, epsilon: Optional[float] = 0.2)
evaluate(inputs: torch.Tensor, targets: torch.Tensor, attributions: Optional[torch.Tensor] = None) -> torch.Tensor
Parameters:
Name | Type | Description | Default |
---|---|---|---|
inputs
|
Tensor
|
The input data. |
required |
targets
|
Tensor
|
The target labels for the inputs. |
required |
attributions
|
Optional[Tensor]
|
The attributions of the inputs. |
None
|
Returns:
Type | Description |
---|---|
Tensor
|
torch.Tensor: The result of the metric evaluation. |
__repr__()
Provides a string representation of the Metric object, displaying its attributes.
Returns:
Name | Type | Description |
---|---|---|
str |
A string representation of the Metric object. |
copy()
Creates a shallow copy of the Metric object.
Returns:
Name | Type | Description |
---|---|---|
Metric |
A copy of the Metric object. |
set_explainer(explainer: Explainer)
Sets the explainer for the metric, ensuring it is associated with the same model.
set_kwargs(**kwargs)
Sets additional attributes for the Metric object using keyword arguments.
pnpxai.evaluator.metrics.pixel_flipping
BaselineFunction = Union[Callable[[Tensor], Tensor], Tuple[Callable[[Tensor], Tensor]]]
module-attribute
PixelFlipping
Bases: Metric
A metric class for evaluating the correctness of explanations or attributions provided by an explainer using the pixel flipping technique.
This class assesses the quality of attributions by perturbing input features (e.g., pixels) in the order of their attributed importance and measuring the resulting change in the model's predictions. Correct attributions should lead to significant changes in model predictions when the most important features are perturbed.
Attributes:
Name | Type | Description |
---|---|---|
model |
Module
|
The model. |
explainer |
Optional[Explainer]=None
|
The explainer whose explanations are being evaluated. |
channel_dim |
int
|
Target channel dimension. |
n_steps |
int
|
The number of perturbation steps. |
baseline_fn |
Optional[BaselineFunction]
|
Function to generate baseline inputs for perturbation. |
prob_fn |
Optional[Callable[[Tensor], Tensor]]
|
Function to compute probabilities from model outputs. |
pred_fn |
Optional[Callable[[Tensor], Tensor]]
|
Function to compute predictions from model outputs. |
forward_arg_extractor |
Optional[Callable[[Tuple[Tensor]], Union[Tensor, Tuple[Tensor]]]]
|
Optional function to extract forward arguments from inputs. |
additional_forward_arg_extractor |
Optional[Callable[[Tuple[Tensor]], Union[Tensor, Tuple[Tensor]]]]
|
Optional function to extract additional forward arguments. |
Methods:
Name | Description |
---|---|
evaluate |
Evaluate the explainer's correctness based on the attributions by observing changes in model predictions. |
channel_dim = channel_dim
instance-attribute
n_steps = n_steps
instance-attribute
baseline_fn = baseline_fn or ZeroBaselineFunction()
instance-attribute
prob_fn = prob_fn
instance-attribute
pred_fn = pred_fn
instance-attribute
forward_arg_extractor = forward_arg_extractor
instance-attribute
additional_forward_arg_extractor = additional_forward_arg_extractor
instance-attribute
SUPPORTED_EXPLANATION_TYPE: ExplanationType = 'attribution'
class-attribute
instance-attribute
model = model.eval()
instance-attribute
explainer = explainer
instance-attribute
device = next(model.parameters()).device
instance-attribute
__init__(model: Module, explainer: Optional[Explainer] = None, channel_dim: int = 1, n_steps: int = 10, baseline_fn: Optional[BaselineFunction] = None, prob_fn: Optional[Callable[[Tensor], Tensor]] = lambda outputs: outputs.softmax(-1), pred_fn: Optional[Callable[[Tensor], Tensor]] = lambda outputs: outputs.argmax(-1), forward_arg_extractor: Optional[ForwardArgumentExtractor] = None, additional_forward_arg_extractor: Optional[ForwardArgumentExtractor] = None)
evaluate(inputs: TensorOrTupleOfTensors, targets: Tensor, attributions: TensorOrTupleOfTensors, attention_mask: Optional[TensorOrTupleOfTensors] = None, descending: bool = True) -> Union[Dict[str, Tensor], Tuple[Dict[str, Tensor]]]
Evaluate the explainer's correctness based on the attributions by observing changes in model predictions.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
inputs
|
TensorOrTupleOfTensors
|
The input tensors to the model. |
required |
targets
|
Tensor
|
The target labels for the inputs. |
required |
attributions
|
TensorOrTupleOfTensors
|
The attributions for the inputs. |
required |
attention_mask
|
Optional[TensorOrTupleOfTensors]
|
Attention masks for the inputs. Default is None. |
None
|
descending
|
bool
|
Whether to flip pixels in descending order of attribution. Default is True. |
True
|
Returns:
Type | Description |
---|---|
Union[Dict[str, Tensor], Tuple[Dict[str, Tensor]]]
|
Union[Dict[str, Tensor], Tuple[Dict[str, Tensor]]]: A dictionary or tuple of dictionaries containing the probabilities and predictions at each perturbation step. |
__repr__()
Provides a string representation of the Metric object, displaying its attributes.
Returns:
Name | Type | Description |
---|---|---|
str |
A string representation of the Metric object. |
copy()
Creates a shallow copy of the Metric object.
Returns:
Name | Type | Description |
---|---|---|
Metric |
A copy of the Metric object. |
set_explainer(explainer: Explainer)
Sets the explainer for the metric, ensuring it is associated with the same model.
set_kwargs(**kwargs)
Sets additional attributes for the Metric object using keyword arguments.
MoRF
Bases: PixelFlipping
A metric class for evaluating the correctness of explanations or attributions using the Most Relevant First (MoRF) pixel flipping technique.
This class inherits from the PixelFlipping class and evaluates the quality of attributions by perturbing input features (e.g., pixels) in descending order of their attributed importance. The average probability change is measured to assess the explainer's correctness (lower better).
Attributes:
Name | Type | Description |
---|---|---|
model |
Module
|
The model. |
explainer |
Optional[Explainer]=None
|
The explainer whose explanations are being evaluated. |
channel_dim |
int
|
Target channel dimension. |
n_steps |
int
|
The number of perturbation steps. |
baseline_fn |
Optional[BaselineFunction]
|
Function to generate baseline inputs for perturbation. |
prob_fn |
Optional[Callable[[Tensor], Tensor]]
|
Function to compute probabilities from model outputs. |
pred_fn |
Optional[Callable[[Tensor], Tensor]]
|
Function to compute predictions from model outputs. |
forward_arg_extractor |
Optional[Callable[[Tuple[Tensor]], Union[Tensor, Tuple[Tensor]]]]
|
Optional function to extract forward arguments from inputs. |
additional_forward_arg_extractor |
Optional[Callable[[Tuple[Tensor]], Union[Tensor, Tuple[Tensor]]]]
|
Optional function to extract additional forward arguments. |
Methods:
Name | Description |
---|---|
evaluate |
Evaluate the explainer's correctness using the MoRF technique by observing changes in model predictions. |
SUPPORTED_EXPLANATION_TYPE: ExplanationType = 'attribution'
class-attribute
instance-attribute
model = model.eval()
instance-attribute
explainer = explainer
instance-attribute
device = next(model.parameters()).device
instance-attribute
channel_dim = channel_dim
instance-attribute
n_steps = n_steps
instance-attribute
baseline_fn = baseline_fn or ZeroBaselineFunction()
instance-attribute
prob_fn = prob_fn
instance-attribute
pred_fn = pred_fn
instance-attribute
forward_arg_extractor = forward_arg_extractor
instance-attribute
additional_forward_arg_extractor = additional_forward_arg_extractor
instance-attribute
__init__(model: Module, explainer: Optional[Explainer] = None, channel_dim: int = 1, n_steps: int = 10, baseline_fn: Optional[BaselineFunction] = None, prob_fn: Optional[Callable[[Tensor], Tensor]] = lambda outputs: outputs.softmax(-1), pred_fn: Optional[Callable[[Tensor], Tensor]] = lambda outputs: outputs.argmax(-1), forward_arg_extractor: Optional[ForwardArgumentExtractor] = None, additional_forward_arg_extractor: Optional[ForwardArgumentExtractor] = None)
evaluate(inputs: TensorOrTupleOfTensors, targets: Tensor, attributions: TensorOrTupleOfTensors, attention_mask: Optional[TensorOrTupleOfTensors] = None) -> TensorOrTupleOfTensors
Evaluate the explainer's correctness using the MoRF technique by observing changes in model predictions.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
inputs
|
TensorOrTupleOfTensors
|
The input tensors to the model. |
required |
targets
|
Tensor
|
The target labels for the inputs. |
required |
attributions
|
TensorOrTupleOfTensors
|
The attributions for the inputs. |
required |
attention_mask
|
Optional[TensorOrTupleOfTensors]
|
Attention masks for the inputs. Default is None. |
None
|
Returns:
Name | Type | Description |
---|---|---|
TensorOrTupleOfTensors |
TensorOrTupleOfTensors
|
The mean probabilities at each perturbation step, indicating the impact of perturbing the most relevant features first. |
__repr__()
Provides a string representation of the Metric object, displaying its attributes.
Returns:
Name | Type | Description |
---|---|---|
str |
A string representation of the Metric object. |
copy()
Creates a shallow copy of the Metric object.
Returns:
Name | Type | Description |
---|---|---|
Metric |
A copy of the Metric object. |
set_explainer(explainer: Explainer)
Sets the explainer for the metric, ensuring it is associated with the same model.
set_kwargs(**kwargs)
Sets additional attributes for the Metric object using keyword arguments.
LeRF
Bases: PixelFlipping
A metric class for evaluating the correctness of explanations or attributions using the Least Relevant First (LeRF) pixel flipping technique.
This class inherits from the PixelFlipping class and evaluates the quality of attributions by perturbing input features (e.g., pixels) in ascending order of their attributed importance. The average probability change is measured to assess the explainer's correctness.
Attributes:
Name | Type | Description |
---|---|---|
model |
Module
|
The model. |
explainer |
Optional[Explainer]=None
|
The explainer whose explanations are being evaluated. |
channel_dim |
int
|
Target channel dimension. |
n_steps |
int
|
The number of perturbation steps. |
baseline_fn |
Optional[BaselineFunction]
|
Function to generate baseline inputs for perturbation. |
prob_fn |
Optional[Callable[[Tensor], Tensor]]
|
Function to compute probabilities from model outputs. |
pred_fn |
Optional[Callable[[Tensor], Tensor]]
|
Function to compute predictions from model outputs. |
forward_arg_extractor |
Optional[Callable[[Tuple[Tensor]], Union[Tensor, Tuple[Tensor]]]]
|
Optional function to extract forward arguments from inputs. |
additional_forward_arg_extractor |
Optional[Callable[[Tuple[Tensor]], Union[Tensor, Tuple[Tensor]]]]
|
Optional function to extract additional forward arguments. |
Methods:
Name | Description |
---|---|
evaluate |
Evaluate the explainer's correctness using the LeRF technique by observing changes in model predictions. |
SUPPORTED_EXPLANATION_TYPE: ExplanationType = 'attribution'
class-attribute
instance-attribute
model = model.eval()
instance-attribute
explainer = explainer
instance-attribute
device = next(model.parameters()).device
instance-attribute
channel_dim = channel_dim
instance-attribute
n_steps = n_steps
instance-attribute
baseline_fn = baseline_fn or ZeroBaselineFunction()
instance-attribute
prob_fn = prob_fn
instance-attribute
pred_fn = pred_fn
instance-attribute
forward_arg_extractor = forward_arg_extractor
instance-attribute
additional_forward_arg_extractor = additional_forward_arg_extractor
instance-attribute
__init__(model: Module, explainer: Optional[Explainer] = None, channel_dim: int = 1, n_steps: int = 10, baseline_fn: Optional[BaselineFunction] = None, prob_fn: Optional[Callable[[Tensor], Tensor]] = lambda outputs: outputs.softmax(-1), pred_fn: Optional[Callable[[Tensor], Tensor]] = lambda outputs: outputs.argmax(-1), forward_arg_extractor: Optional[ForwardArgumentExtractor] = None, additional_forward_arg_extractor: Optional[ForwardArgumentExtractor] = None)
evaluate(inputs: TensorOrTupleOfTensors, targets: Tensor, attributions: TensorOrTupleOfTensors, attention_mask: TensorOrTupleOfTensors = None) -> TensorOrTupleOfTensors
Evaluate the explainer's correctness using the LeRF technique by observing changes in model predictions.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
inputs
|
TensorOrTupleOfTensors
|
The input tensors to the model. |
required |
targets
|
Tensor
|
The target labels for the inputs. |
required |
attributions
|
TensorOrTupleOfTensors
|
The attributions for the inputs. |
required |
attention_mask
|
Optional[TensorOrTupleOfTensors]
|
Attention masks for the inputs. Default is None. |
None
|
Returns:
Name | Type | Description |
---|---|---|
TensorOrTupleOfTensors |
TensorOrTupleOfTensors
|
The mean probabilities at each perturbation step, indicating the impact of perturbing the least relevant features first. |
__repr__()
Provides a string representation of the Metric object, displaying its attributes.
Returns:
Name | Type | Description |
---|---|---|
str |
A string representation of the Metric object. |
copy()
Creates a shallow copy of the Metric object.
Returns:
Name | Type | Description |
---|---|---|
Metric |
A copy of the Metric object. |
set_explainer(explainer: Explainer)
Sets the explainer for the metric, ensuring it is associated with the same model.
set_kwargs(**kwargs)
Sets additional attributes for the Metric object using keyword arguments.
AbPC
Bases: PixelFlipping
A metric class for evaluating the correctness of explanations or attributions using the Area between Perturbation Curves (AbPC) technique.
This class inherits from the PixelFlipping class and assesses the quality of attributions by comparing the area between the perturbation curves obtained by perturbing input features (e.g., pixels) in both ascending and descending order of their attributed importance. The average probability change is measured, providing a comprehensive evaluation of the explainer's correctness.
Attributes:
Name | Type | Description |
---|---|---|
model |
Module
|
The model. |
explainer |
Optional[Explainer]=None
|
The explainer whose explanations are being evaluated. |
channel_dim |
int
|
Target channel dimension. |
n_steps |
int
|
The number of perturbation steps. |
baseline_fn |
Optional[BaselineFunction]
|
Function to generate baseline inputs for perturbation. |
prob_fn |
Optional[Callable[[Tensor], Tensor]]
|
Function to compute probabilities from model outputs. |
pred_fn |
Optional[Callable[[Tensor], Tensor]]
|
Function to compute predictions from model outputs. |
forward_arg_extractor |
Optional[Callable[[Tuple[Tensor]], Union[Tensor, Tuple[Tensor]]]]
|
Optional function to extract forward arguments from inputs. |
additional_forward_arg_extractor |
Optional[Callable[[Tuple[Tensor]], Union[Tensor, Tuple[Tensor]]]]
|
Optional function to extract additional forward arguments. |
lb |
float
|
The lower bound for clamping the probability differences. |
Methods:
Name | Description |
---|---|
evaluate |
Evaluate the explainer's correctness using the AbPC technique by observing changes in model predictions. |
lb = lb
instance-attribute
SUPPORTED_EXPLANATION_TYPE: ExplanationType = 'attribution'
class-attribute
instance-attribute
model = model.eval()
instance-attribute
explainer = explainer
instance-attribute
device = next(model.parameters()).device
instance-attribute
channel_dim = channel_dim
instance-attribute
n_steps = n_steps
instance-attribute
baseline_fn = baseline_fn or ZeroBaselineFunction()
instance-attribute
prob_fn = prob_fn
instance-attribute
pred_fn = pred_fn
instance-attribute
forward_arg_extractor = forward_arg_extractor
instance-attribute
additional_forward_arg_extractor = additional_forward_arg_extractor
instance-attribute
__init__(model: Module, explainer: Optional[Explainer] = None, channel_dim: int = 1, n_steps: int = 10, baseline_fn: Optional[BaselineFunction] = None, prob_fn: Optional[Callable[[Tensor], Tensor]] = lambda outputs: outputs.softmax(-1), pred_fn: Optional[Callable[[Tensor], Tensor]] = lambda outputs: outputs.argmax(-1), forward_arg_extractor: Optional[ForwardArgumentExtractor] = None, additional_forward_arg_extractor: Optional[ForwardArgumentExtractor] = None, lb: float = -1.0)
evaluate(inputs: TensorOrTupleOfTensors, targets: Tensor, attributions: TensorOrTupleOfTensors, attention_mask: Optional[TensorOrTupleOfTensors] = None, return_pf=False) -> TensorOrTupleOfTensors
Evaluate the explainer's correctness using the AbPC technique by observing changes in model predictions.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
inputs
|
TensorOrTupleOfTensors
|
The input tensors to the model. |
required |
targets
|
Tensor
|
The target labels for the inputs. |
required |
attributions
|
TensorOrTupleOfTensors
|
The attributions for the inputs. |
required |
attention_mask
|
Optional[TensorOrTupleOfTensors]
|
Attention masks for the inputs. |
None
|
return_pf
|
Optional[bool]
|
Whether to return the perturbation curves for ascending and descending orders. |
False
|
Returns:
Name | Type | Description |
---|---|---|
TensorOrTupleOfTensors |
TensorOrTupleOfTensors
|
The mean clamped differences in probabilities at each perturbation step, indicating the impact of perturbing the most and least relevant features. |
__repr__()
Provides a string representation of the Metric object, displaying its attributes.
Returns:
Name | Type | Description |
---|---|---|
str |
A string representation of the Metric object. |
copy()
Creates a shallow copy of the Metric object.
Returns:
Name | Type | Description |
---|---|---|
Metric |
A copy of the Metric object. |
set_explainer(explainer: Explainer)
Sets the explainer for the metric, ensuring it is associated with the same model.
set_kwargs(**kwargs)
Sets additional attributes for the Metric object using keyword arguments.