ignite.contrib.metrics

Contribution module of metrics

class ignite.contrib.metrics.AveragePrecision(activation=None, output_transform=<function AveragePrecision.<lambda>>)

Computes Average Precision accumulating predictions and the ground-truth during an epoch and applying sklearn.metrics.average_precision_score

Parameters

• activation (Callable, optional) – optional function to apply on prediction tensors, e.g. activation=torch.sigmoid to transform logits.

• output_transform (callable, optional) – a callable that is used to transform the ignite.engine.Engine’s process_function’s output into the form expected by the metric. This can be useful if, for example, you have a multi-output model and you want to compute the metric with respect to one of the outputs.

class ignite.contrib.metrics.ROC_AUC(activation=None, output_transform=<function ROC_AUC.<lambda>>)

Computes Area Under the Receiver Operating Characteristic Curve (ROC AUC) accumulating predictions and the ground-truth during an epoch and applying sklearn.metrics.roc_auc_score

Parameters

• activation (Callable, optional) – optional function to apply on prediction tensors, e.g. activation=torch.sigmoid to transform logits.

• output_transform (callable, optional) – a callable that is used to transform the ignite.engine.Engine’s process_function’s output into the form expected by the metric. This can be useful if, for example, you have a multi-output model and you want to compute the metric with respect to one of the outputs.

Regression metrics

Module ignite.contrib.metrics.regression provides implementations of metrics useful for regression tasks. Definitions of metrics are based on Botchkarev 2018, page 30 “Appendix 2. Metrics mathematical definitions”.

class ignite.contrib.metrics.regression.FractionalBias(output_transform=<function Metric.<lambda>>)

Calculates the Fractional Bias:

$\text{FB} = \frac{1}{n}\sum_{j=1}^n\frac{2 * (A_j - P_j)}{A_j + P_j}$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.ManhattanDistance(output_transform=<function Metric.<lambda>>)

Calculates the Manhattan Distance:

$\text{MD} = \sum_{j=1}^n (A_j - P_j)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.MaximumAbsoluteError(output_transform=<function Metric.<lambda>>)

Calculates the Maximum Absolute Error:

$\text{MaxAE} = \max_{j=1,n} \left( \lvert A_j-P_j \rvert \right)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.MeanAbsoluteRelativeError(output_transform=<function Metric.<lambda>>)

Calculate Mean Absolute Relative Error:

$\text{MARE} = \frac{1}{n}\sum_{j=1}^n\frac{\left|A_j-P_j\right|}{\left|A_j\right|}$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in the reference Botchkarev 2018.

• update must receive output of the form (y_pred, y)

compute()

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.MeanError(output_transform=<function Metric.<lambda>>)

Calculates the Mean Error:

$\text{ME} = \frac{1}{n}\sum_{j=1}^n (A_j - P_j)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in the reference Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.MeanNormalizedBias(output_transform=<function Metric.<lambda>>)

Calculates the Mean Normalized Bias:

$\text{MNB} = \frac{1}{n}\sum_{j=1}^n\frac{A_j - P_j}{A_j}$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in the reference Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function