CorruptionError

class mmcls.evaluation.CorruptionError(topk=(1,), thrs=0.0, collect_device='cpu', prefix=None, ann_file=None)

Mean Corruption Error (mCE) metric.

The mCE metric is proposed in Benchmarking Neural Network Robustness to Common Corruptions and Perturbations.

参数

• topk (int | Sequence[int]) – If the ground truth label matches one of the best k predictions, the sample will be regard as a positive prediction. If the parameter is a tuple, all of top-k accuracy will be calculated and outputted together. Defaults to 1.

• thrs (Sequence[float | None] | float | None) – If a float, predictions with score lower than the threshold will be regard as the negative prediction. If None, not apply threshold. If the parameter is a tuple, accuracy based on all thresholds will be calculated and outputted together. Defaults to 0.

• collect_device (str) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.

• prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.

• ano_file (str, optional) – The path of the annotation file. This file will be used in evaluating the fine-tuned model on OOD dataset, e.g. ImageNet-A. Defaults to None.

compute_metrics(results)

Compute the metrics from processed results.

参数

results (dict) – The processed results of each batch.

返回

The computed metrics. The keys are the names of the metrics, and the values are corresponding results.

返回类型

Dict

process(data_batch, data_samples)

Process one batch of data samples.

The processed results should be stored in self.results, which will be used to computed the metrics when all batches have been processed. The difference between this method and process in Accuracy is that the img_path is extracted from the data_batch and stored in the self.results.

参数

• data_batch – A batch of data from the dataloader.

• data_samples (Sequence[dict]) – A batch of outputs from the model.