Shortcuts

Accuracy

class mmpretrain.evaluation.Accuracy(topk=(1,), thrs=0.0, collect_device='cpu', prefix=None)[source]

Accuracy evaluation metric.

For either binary classification or multi-class classification, the accuracy is the fraction of correct predictions in all predictions:

\[\text{Accuracy} = \frac{N_{\text{correct}}}{N_{\text{all}}}\]
Parameters:
  • topk (int | Sequence[int]) – If the ground truth label matches one of the best k predictions, the sample will be regard as a positive prediction. If the parameter is a tuple, all of top-k accuracy will be calculated and outputted together. Defaults to 1.

  • thrs (Sequence[float | None] | float | None) – If a float, predictions with score lower than the threshold will be regard as the negative prediction. If None, not apply threshold. If the parameter is a tuple, accuracy based on all thresholds will be calculated and outputted together. Defaults to 0.

  • collect_device (str) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.

  • prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.

Examples

>>> import torch
>>> from mmpretrain.evaluation import Accuracy
>>> # -------------------- The Basic Usage --------------------
>>> y_pred = [0, 2, 1, 3]
>>> y_true = [0, 1, 2, 3]
>>> Accuracy.calculate(y_pred, y_true)
tensor([50.])
>>> # Calculate the top1 and top5 accuracy.
>>> y_score = torch.rand((1000, 10))
>>> y_true = torch.zeros((1000, ))
>>> Accuracy.calculate(y_score, y_true, topk=(1, 5))
[[tensor([9.9000])], [tensor([51.5000])]]
>>>
>>> # ------------------- Use with Evalutor -------------------
>>> from mmpretrain.structures import DataSample
>>> from mmengine.evaluator import Evaluator
>>> data_samples = [
...     DataSample().set_gt_label(0).set_pred_score(torch.rand(10))
...     for i in range(1000)
... ]
>>> evaluator = Evaluator(metrics=Accuracy(topk=(1, 5)))
>>> evaluator.process(data_samples)
>>> evaluator.evaluate(1000)
{
    'accuracy/top1': 9.300000190734863,
    'accuracy/top5': 51.20000076293945
}
static calculate(pred, target, topk=(1,), thrs=(0.0,))[source]

Calculate the accuracy.

Parameters:
  • pred (torch.Tensor | np.ndarray | Sequence) – The prediction results. It can be labels (N, ), or scores of every class (N, C).

  • target (torch.Tensor | np.ndarray | Sequence) – The target of each prediction with shape (N, ).

  • thrs (Sequence[float]) – Predictions with scores under the thresholds are considered negative. It’s only used when pred is scores. None means no thresholds. Defaults to (0., ).

  • thrs – Predictions with scores under the thresholds are considered negative. It’s only used when pred is scores. Defaults to (0., ).

Returns:

Accuracy.

  • torch.Tensor: If the pred is a sequence of label instead of score (number of dimensions is 1). Only return a top-1 accuracy tensor, and ignore the argument topk` and ``thrs.

  • List[List[torch.Tensor]]: If the pred is a sequence of score (number of dimensions is 2). Return the accuracy on each topk and thrs. And the first dim is topk, the second dim is thrs.

Return type:

torch.Tensor | List[List[torch.Tensor]]

compute_metrics(results)[source]

Compute the metrics from processed results.

Parameters:

results (dict) – The processed results of each batch.

Returns:

The computed metrics. The keys are the names of the metrics, and the values are corresponding results.

Return type:

Dict

process(data_batch, data_samples)[source]

Process one batch of data samples.

The processed results should be stored in self.results, which will be used to computed the metrics when all batches have been processed.

Parameters:
  • data_batch – A batch of data from the dataloader.

  • data_samples (Sequence[dict]) – A batch of outputs from the model.

Read the Docs v: latest
Versions
latest
stable
mmcls-1.x
mmcls-0.x
dev
Downloads
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.