Shortcuts

DeiTClsHead

class mmpretrain.models.heads.DeiTClsHead(num_classes, in_channels, hidden_dim=None, act_cfg={'type': 'Tanh'}, init_cfg={'layer': 'Linear', 'type': 'Constant', 'val': 0}, **kwargs)[源代码]

Distilled Vision Transformer classifier head.

Comparing with the VisionTransformerClsHead, this head adds an extra linear layer to handle the dist token. The final classification score is the average of both linear transformation results of cls_token and dist_token.

参数:
  • num_classes (int) – Number of categories excluding the background category.

  • in_channels (int) – Number of channels in the input feature map.

  • hidden_dim (int, optional) – Number of the dimensions for hidden layer. Defaults to None, which means no extra hidden layer.

  • act_cfg (dict) – The activation config. Only available during pre-training. Defaults to dict(type='Tanh').

  • init_cfg (dict) – The extra initialization configs. Defaults to dict(type='Constant', layer='Linear', val=0).

forward(feats)[源代码]

The forward process.

pre_logits(feats)[源代码]

The process before the final classification head.

The input feats is a tuple of list of tensor, and each tensor is the feature of a backbone stage. In DeiTClsHead, we obtain the feature of the last stage and forward in hidden layer if exists.

Read the Docs v: latest
Versions
latest
stable
mmcls-1.x
mmcls-0.x
dev
Downloads
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.