LayerScale

class mmpretrain.models.utils.LayerScale(dim, layer_scale_init_value=1e-05, inplace=False, data_format='channels_last')

LayerScale layer.

Parameters:

• dim (int) – Dimension of input features.

• layer_scale_init_value (float or torch.Tensor) – Init value of layer scale. Defaults to 1e-5.

• inplace (bool) – inplace: can optionally do the operation in-place. Defaults to False.

• data_format (str) – The input data format, could be ‘channels_last’ or ‘channels_first’, representing (B, C, H, W) and (B, N, C) format data respectively. Defaults to ‘channels_last’.