MIMRefiner

Contents

MIMRefiner#

class stable_pretraining.methods.MIMRefiner(pretrained_encoder: Module, freeze_lower_blocks: int = 0, **ibot_kwargs)[source]#

Bases: iBOT

Refine a pretrained MIM encoder with iBOT-style self-distillation.

Parameters:

  • pretrained_encoder – A pre-trained nn.Module (e.g. the encoder of a trained MAE / SimMIM / Data2Vec instance, or a timm ViT loaded with pretrained=True). Required.

  • freeze_lower_blocks – Number of leading transformer blocks to freeze on the student (default 0). The teacher’s EMA already holds the MIM features regardless.

  • **ibot_kwargs –

    Forwarded to iBOT (projector dims, prototypes, mask ratio, etc.).

Contents