feat: add BEiT support

[aymen-000]

Description

This PR adds support for BEiT (BERT Pre-Training of Image Transformers) for masked image modeling to Lightly.

What was added

• Implemented the BEiT masked image modeling architecture.
• Added the BEiT image tokenizer.
• Added the BEiT masking transform.
• Added the masked image modeling prediction head.
• Added training examples (single-GPU and distributed).
• Added notebooks demonstrating BEiT training.
• Added documentation and API references.
• Added unit tests covering the new components.

Relation to the existing MAE implementation

While both MAE and BEiT are masked image modeling methods, they differ in their reconstruction targets:

• MAE reconstructs the raw pixel values of masked image patches.
• BEiT predicts discrete visual tokens generated by a pretrained image tokenizer instead of reconstructing pixels.

References

• BEiT: BERT Pre-Training of Image Transformers
  https://arxiv.org/abs/2106.08254

• Original implementation
  https://github.com/microsoft/unilm/tree/master/beit

Documentation

• I have updated the documentation.

Tests

• I have updated the tests.

[liopeer]

/review

[liopeer]

Thanks a lot @aymen-000, what a cool contribution! This will probably take a bit of time to review, but we will try to do it as quickly as possible.

[liopeer]

This is only an initial review with already some remarks. Will continue over the next few days. 🙂

[aymen-000]

Hi @liopeer thanks for the detailed review! I've addressed the comments:

• Removed the tokenizer pretraining recipe.
• Moved the BEiT implementation into the example.
• Renamed the tokenizer, MIM head, and model class for consistency.
• Froze the tokenizer parameters and set the tokenizer to evaluation mode.
• Updated the documentation, including the suggested capitalization fix.

Thanks again! I'll keep an eye out for any additional comments from the rest of the review.