SOTAX Transformers

In the seminar, the Transformer Neural network architecture is presented. First, basic transformer for sequence processing is explained. We look at how the attention mechanism works and how position embedding is performed. Later we analyze more modern transformer architectures used for vision and PC applications. Specifically: ViT, DETR, Swin Transformer, and Point Transformer.