bottleneck transformer

Bottleneck transformers are a type of transformer architecture that aims to reduce the computational complexity of the standard transformer models. They achieve this by using a bottleneck layer that reduces the dimensionality of the input features before passing them through the self-attention and feedforward layers. This allows the model to have a smaller number of parameters and be more efficient, while still maintaining good performance on various natural language processing tasks.