Encoders Explained

Understand the transformer architecture by learning about encoders with detailed explanations on the architecture and a mini-project

How to use

This repository is not made to use the code inside in itself, but as a summary of differents classes and papers you can find on the internet. It is a complete guide to understand the basics, but in details, of how encoders within the Transformer architecture work and how they can be used as a standalone architecture for certain tasks.

You will find:

1. A explanations.ipynb notebook in which you will find all the information about encoders and their code implementation.

2. A mini-project folder in which you will find a code and a cleaner code for the implemenation of the encoder which can be called directly like a library

References

Original Paper

• Vaswani, A., et al. (2017). "Attention Is All You Need". arXiv:1706.03762. [Paper]

Video Resources

• Hugging Face. (2022). "Transformer: encoder". [YouTube]
• Machine Learning Studio. "A Dive Into Multihead Attention, Self-Attention and Cross-Attention". [YouTube]
• Machine Learning Studio. "Self-Attention Using Scaled Dot-Product Approach". [YouTube]