Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transformer Day Chapter #145

Open
1 of 7 tasks
ramon-astudillo opened this issue Jun 6, 2023 · 1 comment
Open
1 of 7 tasks

Transformer Day Chapter #145

ramon-astudillo opened this issue Jun 6, 2023 · 1 comment
Assignees

Comments

@ramon-astudillo
Copy link
Member

ramon-astudillo commented Jun 6, 2023

Create the labs guide for the Transformers day under pages/transformers/. This should include

  • Introduction and explanation of the Transformer architecture (both encoder and decoder)

    • Add a draft of Transformer formulation (feel free to modify) @ramon-astudillo (expected today)
    • Check other days as reference
  • Explanation in detail of the attention mechanism. Maybe include some plots of attention, in particular causal attention. Attention is the most important part of Transformers so it's worth expanding this in detail. Maybe positional embeddings also deserve some love.

The day will be centered in the decoder and we can leave the encoder-decider as a final section with no exercises for this year.

branch: https://github.com/LxMLS/lxmls-guide/tree/transformer-day

Expected Finishing date:

  • June 12 - June 19 for the Intro and explanation of architecture
  • We depend on the other issues finishing for the rest, but we can have placeholders for exercises and complete as we go. I'd say before June 10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants