diff --git a/.nojekyll b/.nojekyll index 0b40485..4605a75 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -488f4798 \ No newline at end of file +0cf60366 \ No newline at end of file diff --git a/2024/weeks/week04/slides.html b/2024/weeks/week04/slides.html index e453f6e..2d7ac6d 100644 --- a/2024/weeks/week04/slides.html +++ b/2024/weeks/week04/slides.html @@ -564,12 +564,13 @@

\[ \begin{align*} -\alpha_{ij} & = \text{softmax}\left(\frac{q_i \cdot k_j}{\sqrt{d_{k}}}\right) \\ +\alpha_{ij} & = \frac{\exp(A_{ij})}{\sum_{j=1}^{n} \exp(A_{ij})} \end{align*} \]

-

where,

+

explained as: where,

diff --git a/search.json b/search.json index b4a4b70..c5b0266 100644 --- a/search.json +++ b/search.json @@ -158,7 +158,7 @@ "href": "2024/weeks/week04/slides.html#section-8", "title": "Week 04 Intro to Transformers", "section": "", - "text": "Softmax\n\n\nThe scaled dot-products are passed through a softmax function to get the attention weights.\n\n\\[\n\\begin{align*}\n\\alpha_{ij} & = \\text{softmax}\\left(\\frac{q_i \\cdot k_j}{\\sqrt{d_{k}}}\\right) \\\\\n\\end{align*}\n\\]\nwhere,\n\n\\(\\alpha_{ij}\\) is the attention weight for the i-th query and j-th key.\nThe softmax function is applied to each row of the attention score matrix \\(\\text{A}\\)." + "text": "Softmax\n\n\nThe scaled dot-products are passed through a softmax function to get the attention weights.\n\n\\[\n\\begin{align*}\n\\alpha_{ij} & = \\frac{\\exp(A_{ij})}{\\sum_{j=1}^{n} \\exp(A_{ij})}\n\\end{align*}\n\\]\nexplained as: where,\n\n\\(\\alpha_{ij}\\) is the attention weight for the i-th query and j-th key.\n\\(\\exp\\) is the exponential function (e=2.71828).\nThe softmax function is applied to each row of the attention score matrix \\(\\text{A}\\)." }, { "objectID": "2024/weeks/week04/slides.html#section-9", diff --git a/sitemap.xml b/sitemap.xml index fd512f0..3018529 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,66 +2,66 @@ https://llm4linguists.xyz/2024/weeks/week03/slides.html - 2024-11-06T11:56:37.253Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/weeks/week04/slides.html - 2024-11-06T11:56:37.253Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/assignments/assignment01.html - 2024-11-06T11:56:37.249Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/homework/homework02.html - 2024-11-06T11:56:37.249Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/weeks/week03/page.html - 2024-11-06T11:56:37.253Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/weeks/week01/slides.html - 2024-11-06T11:56:37.249Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/syllabus.html - 2024-11-06T11:56:37.249Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/assignments.html - 2024-11-06T11:56:37.249Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/index.html - 2024-11-06T11:56:37.285Z + 2024-11-06T12:10:06.241Z https://llm4linguists.xyz/2024/homework.html - 2024-11-06T11:56:37.249Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/weeks/week01/page.html - 2024-11-06T11:56:37.249Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/weeks/week02/page.html - 2024-11-06T11:56:37.253Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/homework/homework01.html - 2024-11-06T11:56:37.249Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/homework/homework03.html - 2024-11-06T11:56:37.249Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/weeks/week04/page.html - 2024-11-06T11:56:37.253Z + 2024-11-06T12:10:06.205Z https://llm4linguists.xyz/2024/weeks/week02/slides.html - 2024-11-06T11:56:37.253Z + 2024-11-06T12:10:06.205Z