Built site for gh-pages

.nojekyll

@@ -1 +1 @@
-deb4dbf2
+aab39c9e

2024/weeks/week02/slides.html

@@ -366,7 +366,7 @@ <h2 id="toc-title">What we will cover today:</h2>
 </section>
 <section class="slide level2">
 
-<!-- 
+<!--
 # Mathematical concepts {.smaller}
 
 ## {.smaller}
@@ -500,22 +500,13 @@ <h2></h2>
 <h2></h2>
 <p><br> <img src="nn_perceptron.png"> The simplest Neural network only has one layer with one neuron. This single neuron is called a <strong>perceptron</strong>. <br></p>
 </section>
-<section id="perceptron" class="slide level2">
+<section id="perceptron" class="slide level2 smaller">
 <h2>Perceptron</h2>
-<!-- 
-
-
-
-
-::::::{.cell reveal=true layout-align="center"}
-
-:::::{.cell-output-display}
-
-::::{}
-`<figure class=''>`{=html}
-
-:::{}
-
+<div class="cell" data-reveal="true" data-layout-align="center">
+<div class="cell-output-display">
+<div>
+<p></p><figure class=""><p></p>
+<div>
 <pre class="mermaid mermaid-js">graph LR
     subgraph Inputs
         x1((x1))
@@ -543,25 +534,36 @@ <h2>Perceptron</h2>
     style act fill:#98FB98,stroke:#333,stroke-width:2px
     style b fill:#FFFF00,stroke:#333,stroke-width:2px
 </pre>
-:::
-`</figure>`{=html}
-::::
-:::::
-::::::
-
-
-
-
-
-
-
-- Input Nodes (x1, x2, x3): Each input is a number.
-- Weights (w1, w2, w3): Each weight is a number that determines the importance of the corresponding input.
-- Bias (b): A constant value that shifts the output of the perceptron.
-- Sum Node (Σ): Calculates the weighted sum of the inputs and the bias.
-- Activation Function: Introduces non-linearity to the output of the perceptron.
-- Output Node: The final output of the perceptron.
--->
+</div>
+<p></p></figure><p></p>
+</div>
+</div>
+</div>
+<ul>
+<li>Input Nodes (x1, x2, x3): Each input is a number.</li>
+<li>Weights (w1, w2, w3): Each weight is a number that determines the importance of the corresponding input.</li>
+<li>Bias (b): A constant value that shifts the output of the perceptron.</li>
+</ul>
+</section>
+<section id="section-3" class="slide level2 smaller">
+<h2></h2>
+<ul>
+<li>Sum Node (Σ): Calculates the weighted sum of the inputs and the bias.</li>
+<li>Activation Function (<span class="math inline">\(f\)</span>): Introduces non-linearity to the output of the perceptron.</li>
+<li>Output Node: The final output of the perceptron.</li>
+</ul>
+<p><span class="math display">\[
+\text{Output} = f(w_1 \times x_1 + w_2 \times x_2 + w_3 \times x_3 + b)
+\]</span></p>
+<ul>
+<li>The output of the perceptron is a weighted sum of the inputs and the bias passed through an activation function.</li>
+</ul>
+<p><strong>Why do we need non-linearity?</strong></p>
+<ul>
+<li>Non-linearity allows the perceptron to learn complex patterns in the data.</li>
+<li>Without non-linearity, the perceptron would be limited to learning linear patterns.</li>
+<li>Activation functions introduce non-linearity to the output of the perceptron.</li>
+</ul>
 </section>
 <section id="activation-functions" class="slide level2 smaller">
 <h2>Activation functions</h2>
@@ -582,7 +584,7 @@ <h2>Activation functions</h2>
 <li>It is used in the output layer of a binary classification problem.</li>
 </ul>
 </section>
-<section id="section-3" class="slide level2 smaller">
+<section id="section-4" class="slide level2 smaller">
 <h2></h2>
 <p><strong>ReLU function</strong></p>
 <p><span class="math display">\[
@@ -600,7 +602,7 @@ <h2></h2>
 <li>It is a popular activation function used in deep learning models.</li>
 </ul>
 </section>
-<section id="section-4" class="slide level2 smaller">
+<section id="section-5" class="slide level2 smaller">
 <h2></h2>
 <p><strong>Feedforward Neural Network</strong></p>
 <div class="cell" data-reveal="true" data-fig-width="5" data-fig-height="3" data-layout-align="center">
@@ -677,17 +679,22 @@ <h2>Feedforward Neural Network</h2>
 <li>The weights and biases are learned during the training process.</li>
 </ul>
 </section>
-<section id="section-5" class="slide level2 smaller">
+<section id="section-6" class="slide level2 smaller">
 <h2></h2>
 <p><strong>Loss function</strong></p>
 <ul>
-<li><p>During forward propagation, the neural network makes predictions based on input data.</p></li>
-<li><p>The loss function compares these predictions to the true values and calculates a loss score.</p></li>
-<li><p>The loss score is a measure of how well the network is performing.</p></li>
-<li><p>The goal of training is to minimize the loss function.</p></li>
-<li><p>For regression problems, use MSE or MAE.</p></li>
-<li><p>For classification problems, use cross-entropy loss.</p></li>
-<li><p>For multi-class classification problems, use categorical cross-entropy loss.</p></li>
+<li>During forward pass, the neural network makes predictions based on input data.</li>
+<li>The loss function compares these predictions to the true values and calculates a loss score.</li>
+<li>The loss score is a measure of how well the network is performing.</li>
+<li>The goal of training is to minimize the loss function.</li>
+</ul>
+</section>
+<section id="additional-resources" class="slide level2 smaller">
+<h2>Additional resources</h2>
+<ul>
+<li>What is a neural network? <a href="https://www.youtube.com/watch?v=aircAruvnKk">Video</a></li>
+<li>Gradient descent, how neural networks learn <a href="https://www.youtube.com/watch?v=IHZwWFHWa-w">Video</a></li>
+<li>Backpropagation, how neural networks learn <a href="https://www.youtube.com/watch?v=Ilg3gGewQ5U">Video</a></li>
 </ul>
 
 <div class="quarto-auto-generated-content">

search.json

@@ -32,7 +32,14 @@
     "href": "2024/weeks/week02/slides.html#perceptron",
     "title": "Week 02 Basics of Neural Networks",
     "section": "Perceptron",
-    "text": "Perceptron"
+    "text": "Perceptron\n\n\n\n\n\ngraph LR\n    subgraph Inputs\n        x1((x1))\n        x2((x2))\n        x3((x3))\n    end\n\n    sum((Σ))\n    act[Activation]\n    out((Output))\n    b[Bias]\n\n    x1 -->|w1| sum\n    x2 -->|w2| sum\n    x3 -->|w3| sum\n    b --> sum\n    sum --> act\n    act --> out\n\n    style Inputs fill:#87CEFA,stroke:#333,stroke-width:2px, fill-opacity: 0.5\n    style x1 fill:#87CEFA,stroke:#333,stroke-width:2px\n    style x2 fill:#87CEFA,stroke:#333,stroke-width:2px\n    style x3 fill:#87CEFA,stroke:#333,stroke-width:2px\n    style sum fill:#FFA07A,stroke:#333,stroke-width:2px\n    style act fill:#98FB98,stroke:#333,stroke-width:2px\n    style b fill:#FFFF00,stroke:#333,stroke-width:2px\n\n\n\n\n\n\n\nInput Nodes (x1, x2, x3): Each input is a number.\nWeights (w1, w2, w3): Each weight is a number that determines the importance of the corresponding input.\nBias (b): A constant value that shifts the output of the perceptron."
+  },
+  {
+    "objectID": "2024/weeks/week02/slides.html#section-3",
+    "href": "2024/weeks/week02/slides.html#section-3",
+    "title": "Week 02 Basics of Neural Networks",
+    "section": "",
+    "text": "Sum Node (Σ): Calculates the weighted sum of the inputs and the bias.\nActivation Function (\\(f\\)): Introduces non-linearity to the output of the perceptron.\nOutput Node: The final output of the perceptron.\n\n\\[\n\\text{Output} = f(w_1 \\times x_1 + w_2 \\times x_2 + w_3 \\times x_3 + b)\n\\]\n\nThe output of the perceptron is a weighted sum of the inputs and the bias passed through an activation function.\n\nWhy do we need non-linearity?\n\nNon-linearity allows the perceptron to learn complex patterns in the data.\nWithout non-linearity, the perceptron would be limited to learning linear patterns.\nActivation functions introduce non-linearity to the output of the perceptron."
   },
   {
     "objectID": "2024/weeks/week02/slides.html#activation-functions",
@@ -42,15 +49,15 @@
     "text": "Activation functions\n\nActivation functions are used to introduce non-linearity to the output of a neuron.\n\nSigmoid function\n\\[\nf(x) = \\frac{1}{1 + e^{-x}}\n\\]\nExample: \\(f(0) = 0.5\\)\n- f(x): This represents the output of the sigmoid function for a given input x.\n- e: This is the euler's number (approximately 2.71828).\n- x: This is the input to the sigmoid function.\n- 1: This is added to the denominator to avoid division by zero.\n\nThe sigmoid function takes any real number as input and outputs a value between 0 and 1.\nIt is used in the output layer of a binary classification problem."
   },
   {
-    "objectID": "2024/weeks/week02/slides.html#section-3",
-    "href": "2024/weeks/week02/slides.html#section-3",
+    "objectID": "2024/weeks/week02/slides.html#section-4",
+    "href": "2024/weeks/week02/slides.html#section-4",
     "title": "Week 02 Basics of Neural Networks",
     "section": "",
     "text": "ReLU function\n\\[\nf(x) = \\max(0, x)\n\\]\nExample: \\(f(2) = 2\\)\nwhere:\n- f(x): This represents the output of the ReLU function for a given input x.\n- x: This is the input to the ReLU function.\n- max: This function returns the maximum of the two values.\n- 0: This is the threshold value.\n\nThe Rectified Linear Unit (ReLU) function is that outputs the input directly if it is positive, otherwise, it outputs zero.\nThe output of the ReLU function is between 0 and infinity.\nIt is a popular activation function used in deep learning models."
   },
   {
-    "objectID": "2024/weeks/week02/slides.html#section-4",
-    "href": "2024/weeks/week02/slides.html#section-4",
+    "objectID": "2024/weeks/week02/slides.html#section-5",
+    "href": "2024/weeks/week02/slides.html#section-5",
     "title": "Week 02 Basics of Neural Networks",
     "section": "",
     "text": "Feedforward Neural Network\n\n\n\n\n\nflowchart LR\n    %% Input Layer\n    I1((I1)):::inputStyle\n    I2((I2)):::inputStyle\n    I3((I3)):::inputStyle\n    B1((Bias)):::biasStyle\n    %% Hidden Layer\n    H1((H1)):::hiddenStyle\n    H2((H2)):::hiddenStyle\n    H3((H3)):::hiddenStyle\n    B2((Bias)):::biasStyle\n    %% Output Layer\n    O1((O1)):::outputStyle\n    O2((O2)):::outputStyle\n\n    %% Connections\n    I1 -->|w11| H1\n    I1 -->|w12| H2\n    I1 -->|w13| H3\n    I2 -->|w21| H1\n    I2 -->|w22| H2\n    I2 -->|w23| H3\n    I3 -->|w31| H1\n    I3 -->|w32| H2\n    I3 -->|w33| H3\n    B1 -->|b1| H1\n    B1 -->|b2| H2\n    B1 -->|b3| H3\n    H1 -->|v11| O1\n    H1 -->|v12| O2\n    H2 -->|v21| O1\n    H2 -->|v22| O2\n    H3 -->|v31| O1\n    H3 -->|v32| O2\n    B2 -->|b4| O1\n    B2 -->|b5| O2\n\n    %% Styles\n    classDef inputStyle fill:#3498db,stroke:#333,stroke-width:2px;\n    classDef hiddenStyle fill:#e74c3c,stroke:#333,stroke-width:2px;\n    classDef outputStyle fill:#2ecc71,stroke:#333,stroke-width:2px;\n    classDef biasStyle fill:#f39c12,stroke:#333,stroke-width:2px;\n\n    %% Layer Labels\n    I2 -.- InputLabel[Input Layer]\n    H2 -.- HiddenLabel[Hidden Layer]\n    O1 -.- OutputLabel[Output Layer]\n\n    style InputLabel fill:none,stroke:none\n    style HiddenLabel fill:none,stroke:none\n    style OutputLabel fill:none,stroke:none"
@@ -63,11 +70,18 @@
     "text": "Feedforward Neural Network\n\nFeedforward neural network with three layers: input, hidden, and output.\nThe input layer has three nodes (I1, I2, I3).\nThe hidden layer has three nodes (H1, H2, H3).\nThe output layer has two nodes (O1, O2).\nEach connection between the nodes has a weight (w) and a bias (b).\nThe weights and biases are learned during the training process."
   },
   {
-    "objectID": "2024/weeks/week02/slides.html#section-5",
-    "href": "2024/weeks/week02/slides.html#section-5",
+    "objectID": "2024/weeks/week02/slides.html#section-6",
+    "href": "2024/weeks/week02/slides.html#section-6",
     "title": "Week 02 Basics of Neural Networks",
     "section": "",
-    "text": "Loss function\n\nDuring forward propagation, the neural network makes predictions based on input data.\nThe loss function compares these predictions to the true values and calculates a loss score.\nThe loss score is a measure of how well the network is performing.\nThe goal of training is to minimize the loss function.\nFor regression problems, use MSE or MAE.\nFor classification problems, use cross-entropy loss.\nFor multi-class classification problems, use categorical cross-entropy loss.\n\n\n\n\n\nLLMs in Lingustic Research WiSe 2024/25"
+    "text": "Loss function\n\nDuring forward pass, the neural network makes predictions based on input data.\nThe loss function compares these predictions to the true values and calculates a loss score.\nThe loss score is a measure of how well the network is performing.\nThe goal of training is to minimize the loss function."
+  },
+  {
+    "objectID": "2024/weeks/week02/slides.html#additional-resources",
+    "href": "2024/weeks/week02/slides.html#additional-resources",
+    "title": "Week 02 Basics of Neural Networks",
+    "section": "Additional resources",
+    "text": "Additional resources\n\nWhat is a neural network? Video\nGradient descent, how neural networks learn Video\nBackpropagation, how neural networks learn Video\n\n\n\n\n\nLLMs in Lingustic Research WiSe 2024/25"
   },
   {
     "objectID": "2024/homework/homework01.html",

sitemap.xml

@@ -2,42 +2,42 @@
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
     <loc>https://llm4linguists.xyz/2024/weeks/week02/slides.html</loc>
-    <lastmod>2024-10-16T09:15:14.722Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.912Z</lastmod>
   </url>
   <url>
     <loc>https://llm4linguists.xyz/2024/homework/homework01.html</loc>
-    <lastmod>2024-10-16T09:15:14.722Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.908Z</lastmod>
   </url>
   <url>
     <loc>https://llm4linguists.xyz/2024/weeks/week01/slides.html</loc>
-    <lastmod>2024-10-16T09:15:14.722Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.908Z</lastmod>
   </url>
   <url>
     <loc>https://llm4linguists.xyz/2024/syllabus.html</loc>
-    <lastmod>2024-10-16T09:15:14.722Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.908Z</lastmod>
   </url>
   <url>
     <loc>https://llm4linguists.xyz/2024/assignments.html</loc>
-    <lastmod>2024-10-16T09:15:14.722Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.908Z</lastmod>
   </url>
   <url>
     <loc>https://llm4linguists.xyz/index.html</loc>
-    <lastmod>2024-10-16T09:15:14.758Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.944Z</lastmod>
   </url>
   <url>
     <loc>https://llm4linguists.xyz/2024/homework.html</loc>
-    <lastmod>2024-10-16T09:15:14.722Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.908Z</lastmod>
   </url>
   <url>
     <loc>https://llm4linguists.xyz/2024/weeks/week01/page.html</loc>
-    <lastmod>2024-10-16T09:15:14.722Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.908Z</lastmod>
   </url>
   <url>
     <loc>https://llm4linguists.xyz/2024/weeks/week02/page.html</loc>
-    <lastmod>2024-10-16T09:15:14.722Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.912Z</lastmod>
   </url>
   <url>
     <loc>https://llm4linguists.xyz/2024/homework/homework02.html</loc>
-    <lastmod>2024-10-16T09:15:14.722Z</lastmod>
+    <lastmod>2024-10-16T12:09:05.908Z</lastmod>
   </url>
 </urlset>