add char. polynomial to AR(1)

15_hierarchical_modelling.ipynb

@@ -556,12 +556,12 @@
         "\\end{align*}\n",
         "$$\n",
         "\n",
-        "We can rewrite this model as\n",
+        "Often it is more convenient to write models is a slightly different form:\n",
         "\n",
         "$$\n",
         "\\begin{align*}\n",
         "y_k \\sim \\mathcal{N}(\\mu_g + \\alpha_k, \\sigma^2)\\\\\n",
-        "\\alpha_k \\sim N(0, \\tau^2)\\\\\n",
+        "\\alpha_k \\sim \\mathcal{N}(0, \\tau^2)\\\\\n",
         "\\mu \\sim \\mathcal{N}(\\mu_0, v_0)\\\\\n",
         "\\tau^2 \\sim \\mathcal{IG}(t_1, t_2)\\\\\n",
         "\\end{align*}\n",
@@ -573,7 +573,7 @@
         "\n",
         "Random effect variance attributes a portion of uncertainty to a specific source. \n",
         "\n",
-        "The term <font color='orange'>random effects</font> is used to contrast the difference with <font color='orange'>fixed effects</font>\n",
+        "The term <font color='orange'>random effects</font> is used to contrast the difference with <font color='orange'>fixed effects</font> in the linear predictor:\n",
         "\n",
         "$$\n",
         "\\eta  = \\beta X + \\epsilon\n",

16_AR.ipynb

@@ -10,11 +10,11 @@
         "\n",
         "Time series data often exhibit patterns such as trends, seasonal variations, and irregular fluctuations, making them inherently complex to analyze. \n",
         "\n",
-        "There exist many statistical models of time series. For example, autoregressive integrated moving average (ARIMA), exponential smoothing methods, and state space models. These models allow for forecasting future values based on past observations.\n",
+        "There exist many statistical models of time series. For example, autoregressive integrated moving average (ARIMA), exponential smoothing methods, and state space models. These models allow to forecast future values based on past observations.\n",
         "\n",
         "ARIMA is an abbreviation for  <font color='orange'>autoregressive integrated moving average</font> and these models are what the tin says: they include autoregressive, differencing, and moving average components.\n",
         "\n",
-        "In this lecture we are interested looking at time series structure through the temporal random effects glass. We will consider a specific subclass of ARIMA family. Namely, the autoregressive models.\n",
+        "In this lecture we are interested in looking at time series structure through the temporal random effects lens. We will consider a specific subclass of ARIMA family. Namely, the autoregressive models.\n",
         "\n",
         "<font color='orange'>Autoregressive</font> models are statistical models that use a linear combination of past observations of a time series $x_{t-1}, x_{t-2},... $ to predict future values, where each observation is regressed on previous observations.\n",
         "\n",
@@ -23,19 +23,22 @@
         "### AR(1) definition\n",
         "Let us consider AR(1), or Autoregressive model of order 1. In an AR(1) model, each value in the time series $x_t$ is modeled as a linear combination of the previous observation $x_{t-1}$ and a random error term $\\epsilon_t$. Mathematically, an AR(1) process can be represented as\n",
         "\n",
-        "```{margin}\n",
-        "The condition $\\mid \\phi \\mid <1$ here is required for stationarity. It comes from the condition that for stationarity, all roots of the characteristic polynomial $\\lambda_i$ should satisfy $|\\lambda_i|<1.$\n",
-        "```\n",
-        "\n",
         "$$\n",
         "x_t = \\phi x_{t-1} + \\epsilon_t,\\\\\n",
         "\\mid \\phi \\mid <1,\\\\\n",
         "\\epsilon_t \\sim(0, \\sigma^2).\n",
         "$$\n",
         "\n",
-        "\n",
         "The innovations $\\epsilon_t$ are independent from each other.\n",
         "\n",
+        "The condition $\\mid \\phi \\mid <1$ here is required for stationarity. In order to ensure stationarity, we need to check all roots of the characterictic polynomial $\\lambda_i$ satisfy $|\\lambda_i|<1.$ If $B$ is the backshift operator, such that $B^k = X_{t-k}$, we can rewrite the AR(1) model as \n",
+        "\n",
+        "$$\n",
+        "x_t (1 - \\phi B) = \\epsilon_t.\n",
+        "$$\n",
+        "\n",
+        "The characteristic polynimial in this case takes the form $f(\\lambda) = 1 - \\phi \\lambda$. It roots are obtained by solving $1 - \\lambda = 0$. Hence, $\\lambda= \\frac{1}{\\phi}$.\n",
+        "\n",
         "In many modelling cases, the knowledge of this formulation will suffice for modelling. There is a lot of interesting maths realted to AR models, but we will look at that later. Right now, we are ready to implement AR(1).\n",
         "\n",
         "### AR(1) implementation"
@@ -136,7 +139,7 @@
         "Can we fit a Numpyro model to this data to estimate $\\phi$ and $\\sigma$?\n",
         "\n",
         "```{margin}\n",
-        "Remember plates? *Exercise*: reimplement this model with a loop. What do you observe?\n",
+        "Remember plates?\n",
         "```"
       ]
     },
@@ -422,7 +425,7 @@
         " \\end{align*}\n",
         "$$\n",
         "\n",
-        "`````{admonition} Task 20\n",
+        "`````{admonition} Task 24\n",
         ":class: tip\n",
         "\n",
         "- Continue from here and derive precision matrix of AR(2) for $T=5$.\n",
@@ -668,7 +671,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "`````{admonition} Task 21\n",
+        "`````{admonition} Task 24\n",
         ":class: tip\n",
         "\n",
         "- Implement the random walk model. You may find some inspiration [here](https://num.pyro.ai/en/latest/examples/mortality.html). \n",