Added a post about LQR

_posts/2023-11-26-reference-input-tracking.md

@@ -1,6 +1,6 @@
 ---
 title: Reference Input Tracking
-date: 2023-11-23
+date: 2023-11-26
 categories: [Mechanics, Control]
 tags:
  [

_posts/2023-11-27-lqr.md

@@ -0,0 +1,115 @@
+---
+title: Linear-Quadratic Regulators
+date: 2023-11-27
+categories: [Mechanics, Control]
+tags:
+ [
+ study,
+ mechanics,
+ dynamics,
+ mathematics,
+ linear systems,
+ control,
+ state space,
+ state feedback,
+ feedforward control,
+ optimal control,
+ ]
+math: true
+toc: true
+---
+
+<!-- prettier-ignore -->
+* TOC
+{:toc}
+
+## But where should I place my poles?
+
+In [my previous post]({% post_url 2023-11-21-controllability-and-stabilisability %}) about
+controllability, I mentioned that state feedback allows us to place our poles arbitrarily in theory.
+However, in practice we are often limited by how powerful our control input is. Therefore, the
+question naturally arises as to how we can manage this tradeoff to appropriately choose where to
+place the closed-loop poles for a system. A somewhat natural approach could be to define and
+minimise a cost function that encompasses how our state deviates from the steady state \(at 0
+wihtout a reference\) and how much the control input deviates from 0 as well. A Linear-Quadratric
+Regulator \(LQR\) is designed to do exactly this by choosing the optimal _linear_ state feedback
+regulator $\mathbf{K}$ \(so that $\mathbf{u = -Kx}$\) which minimises the quadratic loss function
+
+$$
+J(\mathbf{u}) = \int_0^T \left( \mathbf{x^T Q x + u^T R u} \right) dt + \mathbf{x^T Q_f x}
+$$
+
+or for discrete systems
+
+$$
+J(\mathbf{u}) = \sum_{k=0}^N \left( \mathbf{x}[k]^T \mathbf{Q x}[k] + \mathbf{u}[k]^T \mathbf{R u}[k] \right) + \mathbf{x}[N]^T \mathbf{Q_f x}[N]
+$$
+
+Commonly, the problem is solved for an infinite time horizon meaning $T$ or $N$ are taken to
+infinity.
+
+## Choosing Costs
+
+Haven't we just moved the problem of tuning $n$ poles to even more tunable parameters in these cost
+matrices? How is this better?
+
+Well, tuning poles directly is difficult as it is hard to anticipate exactly what moving each one
+will change in the system response. By contrast, the cost matrices $\mathbf{Q,\, R}$ define
+quadratic costs for state and control effort over time, respectively which often reflect the design
+requirements for a system much more clearly. For example, if the system is powered by a battery and
+the input magnitude is proportional to the power drawn from it then $\mathbf{Q}$ would define a cost
+for the energy expended in order to regulate the system's state to 0. If recharging/replacing the
+battery was difficult or costly and regulating the state could be allowed to happen slowly then it
+would be appropriate to assign costs relatively low for $\mathbf{Q}$ compared to $\mathbf{R}$. On
+the other hand, if energy was not an important consideration and power could be drawn at a very high
+rate then it might be appropriate to set the cost for $\mathbf{Q}$ relatively higher than for
+$\mathbf{R}$.
+
+It should also be obvious that for effective regulation $\mathbf{R} > 0$ since if it were not then
+minimising the cost would result in an unbounded control input. Similarly, $\mathbf{Q} \geq 0$ and
+$\mathbf{Q_f} > 0$ so that the system actually regulates to zero \(or the desired set point\).
+Additionally, all of the cost matrices must be
+[positive \(semi-\) definite](https://en.wikipedia.org/wiki/Definite_matrix). As a result, typically
+these cost matrices are chosen to be diagonal to make them more intuitive to tune. This way, each
+element only affects the cost of one input or state and is therefore easier to reason about and
+adjust as necessary. For example, if the first input $u_1$ is spiking higher than physically
+possible leading to ineffective control, decreasing $r_{(1, 1)}$ may help by allowing it to expend
+more energy over a longer period of time. \(In reality it is a little more complex than this since
+lowering this cost may also increase the relative cost of one of the states it is responsible for
+controlling thereby requiring an increased input regardless but it is still easier to reason about
+than placing poles directly\).
+
+## But how can we actually solve this?
+
+Surprisingly, it turns out the solution can be found by solving what is known as an Algebraic
+Ricatti Equation \(ARE\) which is, as the name suggests, a linear algebra problem. You can read
+about the derivation [here](http://maecourses.ucsd.edu/~mdeolive/mae280b/lecture/lecture2.pdf) for
+the continuous case. The result is that for an infinite time horizon \(as is usualkly considered\)
+where the control law is $\mathbf{u = -Kx}
+
+$$
+\mathbf{K = R^{-1} B^T P}
+$$
+
+where $\mathbf{P}$ is the solution to the ARE
+
+$$
+0 = \mathbf{A^T P + PA - PBR^{-1}B^TP + Q}
+$$
+
+and for discrete cases,
+
+$$
+\mathbf{K = (R + B^T P B)^{-1} B^T PA}
+$$
+
+where $\mathbf{P}$ is the solution to
+
+$$
+\mathbf{P = Q + A^T P A - A^T P B(R + B^T P B)^{-1} B^T P A}
+$$
+
+Practically, this can be easily solved with software using `MATLAB` or `Python` by just providing
+the system matrices $\mathbf{A,\, B}$ and cost matrices $\mathbf{Q,\, R}$. Sometimes, the cost
+function is extended to include a third term with cost $2\mathbf{x^T N u}$ to allow for cross
+weights but often this is not needed.