2024-05-07-dpi-fsvi (#148)

* Fix typos in derivation of FSVI objective

* Touch

_posts/2024-05-07-dpi-fsvi.md

@@ -1232,13 +1232,12 @@ To achieve this, FSVI employs an implicit variational distribution $$\qof{[\w]}$
 
 $$
 \begin{aligned}
-&\Kale{\qof{[\W]}}{\pof{[\W] \given \Dany}} + \Hof{\Dany} \\
-&\quad = 
-\E{\qof{[\w]}}{-\log \pof{\Dany \given [\w]}} + \Kale{\qof{[\W]}}{\pof{[\W] \given \Dany}} \\
-&\quad = \E{\qof{\w}}{-\log \pof{\Dany \given \w}} + \Kale{\qof{[\W]}}{\pof{[\W] \given \Dany}} -  \Hof{\Dany} \\
+&\Hof{\Dany} \le  \Hof{\Dany} + \underbrace{\Kale{\qof{[\W]}}{\pof{[\W] \given \Dany}}}_{\ge 0} \\
+&\quad = \Hof{\Dany} + 
+\E{\qof{[\w]}}{-\log \pof{\Dany \given [\w]}} + \Kale{\qof{[\W]}}{\pof{[\W]}} - \Hof{\Dany}\\
+&\quad = \E{\qof{\w}}{-\log \pof{\Dany \given \w}} + \Kale{\qof{[\W]}}{\pof{[\W]}} \\
 &\quad = \sup_n \E{\qof{\w}}{-\log \pof{\Dany \given \w}} + \Kale{\qof{\Y_n... \given \x_n...}}{\pof{\Y_n... \given \x_n...}} \\
-&\quad \ge \E{\qof{\w}}{-\log \pof{\Dany \given \w}} + \Kale{\qof{\Y_n... \given \x_n...}}{\pof{\Y_n... \given \x_n...}} \quad \forall n \\
-&\quad \ge \Hof{\Dany}.
+&\quad \ge \E{\qof{\w}}{-\log \pof{\Dany \given \w}} + \Kale{\qof{\Y_n... \given \x_n...}}{\pof{\Y_n... \given \x_n...}} \quad \forall n.
 \end{aligned}
 $$
 
@@ -1311,4 +1310,3 @@ More than a bag of technical tricks, the DPI reveals itself to be a powerful con
 ---
 
 **Acknowledgements.** Many thanks to [Freddie Bickford Smith](https://fbickfordsmith.com/) for very helpful comments and feedback on this post and to [Tim Rudner](https://timrudner.com/) for additional pointers to relevant literature and feedback on the FSVI section in particular 🤗 GPT-4 and Claude 3 Opus were used for editing.
-