Hessian Vector Products

[syclik]

Can we compute the Hessian? Can we compute a Hessian Vector Product?

What's the interface to do this?

Request from @rgiordan.

[syclik]

Do we need gradients w.r.t. the hessian vector product? (I'm guessing no)
Do we need the different versions of evaluating the log prob? (I'm guessing yes)

[rgiordan]

• Do we need gradients w.r.t. the hessian vector product? (I'm guessing no)
  The ideal of course would be arbitrarily high-order combinations of forward and reverse mode derivatives. :) But for my current research at least, I'm prevented from using Stan mostly because of the lack of Hessians and HVPs only. So the answer to your question is a soft no.

• Do we need the different versions of evaluating the log prob? (I'm guessing yes)
  I assume you mean with and without the Jacobian, and the answer to your question is yes. Again in an ideal world it might also be with and without the Jacobians of the unconstraining parameterization, but that is possible if tedious to extract by creating a new model without the variable constraints.

[rgiordan]

Possibly relevant, with associated PRs and commentary from the Stan developers:

stan-dev/rstan#587