Which one is faster? Analytical gradients or Automatic differentiation?

[Cfather]

Hi, I have a very general question about the computation time of the gradient of the robot kinematics and dynamics.
It is great that Pinocchio has implemented the analytical gradient and even the hessian of the robot dynamics.
But the recent advancement seems to be in favor of computing the gradient using automatic differentiation, mainly backed up by pytorch or jax, such as brax or IssacGym.
I wonder if people have done any experiments to actually compare the speed of these different approaches, especially in the case where we need to evaluate the gradients for a lot of samples in parallel? Which one would be faster?

[shubhamsingh91]

Hi @Cfather for the second order dynamics derivatives, we were able to prove that the analytical gradients are faster. Following paper shows the results:

https://ieeexplore.ieee.org/document/10449483

For the first-order derivatives, Pinocchio developers also showed that the analytical derivatives are faster. The comparison for some models can be found in
https://m.roboticsproceedings.org/rss14/p38.pdf

I can't find a comparison of compute time by exploiting brax, but I won't be surprised if they are more efficient. I believe the specific computes can be made efficient by exploiting parallelism, but that can done in both cases- Automatic-Diff, and analytical.
I would like to see a similar comparison too.

[jcarpent]

I could provide a benchmark, but the idea is that Pinocchio's gradients are almost 50x faster than auto diff gradients and more than 10x faster when considering code generation, and this is for small-size to medium-size robots.
The advantage of analytical derivatives becomes even greater when considering complex systems such as humanoids.

There is a paper between KU Leuven and my team at Inria considering analytical derivatives in the frame of auto diff frameworks such as CasADi where we highlight the benefits of analytical ones. Please see https://inria.hal.science/hal-03541487/file/Mixed%20Use%20of%20Analytical%20Derivatives%20and%20Algorithmic%20Differentiation%20for%20NMPC%20of%20Robot%20Manipulators.pdf.

[jcarpent]

I was able to run some benchmarks using CppAD and CppADCodeGen frameworks on a humanoid model.
Just below, these are the timings for CRBA

Calculate data.M (JSIM) with CRBA =		1.43409 us
Calculate data.M (JSIM) using CppAD =		20.1458 us
Calculate data.M (JSIM) using CppAD-jit =	11.0651 us
Calculate data.M (JSIM) using CppADCodeGen =	0.72905 us

and just below for the RNEA derivatives:

Calculate dtau_dx with computeRNEADerivatives =	5.01907 us
Calculate dtau_dx (∂RNEA) using CppAD =		1268.71 us
Calculate dtau_dx (∂RNEA) using CppAD-jit  =	155.271 us
Calculate dtau_dx (∂RNEA) using CppADCodeGen =	23.2511 us

It is definitely worth using analytical derivatives of Pinocchio for fast applications.

[jcarpent]

Another advantage of Pinocchio's derivatives and further extensions is that they operate on the right tangent space. In contrast, auto diff frameworks will tend to operate on the parametrization itself (e.g., differentiation Quaternion components, while we should instead use exponential mapping related to SO(3), the Special Orthogonal group of dimension 3 defining the group of 3d rotations).

[Cfather]

@jcarpent @shubhamsingh91 Thank you very much for your responses! I have read your work and they are amazing! I appreciate such detailed answers!
I wonder what would be the results of the comparison if we consider evaluating crba or rnea's derivates for a large number of samples in parallel? Pinocchio is implemented in C++ so it can only be parallelized in CPU. But recent advancements in MuJoCo or brax seem to achieve the batched evaluation of the robot dynamics, backed by pytorch or jax. I wonder if Pinocchio's analytical gradient implementation would still be superior.

[jcarpent]

We can parallelize everything, so it seems reasonable to say that this should remain competitive.
@Cfather Why don’t you give a try?

[Cfather]

I am currently busy with other things but I will definitely give it a shot to set the benchmark and the comparison.
Could you elaborate a little on what you mean by "we can parallelize everything"?

[jcarpent]

Modern CPUs have several cores that can be used to compute in parallel.
In Pinocchio, I've started to integrate OpenMP for parallel computations for ABA, RNEA and the collision detection.

[Cfather]

Just found a paper: https://arxiv.org/pdf/2109.06976