Camera drifts downwards despite being still

[martinakos]

I'm testing granite with a VIO dataset collected with realsense d435i. I've tested this dataset with basalt and I could track it without issues. But when I run the same dataset with granite (and using only the left camera) the camera seems to drift downwards from the beginning of the sequence despite it being still. The sequence starts with camera still and then a forward motion, which I presume is good enough for granite's monocular initialisation. If I start the sequence at the point the forward motion is happening the granite output pose follows the forward motion while drifting downwards as before. In both cases Granite eventually crashes with and error SO3::exp failed! omega: nan nan nan, real: nan, img: nan.

I'm doing these tests with a ros wrapper I made myself for both basalt and granite projects. This way I can feed the dataset from a rosbag and display the camera pose in rviz. The calibration and configuration files are essentially the same for basalt and granite, just adapting for a few different parameters in each project. I think I've setup the granite pipeline correctly, so I'll assume, for the moment, I could rule out my ros wrapper for granite as the cause of this downwards drift. In this case, would you have any advice for what could be the reason for this downwards drift? any configuration parameters that could be the cause? something related with gravity?

[martinakos]

Ok, I've realised what was the issue. I set up the Granite pipeline to use IMU but there is not monocular visual inertial initialisation/calibration stage of the IMU, and that's why get the pose drifts. If I set the use_imu parameter in VioEstimator to false then I can track the sequence. Although the scale is wrong and each time tracking is lost and restarted the scale is different.

I guess the question now is, Is support for IMU use going to be added to Granite?