-
Notifications
You must be signed in to change notification settings - Fork 137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
having the error "Not enough features or parallax; Move device around" when running VINS Mono+IMU on tx2 #6
Comments
After debugging in Estimator::relativePose function I have figured out that I have an average parallax of "nan", and then I printed my "corres" vector's size and I saw reasonable values in between 25-100+. Then I have printed out every parallax in the loop, when Estimator::relativePose is calculating the sum_parallax (which is used to calculate average_parallax). I do not know why but while most of the parallax'es in the sum has reasonable values (comparable with the euroc datasets') I have a few of them as "nan"s and that cause all the trouble and result in a nan average. What can be the reason having a "nan" parallax inside the corres? thank you for your help, Sincerely Yours, |
I have solved the problem finally - I have re-calibrated my camera using a KANNALA-BRANDT model instead of fisheye-mei model. Reading this HKUST-Aerial-Robotics/VINS-Fusion#57 inspired me, I believe there are more people having this problem but there is no answer or suggestion from the original authors. I've done the intrinsic calibration of camera from scratch with kannala-brandt camera-model option, then I have used the resulting new yaml. I was using estimate_extrinsic property of the VINS-Fusion previously (with manually calculated initial guess for body_T_cam0 matrix), I kept doing so. I did not change anything in intrinsics of IMU. Before changing camera model, setting estimate_extrinsic=0 or =1 did not help system work. |
Thank you for your comments, which helped me solve the problem.I also had an infinite problem with the results. When I re-calibrated the camera using the kannala-brandt model, I was able to output the pose, but it was not correct.Now, the body_T_cam0 matrix is wrong, what does this matrix mean, the parameters of the imu and the camera?How to calibrate it? |
Hey @tianxinyu0509 , sorry for the late reply. I hope this helps, |
Hey @mmp52. Thanks for your detailed reply, it is a good idea to use the results of vins-fusion optimization as the initial value.I'll give it a try. |
Hello, friend @mmp52 , I have some additional questions about GPU acceleration that I want to discuss with you.Are you running code on TX2 in real time?I tested it on the TX1 and desktop (the graphics card is nvidia GT730) and Shared some data with you. I also tested it on the desktop (i5-7300,GT730), and the results are as follows:
|
Sorry, there is something wrong with the layout of the table. |
Hello dear @tianxinyu0509 , I have not succeeded to make it work on TX2 in real time, I am still trying to do so, but I completely believe that CPU capacity limits the potential of TX2 - which means GPU acceleration does not accelerate much. As soon as I make it work on my new setup (I have rebuilt everything on a new xavier) I will write down to you. Meanwhile if you find a way to make it faster please let me know :) |
Dear friend @mmp52 , my thoughts are as follows and I hope to be of some help. |
Thank you very much @tianxinyu0509 , one more question: |
@mmp52 I'm going to do it, and if there's any progress, I'll share it here.Thank you very much. |
Hello, friend. I have a result to tell you.I found a big pitfall: the GPU version of VINS behaves differently than the CPU version.I recorded a dataset with a camera and tested it with two different versions of VINS, and found that the GPU version's trajectory performed worse.Reasons should be CV: : cuda: : createGoodFeaturesToTrackDetector and CV: : goodFeaturesToTrack result is different.The results are disappointing: we get worse results using gpu. |
@mmp52, I'm sorry that I haven't replied to your question about IMU for a long time. I've only recently finished adding an imu. There are some things to note here, but the first thing to note is the units of the imu, the imu data that goes into the code has to be in international units: g/m2,rad/s. Second, imu will output 9.8m/s2 when it is stationary. There's a function in the code that deals with gravity. |
@tianxinyu0509 , Thank you for your answers friend. After a few iterations of estimating extrinsic parameters (body_T_cam) I have decided on a fixed value matrix, I give that one as an input to the system each time with estimate_extrinsic:1 and it gives similar results each time. Maybe you have a wrong axis configuration, it took me long time to find correct transformation matrix because I was wrong on IMU axis, I have confused +/- values on IMU. |
@mmp52 ,Thank you for your reply.I can make sure that my imu axis is correct.The optimized body_T_cam matrix, do you add "printf” or "cout" in the code to show it? |
@tianxinyu0509 I have directly used/copied the comments on the thread in which I run "vins_node", VINS automatically prints you the required correction matrix , if that is pretty close to the identity matrix that means it did not change much when it is esimating extrinsic values when it started from your iteration. I have previously used cout to print matrices but I remember it requires iteration of elements. Also, as you've written VINS writes down resulting matrix to output folder, I believe that can also be used, also commented matrix on the thread happens more or less 10 seconds after start. |
@tianxinyu0509 Hello friend, do you know how can I check if there is a lag in the algorithm when I run it on real time? also did you solve the problem of transformation matrix ? |
@mmp52 In the code rosnodetest.cpp, the author uses the buffer :img0_buf,imu_buf to store the received image data and imu data.So low CPU performance can lead to algorithms that are not real-time.You can play a .bag dataset.If the data set is finished and the algorithm is still running, then the delay can be determined. |
@tianxinyu0509 Thank you for your answer |
Hello,
I am having the error "Not enough features or parallax; Move device around" when running VINS Mono+IMU on tx2. I have a 1280*720 camera. I have an initial guess of the transformation matrix, even though I try the algorithm with estimate_extrinsic = 0 , 1 or 2 none of them works. I have tried to increase the feature number to 400, it also did not work. Moving the device was also useless for me. I have checked the IMU signal and visualized camera track, camera seems to detect the features well and IMU is working at around 50 hz, increasing it up to 80 is also useless.
This message is sent to ROS_INFO @566th line of the estimator.cpp of the code, I got that the problem is with initialization process but I cannot figure out how to solve it.
Also I have realized that in your rosNodeTest.cpp code, feature_callback function feeds an estimator object with estimator.inputFeature(t,feature) but /feature_tracker/feature topic has no advertiser, I don't think this is the cause of the problem because this topic was also void when I ran euroc examples.
Finally, is just publishin IMU with linear accelerations and angular velocity enough? I could not figure out that should I pre integrate imu and publish it in another topic just for initalization process or not.
Thanks for your help
Metin
The text was updated successfully, but these errors were encountered: