Fusion camera and radar

[xiaohedu]

Hi Junsheng, Thanks for your contributions. I'm a postgraduate in Chongqing university. Recently,we want to do some research in fusing radar and camera data for multi-objects tracking. How can we modify your project to fuse camera and radar or recommend some information, Looking forward to your advice. Thanks a lot

[JunshengFu]

Hi @xiaohedu

If I were you I would analysis these two questions first.

1. what motion model to choose for the objects need to be tracked?
   For my Extended Kalman Filter project, I use CV (Constant Velocity); For my Unscented Kalman Filter, I use CTRV (Constant Turn Rate and Veloctity). But it totally depends on the characters of your targets.

2. What type of Kalman Filter variance to use?
   Extended Kalman Filter uses the Jacobian matrix to linearize non-linear functions; Unscented Kalman Filter, instead, uses sigma points to handle non-linear functions. The math is a bit different and it also depends on your needs.

Once you figure these out, you may want to substitute some of the equations in my code for your model and filter. Good luck!

[YangSiri]

Great suggestion! @JunshengFu
May i ask a question about multi-targets tracking by 3d laser sanner (VLP-16) ? Can i just use KF ? or your EKF presented here is a better choice ? I found it not accurate estimating velocity of pedestrians by KF.
Looking forward to your advice, thank you.

[yl115615]

Hi Junsheng, Thanks for your contributions. I'm doing research on target tracking. The data you provided has got the location of the target. How do you achieve target tracking using lidar and radar alone? Looking forward to your reply. Thanks a lot.

[ghost]

@JunshengFu you appear to be predicting based on one observation at a time:
https://github.com/JunshengFu/tracking-with-Extended-Kalman-Filter/blob/master/src/FusionEKF.cpp#L80-L101

Wouldn't an improvement be to simultaneously include both radar and laser observations? Am I missing something, or is your approach alternating between the two observations rather than fusing them?