How to train on ONCE-3DLanes without extrinsics?

[Aguin]

As mentioned in this issue, ONCE dataset doesn't release camera extrinsics.
How to train Persformer on ONCE and reproduce your latest results?

[dyfcalid]

We just roughly give a fixed camera height and pitch angle to train the model (e.g., 1.5m camera height and 0.5 degree pitch angle). Although the BEV images generated in this way are not good enough, the model is still able to achieve a well detection result.

[Aguin]

@dyfcalid thanks for your answer.
I trained the model on ONCE with 1.5 height and 0.5 pitch but got bad results. Did you also modify other configs such as anchors and bev ranges?

[dyfcalid]

We did not modify other configs for ONCE-3DLanes dataset. You may need to check the DataLoader to make sure the data is loaded and processed correctly. There are some differences in the data format between the ONCE dataset and the OpenLane dataset, including coordinate system definitions, etc. And when args.dataset_name='openlane', we have some special functions to handle it, so for ONCE-3DLanes dataset you will also need to modify the relevant part of the code at the same time.

[Aguin]

Thank you for your advice, but I'm still a bit confused. I found that OpenLane filters and resamples points before anchor generation. Should I also perform these steps on ONCE?
https://github.com/OpenPerceptionX/PersFormer_3DLane/blob/main/data/Load_Data.py#L1550

In addition, it seems that ONCE and OpenLane use different interpolation methods. Does that matter?
https://github.com/once-3dlanes/once_3dlanes_benchmark/blob/master/evaluation/eval_utils.py#L252

[dyfcalid]

Sorry for late reply. The answer to the first question is yes, in OpenLane you need to make sure that the order of the coordinate points is from smallest to largest according to the y-value, but ONCE is the opposite.

The OpenLane and ONCE datasets have different evaluation metrics and methods, and do differ in their data processing. But this does not affect the model training, and you can calculate the corresponding evaluation results respectively in two benchmarks.

[canteen-man]

这也太神奇了，如果可以随便设一个相机pitch角，也超过了深度估计的方法，那岂不是也可以算不需要pitch角了吗，直接用标定值就行了。推理的时候，直接用固定pitch，不管是否有颠簸这些。

[ilnehc]

@canteen-man Hi, the pitch angle and camera height here are solely used for the BEV transformation, which will not affect the evaluation. In another word, we can transform the 'incorrect' predicted 3D lane points in the world frame back to the camera frame with the 'wrong' pitch angle to compare with the ground truth. If you want accurate 3D points in the world, the calibrated extrinsic params are needed.

We did not ablate the influence of different fixed values of the pitch angle for extrinsic-free datasets such as ONCE. However, we believe a decent pitch angle in this case indeed can affect the difficulty for the model to regress.

[canteen-man]

@ilnehc ，感谢你的回复。但是我理解readme里CD error就是按照ONCE论文里的评价方法计算的吧。ONCE的数据集里的json保存的就是“正确”的世界坐标吧。

[ilnehc]

@canteen-man

1. Correct, it is under CD error metric with the evaluation code provided in ONCE dataset.
2. It is in the camera frame, rather than the world or vehicle coordinate. We refer you to Is there the camera extrinsics provided? once-3dlanes/once_3dlanes_benchmark#1, and the original words (Sec. 3.1 Lane representations) in their paper.

[canteen-man]

@canteen-man

1. Correct, it is under CD error metric with the evaluation code provided in ONCE dataset.
2. It is in the camera frame, rather than the world or vehicle coordinate. We refer you to Is there the camera extrinsics provided? once-3dlanes/once_3dlanes_benchmark#1, and the original words (Sec. 3.1 Lane representations) in their paper.

thank you。我之前错误的理解了你写的camera frame的意思，以为你说的是2D图像坐标了。
所以随便设一个pitch，还是能拿到正确的相机坐标系的点的。

[gggcy]

@ilnehc Hello，请问下，PersFormer在ONCE-3D的结果，是在3k+张验证集上得到的指标吗，因为我看ONCE-3D并没有公开8k+测试集的标注结果。

[ilnehc]

@gggcy Yes.

[zihanding819]

We just updated support for Persformer train & eval on Once dataset.