Occupancy grid for Ray Sampling in NeRFs

[peerschuett]

Hi,
thanks for you amazing work! I am trying to use your occupancy grid, that is explained in Section C.2 of your paper, for one of my models. But I am not sure how it exactly works. I understand the basic idea, that you want to skip empty cells and only consider occupied ones, but during the first training iterations the NeRF doesn't know what areas are occupied and what not, because the density values output from it are too small. If you would therefore update the grid immediately with the density values of this NeRF you wouldn't get any samples anymore, because the NeRF thinks that nearly everything is unoccupied (if I am not mistaken). How do you therefore decide when the network has enough knowledge?

[neilgogogo]

Hi, in the very beginning, the author firstly uses the mark_untrained_density_grid func to filter out the unseen grids, marking the unseen grid as -1 and seen grid 0.

And when the iteration is 0 at the very beginning, the mlp_output is 0, but after exponential activation, the mlp is 1. Then the grid_out, which is density_grid_tmp is scalbnf(MIN_CONE_STEPSIZE(), level) (I cannot understand the __float_as_uint operation in the code).

Then in the ema_grid_samples_nerf function, the density_grid_tmp value will be assign to density_grid=0 part, and density_grid=-1 part will be unchanged.

So perhaps because of the exponential activation, the mlp is 1 although the mlp_out=0.

[Tom94]

Hi there, there exists a safeguard against zero-everywhere occupancy here:

instant-ngp/src/testbed_nerf.cu

Line 562 in 8625c25

float thresh = std::min(NERF_MIN_OPTICAL_THICKNESS(), *mean_density_ptr);

The threshold for setting a cell as occupied may never be larger than the mean density.