Basic billboard functionality now available

[robertosfield]

Yesterday I checked into VulkanSceneGraph point rotation billboard support to the standard.vert shader so that the flat shaded, phong and pbr ShaderSet's now all have the ability to billboard so that geometries are aligned in screen space. I have also added support for setting up billboards in vsg::Builder class. The approach taken is the same as the billboard support provided by vsg::Text.

I have also updated vsgExamples, the shader.vert and vsgbuilder and vsgintersection examples.

With vsgbuilder add the --billboard command line option to tell it to billboard the geometries it creates, it works with instancing so you can create many billboard shapes if you want. For instance here's 100 billboard quads:

vsgbuilder --billboard -n 100 -i textures/lz.rgb

It scales quite nicely as geometry instancing is used and the billboard modelview computation is done entirely in vertex shader so no CPU overhead. With a million quads, on my AMD5700G (GPU integrated with CPU) I'm get 430fps for a small window, and 117fps for 1280x1024 - so it's fill limited thanks to the overdraw in this particular test.

I modified the vsgintersection example so that if you press the ctrl key and one of the shape insertion key then you can insert 'q' quad, 'b' box, 's' sphere, 'c' cone or 'p' capsule. This code path will also illustrate how you can use the keyPress.keyModifier to detect the ctrl key being pressed. I tried to capture what it looks like but you only really see the billboarding effect clearly when you rotate the view:

I have implemented the same distance based scaling as I did for vsg::Text so you can set the scaleDistance as a w value when passing the positions as a vec4Array, so for each instance you pass {x, y, z, scaleDistance}. For viewing distances further than the scaleDistance the scale is 1.0 so the object appears unscaled with it's size determined by it's object coordinate size alone, this means further than this distance the object just scales like other objects in the scene, shrinking in screen space as you zoom out.

For distances less than the scaleDistance the object is scaled down to maintain a constant size in screen space, if the value is zero then you'll never see this screen space scaling - it will always just behave like a normal object in the scene and grow bigger in screenspace the closer you get.

This scaling to screen space when you are near to an object is really useful for GIS applications where you have thousands of markers in a scene and won't to avoid the markers obscuring the scene when close, but also far away with many markers visible they scale down so less likely to overlap each other and the scene around them.

For GIS systems you'll still need to decorate the billboarded objects with a MatrixTransfom and place them underneath so they a local coordinate frame, with the transform placing them in final world/ECEF coorindates. The vsgitersection vsgbuilder examples don't attempt to do this so I'm consider creating an example to specifically illustrate how this is done and when it's so important for handling the numerical precision issues that can occur if you aren't careful.

[robertosfield]

Forgot to mention, I have only implemented billboard around a point so that objects remaining aligned to screen space, I haven't attempted to implement axial billboard. The maths is different for this case, and we'll need to pass in the axis or have separate code paths for the X, Y, Z axis.

There is some shader code osg2vsg for axial billboard which is how the lz.vsgt is able to have billboard trees, so we have some prior art that we could look at. At this point in time I just had client that needed point rotation billboards so that was my focus for the current work.

The other area that is not tackled is how we might tell loaders that we want to instance or billboard and loaded model. The loaders create all the state and mesh required for rendering so to post process a loaded model to add instancing or billboard requires the shaders to be replaced and meshes tweaked for the extra instance array, which is awkward to do. I'm thinking that perhaps we could assign our instance array to vsg::Options so that the loader can use it, something like:

auto positions = vsg::vec3Array(1000); 
// fill in positions
options->setObject("positions", positions);
auto model = vsg:read_cast<vsg::Node>("model.fbx", options);

I'll return to looking to see if this might be possible at a late date - I have other plates to keep spinning right now.

For those curious about the shader code that is doing the billboarding, it's nothing clever - all contained #ifdef VSG_BILLBOARD blocks in vsgExamples/data/shaders/standard.vert.

[robertosfield]

If you are using vsg::Builder and want to set up billboard all you need to do is set the vsg::StateInfo.biilboarng bool to true, and assign the vec4Array to geomInfo.positions this is what vsgbuillders billboard code path looks like:

            if (billboard)
            {
                stateInfo.billboard = true;

                float w = std::pow(float(numVertices), 0.33f) * 2.0f * vsg::length(geomInfo.dx);
                float scaleDistance = w*3.0;
                auto positions = vsg::vec4Array::create(numVertices);
                geomInfo.positions = positions;
                for (auto& v : *(positions))
                {
                    v.set(w * (float(std::rand()) / float(RAND_MAX) - 0.5f),
                        w * (float(std::rand()) / float(RAND_MAX) - 0.5f),
                        w * (float(std::rand()) / float(RAND_MAX) - 0.5f), scaleDistance);
                }

[lufriem]

With vsgbuilder add the --billboard command line option to tell it to billboard the geometries it creates, it works with instancing so you can create many billboard shapes if you want. For instance here's 100 billboard quads:

vsgbuilder --billboard -n 100 -i textures/lz.rgb

It scales quite nicely as geometry instancing is used and the billboard modelview computation is done entirely in vertex shader so no CPU overhead. With a million quads, on my AMD5700G (GPU integrated with CPU) I'm get 430fps for a small window, and 117fps for 1280x1024 - so it's fill limited thanks to the overdraw in this particular test.

FWIW I'm using a Nvidia Geforce RTX2070 (laptop version) and with a million quads (--billboard -n 1000000 -i textures/lz.rgb) it crawls along at 6 fps..

I modified the vsgintersection example so that if you press the ctrl key and one of the shape insertion key then you can insert 'q' quad, 'b' box, 's' sphere, 'c' cone or 'p' capsule. This code path will also illustrate how you can use the keyPress.keyModifier to detect the ctrl key being pressed. I tried to capture what it looks like but you only really see the billboarding effect clearly when you rotate the view:

Billboarding doesn't work at all for me in that example.. the first issue is that I noticed that, while the 'Numlock' is active, the 'if (keyPress.keyModifier == vsg::MODKEY_Control)' condition will never be true (shouldn't this be a bitwise AND check?)? It also never actually billboards, I think because of the code at the end of the IntersectionHandler::apply function

// switch off billboarding so other shapes aren't affected.

However if I comment that code the example will just crash a little after inserting the (now billboarded!) geometry..

[robertosfield]

The low performance with a million instances is could be due to high over-draw in this particular example when using such a large number of instances.

I have just tried toggling on the NumLock and find that the Control Key no longer selects the billboarding of the quads. This looks like a bug, though I'm deep in complex development topic right now so not in place to go investing debugging an example.

[lufriem]

I think the 'Num Lock' thing is trivial (since keyPress.keyModifier is a bitfield a bitwise check should be used). I'll create a PR for that

However, it seems to me, that at least on the Windows platform they keyboard handling of modifiers is broken, ie the keyPress.modifier flag is not set if a modifier+key combination is being pressed (so pressing just 'n' works fine (the key is being reported as being presseD), pressing just 'Ctrl' works fine (the modifier is being reported as being pressed), but 'ctrl+n' returns a keyPress even with the 'modifier' set to 0 - so the sample simply does not work correctly). I'll create a new issue for that (I looked a bit at the keyboard handling on the Windows platform in the vsg code but I'm not really sure I understand what's going on there).

[lufriem]

I think the 'Num Lock' thing is trivial (since keyPress.keyModifier is a bitfield a bitwise check should be used). I'll create a PR for that

See PR vsg-dev/vsgExamples#248

However, it seems to me, that at least on the Windows platform they keyboard handling of modifiers is broken, ie the keyPress.modifier flag is not set if a modifier+key combination is being pressed (so pressing just 'n' works fine (the key is being reported as being presseD), pressing just 'Ctrl' works fine (the modifier is being reported as being pressed), but 'ctrl+n' returns a keyPress even with the 'modifier' set to 0 - so the sample simply does not work correctly). I'll create a new issue for that (I looked a bit at the keyboard handling on the Windows platform in the vsg code but I'm not really sure I understand what's going on there).

See issue #919