Cascaded Shadow Maps support for DirectionalLights added

[robertosfield]

Introduction to Shadows work

Hi All,

My main development goal over the last couple of months has been adding shadow support into the core VSG, today I merged the first major milestone of this work into VSG and vsgExamples master. The changes were merged with these PR's:

VulkanSceneGraph
vsgExampels

This work required significant work to the VSG to improve the ease and efficiency of creating render to texture effects, these were already merged and release in most recent VSG stables releases, so this latest merge of the Shadows branch builds on these and also acts a test case for them. The overall aim is to make it easier for end users to add advanced features to their applications, shadows being just one example.

To help with testing the shadow functionality I created a vsgshadow example, in this thread I'll use this example to help illustrate the capabilities of shadow functionality. Lets start with a single overhead vsg::DirectionalLight with the new uint32_t shadowMaps member set to 1, so just a single shadow map is asked for:

'vsgshadow'

The shadows are completely black at this point as the example by default just creates a single directional light, but we can add an vsg::AmbientLight so even areas in shade have some lighting:

'vsgshadow -n 2'

While the shadow looks relatively crisp in this screenshot if we zoom into the base we start to see pixelation, even with the default resolution of a 2048x2048 shadow map:

The title of this thread is Cascaded Shadow Maps so it's time to up the directionalLight->shadowMaps value to 2 using the --sm 2 on the command line (the -p saved_animation.vsgt is how I got exactly the same view as above):

'vsgshadow -n 2 -p saved_animation.vsgt --sm 2'

Under the hood the vsg::View's ViewDependentState class that previously was just collecting light data is now computing the light space for each of the cascaded shadow maps, setting up a nested vsg::CommandGraph to do the rendering to texture and then a RenderGraph/Framebuffer/View chain for each shadow map, traverses this nested CommandGraph to render the shadow maps, then passes the light space texture matrix along with the rest of the original light data so that the fragment shader can then apply the shadow maps. This nested CommandGraph is automatically rendered before the main 3d scene thanks to it's submitOrder being set to -1, so it's always rendered before the main 3d view's rendering.

To invoke all this work all you have to do as a user is set the light->shadowMaps number to a non zero value and everything will be done automatically for you. This isn't the whole story, which we'll expand on in the rest of this post, but at a basic level added shadows is as simple as setting the light's shadowMaps value.

I'll go into more detail in follow up posts in this thread.

[robertosfield]

Depth clamp

At present the set up of the light space extents for each shadow map is computed using the direction of the light and the view frustum, when a single shadow map is required the light space extents will cover the entirely view frustum, and can be pictured as box with it's z axis aligned to light direction, the light space x axis aligned along the XY plane, an Y axis perpendicular to this.

To compute the extents of this box the 8 corners of view frustum are transformed into this light space, this is done on each new frame so as the view frustum moves around the scene the extents automatically follow it, if the view frustum is small then lights space extents will be small and the effective resolution of the shadow map will be higher, there is one downside with this clamping of the light space/shadow map extents to the view frustum and this is clipping of the near plane of the light space/shadow map volume which happens when objects in the scene that should cast a shadow are clipped out before even making into the shadow maps depth buffer. Visually this clipping looks like:

What we see is the shadow of top of sphere is clipped out creating a hole in the shadow. As the present shadow implementation knows nothing about about the extents of the objects in the scene it has nothing to guide moving the near plane of the light space/shadow map to encompass the full extents of the scene. However, all is not lost as there is a feature in modern graphics hardware & Vulkan that we can enable to clamp the depth values to a 0 to 1 range, so that all fragments that would normally be discarded by the near clipping plane are clamped to 1.0 and are written to the shadow map.

The depth clipping feature can be enabled in the vsgshadow example using the --dc command line option:

'vsgshadow -n 2 -p saved_animation.vsgt --dc'

There are two blocks of code in vsgshadow that enable the depth clam feature, first we have to enable the DeivceFeature:

    bool depthClamp = arguments.read({"--dc", "--depthClamp"});
    if (depthClamp)
    {
        std::cout<<"Enabled depth clamp."<<std::endl;
        auto deviceFeatures = windowTraits->deviceFeatures = vsg::DeviceFeatures::create();
        deviceFeatures->get().samplerAnisotropy = VK_TRUE;
        deviceFeatures->get().depthClamp = VK_TRUE;
    }

The second part is modifying the vsg::RasterizationState that is used to set up the graphics pipeline, the example does this for both the Phong and BPR ShaderSet thus:

         if (depthClamp)
           {
               auto rasterizationState = vsg::RasterizationState::create();
               rasterizationState->depthClampEnable = VK_TRUE;
               phong->defaultGraphicsPipelineStates.push_back(rasterizationState);
           }

It's worth having a look at the whole set up, so look at the vsgshadow.cpp block from line 213 onwards.

I have mind automating more of the work required to setup up the vsg::DeviceFeatures required for a scene graph, but this will be separate work I'll need to do outside of the Shadows work specifically. One this functionality is part of the VSG we'll be able to leverage when setting up Shadows, but for now if you have a scene that could benefit from depth clamping the shadow map then you'll need to follow the setup done by vsgshadow.cpp.

[robertosfield]

Multiple lights with cascade shadow maps and PBR supported

So far we're just seen the results on a very simple model, with a single directional light and single ambient light, with just two cascaded shadow maps, and with a very simple model created using vsg::Builder - so leveraging the built-in PhongShaderSet. The design of cascaded shadow map support now in ViewDependentShadowMap and the associated shaders fully supports cascaded shado0w maps for multiple directional lights. Lets look at what happens when we add a 2nd direction light, colour yellow and pointing downward in the direction 0.9,1,-1:

'vsgshadow -n 3 --dc'

The code in vsgshadow.cpp for setting up these three lights is simple:

   vsg::ref_ptr<vsg::DirectionalLight> directionalLight;
    if (numLights >= 1)
    {
        directionalLight = vsg::DirectionalLight::create();
        directionalLight->name = "directional";
        directionalLight->color.set(1.0, 1.0, 1.0);
        directionalLight->intensity = 0.9;
        directionalLight->direction = direction;
        directionalLight->shadowMaps = numShadowMapsPerLight;
        group->addChild(directionalLight);
    }

    vsg::ref_ptr<vsg::AmbientLight> ambientLight;
    if (numLights >= 2)
    {
        ambientLight = vsg::AmbientLight::create();
        ambientLight->name = "ambient";
        ambientLight->color.set(1.0, 1.0, 1.0);
        ambientLight->intensity = 0.2;
        group->addChild(ambientLight);
    }

    if (numLights >= 3)
    {
        directionalLight->intensity = 0.7;
        ambientLight->intensity = 0.1;

        auto directionalLight2 = vsg::DirectionalLight::create();
        directionalLight2->name = "2nd directional";
        directionalLight2->color.set(1.0, 1.0, 0.0);
        directionalLight2->intensity = 0.7;
        directionalLight2->direction = vsg::normalize(vsg::vec3(0.9, 1.0, -1.0));
        directionalLight2->shadowMaps = numShadowMapsPerLight;
        group->addChild(directionalLight2);
    }

There isn't any extra setup required, the VSG during setup will traverse the scene graph with the vsg::CollectResourceRequirements and collate all the lights source and shadow map requirements. combine this with any further setting in the vsg::ResourceHnts, to help setup the vsg::View's ViewDependentState with the appropriately sized lightData and shadow map rendering backend.

This mechanism means that support for shadow maps will only be allocated when required, and allows users to also set up at start up the minimum and maximum capabilities which will be useful for applications that load subgraphs on the fly such as database paging new subgraphs that contain light sources that will need shadow maps.

There is much made of the complexity of Vulkan compared to OpenGL, and how "low level" it is, and up to recently the VulkanSceneGraph has made it easier to use Vulkan but still lagged behind the ease of use of OpenGL and with it OpenSceneGraph. This shadow maps functionality finally takes us past what the OpenSceneGraph is capable of, and how easy it is to setup and control.

Shadows have better performance, better visual quality and far easier to setup than what the osgShadow library is capable of. With osgShadow you are stuck with support for a single light source and a collection of shadow techniques, none of which produce really stable shadow quality.

The next party trick is that the new shadow mapping capability works with both the Phong and PBR ShaderSet, the means we can load PBR models such as this FlightHelmet.gltf using vsgXchange::assimp:

'vsgshadow -n 3 --sm 2 ~/Data/glTF-Sample-Models/2.0/FlightHelmet/glTF/FlightHelmet.gltf'

Note the two different shadows on the base and two different specular highlights on the top of the helmet.

Adding the support for cascaded shadow maps required no modifications to the vsgXchange::assimp loader, or vsg::Builder used for the simpler test models, this is all provided completely orthogonality to the scene graph creation side that just uses ShaderSets as a guide to help set up state. If one has been wondering why the VulkanSceneGraph has what seems like a complex approach to encapsulating shader setup, the value in decoupling rendering functionality from scene graph loaders makes adding advance features far easier.

If your application is using the built-in Phong or PBR ShaderSet then you'll get all this shadow map functionality essentially for free, all you need to do is set the Light::shadowMaps number to the number of cascades you need. If you use your own shaders and wish to leverage the built-in shadow mao functionality provided by ViewDependentState then you'll need to add shader code similar to what I added to the standard_phong.frag and standard_pbr.frag shaders.

The shader code in the standard_phong.frag and standard_pbr.frag looks like:

declarations:

layout(set = 1, binding = 0) uniform LightData
{
    vec4 values[2048];
} lightData;

layout(set = 1, binding = 2) uniform sampler2DArrayShadow shadowMaps;

Within the handling of each light source:

       // directional lights
        for(int i = 0; i<numDirectionalLights; ++i)
        {
            vec4 lightColor = lightData.values[index++];
            vec3 direction = -lightData.values[index++].xyz;
            vec4 shadowMapSettings = lightData.values[index++];

            float brightness = lightColor.a;

            // check shadow maps if required
            bool matched = false;
            while ((shadowMapSettings.r > 0.0 && brightness > brightnessCutoff) && !matched)
            {
                mat4 sm_matrix = mat4(lightData.values[index++],
                                      lightData.values[index++],
                                      lightData.values[index++],
                                      lightData.values[index++]);

                vec4 sm_tc = (sm_matrix) * vec4(eyePos, 1.0);

                if (sm_tc.x >= 0.0 && sm_tc.x <= 1.0 && sm_tc.y >= 0.0 && sm_tc.y <= 1.0 && sm_tc.z >= 0.0 /* && sm_tc.z <= 1.0*/)
                {
                    matched = true;

                    float coverage = texture(shadowMaps, vec4(sm_tc.st, shadowMapIndex, sm_tc.z)).r;
                    brightness *= (1.0-coverage);

#ifdef SHADOWMAP_DEBUG
                    if (shadowMapIndex==0) color = vec3(1.0, 0.0, 0.0);
                    else if (shadowMapIndex==1) color = vec3(0.0, 1.0, 0.0);
                    else if (shadowMapIndex==2) color = vec3(0.0, 0.0, 1.0);
                    else if (shadowMapIndex==3) color = vec3(1.0, 1.0, 0.0);
                    else if (shadowMapIndex==4) color = vec3(0.0, 1.0, 1.0);
                    else color = vec3(1.0, 1.0, 1.0);
#endif
                }

                ++shadowMapIndex;
                shadowMapSettings.r -= 1.0;
            }

            if (shadowMapSettings.r > 0.0)
            {
                // skip lightData and shadowMap entries for shadow maps that we haven't visited for this light
                // so subsequent light pointions are correct.
                index += 4 * int(shadowMapSettings.r);
                shadowMapIndex += int(shadowMapSettings.r);
            }

            // if light is too dim/shadowed to effect the rendering skip it
            if (brightness <= brightnessCutoff ) continue;

            float unclamped_LdotN = dot(direction, nd);

            float diff = max(unclamped_LdotN, 0.0);
            color.rgb += (diffuseColor.rgb * lightColor.rgb) * (diff * brightness);

            if (shininess > 0.0 && diff > 0.0)
            {
                vec3 halfDir = normalize(direction + vd);
                color.rgb += specularColor.rgb * (pow(max(dot(halfDir, nd), 0.0), shininess) * brightness);
            }
        }
    }

I have trimmed out a bit of the lightData processing for the sake of berevity, but when the time comes for you to add the full shader code to your own shaders you can have a look at the shaders in vsgExamples/data/shaders, and ask for any further clarifications required.

I may spin out some of the shadow mapping functionality into a dedicated shadow.frag shader that gets included by the main fragment shaders but as this is still early days for this functionality I'm happy to leave figuring these niceties out to a later date once the codebase has had more real-world testing.

[robertosfield]

Larger models & maxShadowDistance

Very simple models and single PBR models are OK as an introduction, but real-world applications have more serious demands, both performance and visual quality of the shadow maps have to remain good as the model complexity and extents increase.

To help with testing I added a --large model option to the vsgshadow example that creates a 1km by 1km scene with 1000 shapes - a mixture of spheres, boxes and capsules. For a distance it looks like:

'vsgshadow --large'

When we zoom in what used to be OK results when using just single shadow map, now the shadows are horrible pixelated:

Enabling a second cascaded shadow map:
'vsgshadow --large --sm 2 -p saved_animation.vsgt'

Visual quality is much better, but still sucks. Adding a 3 shadow map:
'vsgshadow --large --sm 3 -p saved_animation.vsgt'

Still not great, but there is trick we can use to shrink the extents of shadow maps to closer to the eye point, rather than going all the way out to the further point of the view frustum far plane, we can set the ViewDependentState's maximum shadowDistance using:

'view->viewDependentState->maxShadowDistance = maxShadowDistance'

In the vsgshadow example we can set this with the command line --sd distance parameter, as the scene is 1km by 1km then using 1000 is good place to start when constraining the shadow map sizes, this is what we see:

We can even use the maxShadowDistance to make the single shadow map look much better, but we need to pull in the maxShadowDistance all the way to 200m:
'vsgshadow --large --sm 1 -p saved_animation.vsgt --sd 200'

While the near shadows look fine, as you look further away he shadows suddenly stop. In normal applications we'd pick a maxShadowDistance that works well with our scenes, the further in your pull it the high fidelity the shadow maps will be, but high the chance that users will see an ugly cutoff.

Use of scene fog is a good way to help hide to point. Another is fade the shadow at the far ends of the shadow map to avoid the abrupt cut off. Neither are implemented with the current shaders but are both things I think would be useful to add in the future.

[robertosfield]

Whole earth models

The VulkanSceneGraph is written from the ground up to support large scenes who's extents can cover the whole earth and beyond, mostly folks will know about this in the context of database paging, for instance GIS applications through to terrestrial and space simulators.

One of the techniques the VSG and OSG use to handle really large coordinate systems associated with these scenes is to use doubles for all Camera and in scene graph Transform matrices, accumulating model view matrix in doubles before casting to floats before passing to Vulkan/OpenGL. When creating large paged databases we place a MatrixTransform above each tile so that the tile geometry has a local origin when the MatrixTransform::matrix places it in world coords.

When the record/cull traversal happens the view matrices values are combined with the values in the in scene graph transform and when the camera is near by a tile the offsets of the view and tile transforms largely cancel out before they are ever converted to floats and passed to the GPU. When the view is far away from a tile the offsets remain large but this is proportional to the distance so the divide by w done for each vertex in the vertex shader reduces any numerical precision issues so they are sub-pixel and you never see a jitter on screen.

For scenes that are lit and have shadows we also want to make sure that it also works seamlessly with large coordinate systems as well. Unfortunately almost all the literature you'll come across explaining how to do lighting and shadows for games type applications they show shaders converting points , normals and light direction into world coordinates and doing the lighting and shadow calculations in the world coordinate frame. There is a big problem with this approach - as soon as the world coordinate become large numerical precision tanks and those errors are no longer sub-pixel but start jitter all of the place randomly as the view point or objects move.

Those familiar with the VSG's Phong and PBR shaders may already know that they don't use world coordinates in the vertex or fragment shaders, data goes from local object coordinates in the vertex shader and then are transform into clip space directly or into eye space, there is never an intermediate conversion into world coordinates, so there is never a point where the numerical precision issues come into play.

The new shadow map set up code in vsg::ViewDependentState uses the same principles as used for the modelview matrix to keep precision in doubles for as as possible before finally converting to floats when passing data to the GPU. This means all view frustum, light space and shadow map extents are computed in doubles, on only when the shadow map tex coord matrix is passed to the GPU via the lightData uniform array does it finally get convert to floats.

The other part of the solution is that all lighting and shadow computations done in the fragment shader are done in eye space, but as I had already made sure the previous lighting code was all done in eye space no extra work is required beyond just the shadow mapping code itself.

What does this all mean in practice? It means out of the box you can place objects on the surface of the earth and zoom in from space down to the metres above the ground and see solid objects that have no numerical precision jitter in vertex positions, lighting or shadows. It just works.

For the following tests I place the 1km by 1km set of randomly placed spheres, boxes and capsules next to Buckingam Palace, lets zoom in:

Further so we can finally see our objects and shadows:

'vsgshadow --earth models/openstreetmap.vsgt --location 51.502 -0.14 0 --large --direction -1 0.5 0 --sd 2000 --dc --sm 3'

Sure's a goofy image and you can't appreciate that the shadows and lighting are jiggling around because it's a still so I'll need to record a video at some point to demonstrate it. In the meantime run the above command line and have a play.

[robertosfield]

The vsgshadow example also allows placement of loaded models, here I've placed the DamagedHelment.gltf at 100 normal scale outside Buckingham Palace and with 2 directional lights and an ambient light. Hope the King doesn't mind...

vsgshadow --earth models/openstreetmap.vsgt --location 51.502 -0.14 0 --scale 100 ~/Data/glTF-Sample-Models/2.0/DamagedHelmet/glTF/DamagedHelmet.gltf --direction -1 0.5 0 --sd 2000 --dc -n 3

What we have here is an OpenStreetMap paged database using the PhongShaderSet and the Helmet using the PBRShaderSet being loaded by the vsg::tile ReaderWriter and vsgXchange::assimp loaders, rendering with 2 directional lights each with 3 shadow maps.

For this scene I'm getting 206fps on my AMD5700G using it's integrated GPU, and rendered on a 3440x1440 display.

[robertosfield]

Here's the Sponza.gltf test model without shadows:

And then with cascaded shadow maps:

'vsgshadow ~/Data/glTF-Sample-Models/2.0/Sponza/glTF/Sponza.gltf -n 2 --sd 100 --dc --sb 0.001 -p saved_animation.vsgt --direction -0.1 -0.1 -1 --sm 3'

[robertosfield]

I have now got the Shadows work far enough along for wider testing. It's only been tested on examples of the sort shown above, the user community will have a much wider range of use cases and model types so wider testing will hopefully shake out bugs and help refine the implementation.

As discussed in this thread so far, there are features I would like to see added in the near term, although this might be for after the next stable release, given the significance of this new functionality it probably makes sense to bump the release to 1.1.0. The items I have in mind for future development are (in no particular order):

• Support for vsg::SpotLight
• Support for vsg::PoiintLight
• Percentange Close Soft Shadows
• Perspective Shadow Maps (as part of the existing Cascaded Shadow Maps)
• Use of per shadow map CommandGraph and mulit-threaded of recording the each shadow map
• Greater user control of extents of light space/shadow map
• User control over shadow map allocation when large numbers of lights are active
• Some for of Cellular scheme for handling large numbers of lights
• Restructuring of shadow shader functions to make it easier to reuse.
• DeviceFeature setup
• Fade out of shadow map at distance
• Use the View masks for specializing state used for 3d and shadow map rendering

We could all raise our hands and vote for all these items to magically done instantly by some clever shadow map elves in the dead of night, but reality is we'll need figure our which are the priorities and tackle them one by one.

While I have head my head down completing the Shadows work I have not tackle much in the way of review/testing of community submissions and reports of bugs, so I need to now spend some time catching up with this side of support. It's likely the Shadows work itself will kick up a few issues from the community that I'll need to attend to as well, so I expect I'll be busy juggling my time for the next few weeks.

Overall I'm chuffed how the Shadows work has finally coalesced into something useful and coherent with the rest of the VulkanSceneGraph design and implementation.

[rolandhill]

Hi Robert, I just wanted to thank you for this and all your other wonderful work. The shadow results look amazing and very easy to use. Thanks & regards, Roland

…

On Fri, 6 Oct 2023 at 02:48, Robert Osfield ***@***.***> wrote: I have now got the Shadows work far enough along for wider testing. It's only been tested on examples of the sort shown above, the user community will have a much wider range of use cases and model types so wider testing will hopefully shake out bugs and help refine the implementation. As discussed in this thread so far, there are features I would like to see added in the near term, although this might be for after the next stable release, given the significance of this new functionality it probably makes sense to bump the release to 1.1.0. The items I have in mind for future development are (in no particular order): - Support for vsg::SpotLight - Support for vsg::PoiintLight - Percentange Close Soft Shadows <https://developer.download.nvidia.com/shaderlibrary/docs/shadow_PCSS.pdf> - Use of per shadow map CommandGraph and mulit-threaded of recording the each shadow map - Greater user control of extents of light space/shadow map - User control over shadow map allocation when large numbers of lights are active - Some for of Cellular scheme for handling large numbers of lights - Restructuring of shadow shader functions to make it easier to reuse. - DeviceFeature setup - Fade out of shadow map at distance - Use the View masks for specializing state used for 3d and shadow map rendering We could all raise our hands and vote for all these items to magically done instantly by some clever shadow map elves in the dead of night, but reality is we'll need figure our which are the priorities and tackle them one by one. While I have head my head down completing the Shadows work I have not tackle much in the way of review/testing of community submissions and reports of bugs, so I need to now spend some time catching up with this side of support. It's likely the Shadows work itself will kick up a few issues from the community that I'll need to attend to as well, so I expect I'll be busy juggling my time for the next few weeks. Overall I'm chuffed how the Shadows work has finally coalesced into something useful and coherent with the rest of the VulkanSceneGraph design and implementation. — Reply to this email directly, view it on GitHub <#991 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPOEQ3FQG72BJ2JJ2PKJDLX53JFRAVCNFSM6AAAAAA5UETRIGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TEMBQGI3DQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** .com>

[robertosfield]

Shadow Map debugging/visualization

As an aid to development/debugging shadow maps I created a debug code path in the fragment shaders that colour the different shadow maps so we can see which shadow maps cover what areas. You can enable this is in the vsgshadow exmaple with the --shader-debug command line option:

'vsgshadow --large --shader-debug --sm 5 --sd 2000'

Add the OpenStreetMap whole earth database we can see how the shadow maps cover terrain and objects:

'vsgshadow --large --shader-debug --sm 5 --earth models/openstreetmap.vsgt --location 51.5 -0.14 0 --sd 10000'

The code is the shader leverages the #pragma(tic) shader composition support:

                if (sm_tc.x >= 0.0 && sm_tc.x <= 1.0 && sm_tc.y >= 0.0 && sm_tc.y <= 1.0 && sm_tc.z >= 0.0 /* && sm_tc.z <= 1.0*/)
                {
                    matched = true;

                    float coverage = texture(shadowMaps, vec4(sm_tc.st, shadowMapIndex, sm_tc.z)).r;
                    brightness *= (1.0-coverage);

#ifdef SHADOWMAP_DEBUG
                    if (shadowMapIndex==0) color = vec3(1.0, 0.0, 0.0);
                    else if (shadowMapIndex==1) color = vec3(0.0, 1.0, 0.0);
                    else if (shadowMapIndex==2) color = vec3(0.0, 0.0, 1.0);
                    else if (shadowMapIndex==3) color = vec3(1.0, 1.0, 0.0);
                    else if (shadowMapIndex==4) color = vec3(0.0, 1.0, 1.0);
                    else color = vec3(1.0, 1.0, 1.0);
#endif
                }

In the vsgshadow.cpp code we have to override the PhongShaderSet to inject the required SHADOWMAP_DEBUG define, the block includes enabling other features too:

        auto phong_vertexShader = vsg::read_cast<vsg::ShaderStage>("shaders/standard.vert", options);
        auto phong_fragShader = vsg::read_cast<vsg::ShaderStage>("shaders/standard_phong.frag", options);
        auto phong = vsg::createPhongShaderSet(options);
        if (phong && phong_vertexShader && phong_fragShader)
        {
            // replace shaders
            phong->stages.clear();
            phong->stages.push_back(phong_vertexShader);
            phong->stages.push_back(phong_fragShader);

            if (shaderDebug)
            {
                phong->optionalDefines.insert("SHADOWMAP_DEBUG");
                phong->defaultShaderHints = vsg::ShaderCompileSettings::create();
                phong->defaultShaderHints->defines.insert("SHADOWMAP_DEBUG");
            }

            if (depthClamp)
            {
                auto rasterizationState = vsg::RasterizationState::create();
                rasterizationState->depthClampEnable = VK_TRUE;
                phong->defaultGraphicsPipelineStates.push_back(rasterizationState);
            }

            // clear prebuilt variants
            phong->variants.clear();

            options->shaderSets["phong"] = phong;

            std::cout<<"Replaced phong shader"<<std::endl;
        }

[robertosfield]

Shadow map bias and z fighting

One of the challenges with shadow mapping is the numerical precision of rendering the depth map, rendering of the 3d scene and the lookup into the shadow map, while every attempt to maintain precision where possible this issue is still present and the VSG's shadow map implement tackles it like most other implementations - to bias the shadow map pushing it away from the light source so that surfaces don't self shadow.

This bias value is user controllable in vsgshadow/ViewDependentState, you can even set it to 0.0 to see what happens without it:

'vsgshadow --sb 0'

Programmatically it can be set by setting the ViewDependentState::shadowMapBias member, the defaults can be see alongside the maxShadowDistance and lamda values in the (ViewDependentState header](https://github.com/vsg-dev/VulkanSceneGraph/blob/master/include/vsg/state/ViewDependentState.h#L148) :

       // shadow map hints
        double maxShadowDistance = 1e8;
        double shadowMapBias = 0.005;
        double lambda = 0.5;

The default of 0.005 is one I've found to work reasonable well for a range of model types, you may find for your own usage you'll want to come up with a different value of shadowMapBias.

Lowering the shadowMapBias too low causes the acne effect seen above, while increasing it too far will mean the shadow is pushed too far away that it stops influencing objects it should and you get inccorect shadows - note only the top of the sphere shadow is seen, and slight move of the viewpoint and it disappears completely.

'vsgshadow --sb 0.1'

The value of shadowMapBias is in non dimensional coords of the light space/shadow map, if you enable multiple shadow maps then the extents of the nearer shadow maps is smaller so the bias becomes small in object coords, the depth resolution also improves at the same time so you should fine enabling cascaded shadow maps should improve both screen space resolution of the shadow as well as ability to avoid the z fighting precision issues.

[robertosfield]

I have found a bug in the shadow maps extents computation that was causing problems shadow disappearing for some light direction/view direction combinations. The fix is now checked into VSG master Shadows branches. The commit is: 30d8d4f.

[robertosfield]

I have just checked support for culling in Render to Texture traversals to use the parent View's projection and view matrix when doing LOD distance checks. This feature is required for techniques like shadow maps as you want the shadow to appear/disappear/change in sync with the objects casting the shadows that appear in the main 3D view. The changes are:

As LOD's and shadow maps are used together games, vis-sim and GIS apps this is crucial feature for a scene graph and is the last major infrastructure feature required for the VSG Render to Texture/Shadow support. There is still more we can add to shadowing support but the foundation is now all in place.

To enable this support for your own render to texture techniques when you create a nested View you set the FeatureMask to include the new INHERIT_VIEWPOINT enum as now done in the ViewDependentState.cpp.

The results can be see by vsgshadow (you'll need the latest vsgExamples master) by adding --lod command line parameter to tell the example to insert vsg::LOD above subgraphs it creates:

$ vsgshadow --earth models/openstreetmap.vsgt --location 51.502 -0.14 0 --large --sd 5000 --dc --sm 3 --lod

Further out so the smaller objects are now culled the vsg::LOD decorating them, note both the object and it's shadow disappear at the same time in all the shadow map cascade.

Further out still and most of the objects are now culled.

[AnyOldName3]

Something to note is that PCSS (and similar techniques that apply the same idea to variance shadow maps, exponential shadow maps and moment shadow maps) are incompatible with the depth clamp you've done to increase depth precision - you can't tell how far away the occluder is if its depth has been clamped onto the near plane, so can't tell how big the penumbra needs to be.

Also, it's fairly common that papers dealing with shadows say world space when they mean view space, and I've seen bits of maths that only work if the eye point is the origin, so it's genuinely specifically view space rather than just the two terms being used interchangeably. Realising this can save some bother when the eye's a long way from the world origin.

[robertosfield]

The depth clamp is currently used to avoid pulling the near plane beyond the extents of the view frustum in light space, which has the bonus of keeping depth precision higher. Each shadow map cascade also gets it's own depth range as the extents of it's portion of the view frustum has different extents in light space.

When the time comes to implement Percentage Closer Soft Shadows I will need to come up with a way of doing the initial occluder query in a way that handles the cascades having different depth ranges, or to change the cascades so they all use the same depth range and accept a lost in precision.

Occluders that are outside the light space depth range of view frustum will introduce a problem as the depth will be treated as no further away than the near plane of the shadow map cascade. This want cause the occluder to be missed entirely, just that it won't have the current position. Allowing the near plane to outside the view frustum and encompass the extents of any possible occlude would solve this, but have a depth precision penalty.

As for papers using world space and view space, I read the papers to get the jist of what they are doing then ignore the actual details of the implementation and maths and just derive the maths for what is required for the way the VSG works i.e. double matrix for Camera view matrix, double matrices for model matrices, accumulate into a single modelview matrix, transform all lighting/shadow related data into eye space so that the GPU does all the key computations local to eye coords to maximize precision where it counts.

This approach is what OpenGL was already doing this for lighting/eye linear texgen in the early nineties and works really well so not that novel, but alas it seems lots of papers on games, and games engines miss the importance of doing everything in eye space, it's like a missing nugget has been lost in time.

On the VSG shadow implementation side, I view what we have as a solid foundation, but we'll still need to keep building upon it to implement support for things like soft shadows, spot and point lights, improve visual quality and provide more user to control.