How to dynamically load big scene with models loaded from custom format files?

[avilkinnick]

Hello, I have model files with format .dmd which is not supported neither by vsgconv nor assimp, so I made function that parses them and for each of them creates it's own GraphicsPipilineConfigurator, Commands, StateGroup and finally returns MatrixTransform with stateGroup as a child.
When I try to add to scene graph more than 500 of these models, it shows "Error: Failed to allocate DeviceMemory".
I had seen some discussions about PagedLOD and corresponding vsgExample, but I still don't understand how to implement it in vsg with custom format files

[robertosfield]

I'm not familiar with the DMD format. Do you have any links or examples of the data to give us an idea where you are coming from?

In general new loaders are best implemented by subclassing from vsg::ReaderWriter as is illustrates by the vsgXchange implementations such as assimp.

Do you use a third party library to load the files then convert to the VSG or do you parse the file format yourself? Is DMD something others might find useful as well? If so do you plan to open source your implementation?

When you load "500 of these models" what is the nature of them? Are they big models? Do they share textures etc?

The vsg::SharedObjects class provides a mechanism for sharing objects between multiple loads - you assign an instance of a vsg::SharedObjects to vsg::Options then the ReaderWriter can utilize this instance when creating graphics pipelines and textures through to whole files. The vsgXchange::assimp utilizes it so have a search for sharedObjects and recommend using it where possible.

Sharing as much possible helps lower the CPU and GPU memory requirements and reduces state changes so can have a big impact on performance.

[avilkinnick]

Oh, the thing is that we have released open-source project: "Russian railway simulator" (https://github.com/maisvendoo/RRS), which now uses OSG for 3d. All the graphics code was written by other person and I was given a task to switch to VSG.

Models represent railroads, trains, building and nature along the route. Most part of models have format .dmd that looks like this:
most.txt

It is quite unconvenient because it is practically unknown, unsupported by 3d libraries and it doesn't have any reference on texture (like .obj with .mtl). I thought about making convertator from .dmd to .obj but I was said that it will take very long time to convert all our models that are currently used.

Already finished parser of .dmd for OSG you can find here: https://github.com/maisvendoo/osgdb_dmd

Rail route definitely have many repeating instances so thanks for mention of vsg::SharedObjects. Will try it. But in any case, rail route by it's nature is large stretched series of models and current OSG implementation uses PagedLOD for their loading. But with VSG I can't understand how it works yet

[robertosfield]

Converting to .obj won't gain you anything.

I'd suggest just converting the existing dmd OSG loader to be a VSG equivalent.

vsg::PagedLOD and osg::PagedLOD are very similar in principle but the API varies a little, and LOD ranges in the VSG are a screen space ratio rathe than a distance like the OSG uses by default. The vsg::PagedLOD has just two children, while the osg::PagedLOD can have a variable number, but in practices they will almost always just have 2 children like the vsg::PagedLOD.

The vsg::DatabasePager is assigned automatically by the vsg::Viewer::compile() when your scene graph has vsg::PagedLOD in it so out of the box you will have paging support as long as the scene graph is setup correctly.

If you want to use a custom ReaderWriter with a paged format then all you need to do is assign your custom ReaderWriter to the vsg::Options as is done for vsgXchange::all in most of the vsgExamples, you can assign both the vsgXchange::all and your ReaderWriter. The when you load files that include PagedLOD this vsg::Options will be cloned along with the list of ReaderWriter to use.

Hopefully this will help you make more sense of the vsgpagedlod example etc.