Failed to allocate DeviceMemory and Crash

[newmyl]

I have convert a .osgb model to .vsgb. The .vsgb file is very large and its size is 5GB.
I have two computers. In computer A, the GPU is Quadra P2000 whose GPU memory is 5GB.
In computer B, the GPU is an integrated type and has no GPU memory at all.
When I loaded the model in computer A, it throwed and expction "Failed to allocate DeviceMemory" and the program crashed.
When I loaded the model in computer B, it loaded the model succesfully and performed very well.
I opened the task manager in computer A and found that the program had used up the whole GPU memory, but in computer A, the shared GPU memory is 32GB which has been used only about 4.5GB.
Computer B has no discrete GPU and the program used the shared GPU memory directly. The shared GPU memory in computer B is 32GB, so the model is loaded succesfully.
Is there any way to manually manage which memory the program uses?
The computer A's GPU is far better than the computer B's, but the model can be loaded on the computer B instead of the computer A. How can I avoid this situation?
Many thanks.
The two pictures below are the captures of the task manager.

[robertosfield]

How did you convert the file from .osgb to .vsgb? I presume osg2vsg.

How big is the .osgb?

What type of model is it?

As a general note, osg2vsg was written early during the development of the VulkanSceneGraph and needs to be rewritten to use modern VSG functionality. I would like to see this happen early this year. A rewrite might not address memory issues for your particular model.

[newmyl]

Yes. I use the osg2vsg to generate the .vsgb file. The .osgb file is over 4GB. It's a building model which contains about 160,000 elements.
Thanks for you reply. I will study the osg2vsg to see whether I can aply the modern functionality in it. Would you give me some tips?

[robertosfield]

The number of elements is probably not the cause of high memory usage but the amount of geometry and texture data associated with them.

The main change I'm wanting to see in osg2vsg is utilize the vsg::ShaderSet/GeometryPipelineConfigurator scheme that will open the door for extensible shader setup as well all the lighting and shadow map functionality that the VSG's standard PBR and Phong ShaderSet provide. These changes will mainly help for visual quality rather than lower memory footprint.

My best guess at this point is the VSG is trying to allocate all the vertex and texture data resident in GPU memory and it's running out on your Quadro. Vulkan allows to specify exactly where the memory you want to allocate will be allocated from, in the case of vertex and texture data the VSG is choosing to allocate that in GPU memory, but one possibility would be to have the VSG use hints to decide what classes of memory to allocate where, so perhaps for your model type allocating the vertex data on the CPU but visible to the GPU would be the solution.

In the case of OpenGL the application has less control over where memory is held during rendering, and the driver can decide to move data around if it's running out on GPU local memory. This doesn't happen with Vulkan as the responsibility is down to the application, or scene graph.

In the short term it may be possible to pack the vertex or texture data in a more compact form to fit within the constraints of the GPU memory - for instance compressed texture.

For really big models I would also recommend using PagedLOD, this applies the same to OSG and VSG, they both have similar functionality for managing paged databases. The advantage of paged databases is that the model size can scale to terabyte size but still run at 60fps on modest hardware.

[newmyl]

I will have a try as your suggestion. Thank you very much.

[newmyl]

During these days, I have tried to allocated the memory myself. It's really difficult. It is there some examples to show how to manually control over where memory is held? Which API should I use in VSG?

[robertosfield]

Unfortunately there isn't an example showing exactly what you need - you are working on the bleeding edge :-)

The GPU memory associated with vertex array/index data is managed by Vulkan with it's vkBuffer, with vkBufferInfo used when passing what part of buffer to use for a vertex array/index dta. The VSG maps these to vsg::Buffer and vsg::BufferInfo. For ease of use the VSG sets these up for you if you don't explicitly set them up yourself, so for most users they'll be able to use the VSG without having to think about these topics.

Where a vk/vsg::Buffer is allocated is determined by the vk/vsg::DeviceMemory that it's associated with, so to set up memory in CPU memory you'll need to allocate a vsg::DeviceMemory with the CPU memory hints, and then allocated vsg::Buffer from this.

The vsg::BufferInfo you can also set up yourself by setting assign the vertex/index arrays to the BufferInfo that you set up with your vsg::Buffer, then assign these to the VertexIndexDraw/VertexDraw/Geometry arrays and indices entries, the VertexIndexDraw::assignArrays(..) and assignIndices(..) illustrates how it does this BufferInfo setup for you. While these implementations just assign the data to the required BufferInfo they don't assign the vsg::Buffer, that's left to the compile traversal to fill in the blanks if not already assigned.

What should be possible to set up the BufferInfo with the array and the buffer then assign the BufferInfo associated with each array to the VertexIndexDraw::arrays/indices etc. Perhaps a custom scene graph traversal that us run before the viewer.compile() might be able to see all the BufferInfo.buffer entries that are still not assigned to a buffer and step in and assign it.

A quick search of vsgExamples for any code that explicitly sets up DeviceMemory shows the following hits:

$ grep -r "DeviceMemory::create" .
./examples/app/vsgscreenshot/vsgscreenshot.cpp:        auto deviceMemory = vsg::DeviceMemory::create(device, destinationImage->getMemoryRequirements(device->deviceID), VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
./examples/app/vsgheadless/vsgheadless.cpp:    auto deviceMemory = vsg::DeviceMemory::create(device, destinationImage->getMemoryRequirements(device->deviceID), VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
./examples/app/vsgoffscreenshot/vsgoffscreenshot.cpp:    auto deviceMemory = vsg::DeviceMemory::create(device, memReqs, memFlags);

These will be related to allocating memory of images rather than vertex arrays but this might help.

As we don't yet have an example illustrating what you are trying to do, and it's bleeding edge so might require some help with getting right, or perhaps even some help from the community to debug could I suggest that you write an example program that loads data, runs a visitor over it to allocates required memory for the arrays/BufferInfo in CPU memory. We can then include this as an example in vsgExamples and if even if doesn't work right away the community can help debug it. The end result would be an example that you can use in your application, and the rest of the community can also use it as reference.

[rolandhill]

Hi Robert, Regarding your comment about using PagedLOD, I'm assuming that there is no way of doing this automatically when you're loading an existing model and you would have to break it up manually (or write some code to do it). Cheers, Roland

…

On Tue, 16 Jan 2024 at 20:54, Robert Osfield ***@***.***> wrote: The number of elements is probably not the cause of high memory usage but the amount of geometry and texture data associated with them. The main change I'm wanting to see in osg2vsg is utilize the vsg::ShaderSet/GeometryPipelineConfigurator scheme that will open the door for extensible shader setup as well all the lighting and shadow map functionality that the VSG's standard PBR and Phong ShaderSet provide. These changes will mainly help for visual quality rather than lower memory footprint. My best guess at this point is the VSG is trying to allocate all the vertex and texture data resident in GPU memory and it's running out on your Quadro. Vulkan allows to specify exactly where the memory you want to allocate will be allocated from, in the case of vertex and texture data the VSG is choosing to allocate that in GPU memory, but one possibility would be to have the VSG use hints to decide what classes of memory to allocate where, so perhaps for your model type allocating the vertex data on the CPU but visible to the GPU would be the solution. In the case of OpenGL the application has less control over where memory is held during rendering, and the driver can decide to move data around if it's running out on GPU local memory. This doesn't happen with Vulkan as the responsibility is down to the application, or scene graph. In the short term it may be possible to pack the vertex or texture data in a more compact form to fit within the constraints of the GPU memory - for instance compressed texture. For really big models I would also recommend using PagedLOD, this applies the same to OSG and VSG, they both have similar functionality for managing paged databases. The advantage of paged databases is that the model size can scale to terabyte size but still run at 60fps on modest hardware. — Reply to this email directly, view it on GitHub <#1072 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPOEQ5MFXLZBVY5WLEHVUDYOZE3TAVCNFSM6AAAAABB4IOFHCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DCNBSGE3TI> . You are receiving this because you are subscribed to this thread.Message ID: <vsg-dev/VulkanSceneGraph/repo-discussions/1072/comments/8142174@ github.com>

[robertosfield]

The vsg::TileDatabase node and vsg::tile ReaderWriter combination creates vsg::PagedLOD on the fly, but this is for already tiled imagery.

For conventional 3d models neither the OSG and VSG have a mechanism for inserting PagedLOD. To do so would require the loader to be know about the load balancing requirements and how to insert LOD/PagedLOD where appropriate. If the model has LOD already then potentially these could be converted to PagedLOD and would be the easiest way to attempt such a loader.