Rendering on the GPU on Linux, not progressing

[plbpietrz]

Hi, I'm on Fedora 40, with the latest available kernel (6.11.4) and Nvidia drivers (560.35.03) for RTX4090.

I'm building pbrt using a docker environment based on Ubuntu 22, CUDA 12.1, Optix 7.1 link. The application builds without any problems, and running pbrt outside of the container in CPU mode works flawlessly. The problem is that when ran on the GPU, the renderer doesn't show any progress and spins on one cpu core 100%.

Any ideas on what could be the problem?

[mmp]

No ideas offhand. What is the output if you run pbrt --gpu --log-level verbose? Also, can you get a stack trace of where it's spinning?

[plbpietrz]

Logs stop here:

...
[ tid 113862 @     2.975s src/pbrt/gpu/optix/aggregate.cpp:1219 ] Starting OptiX initialization
[ tid 113862 @     3.017s src/pbrt/gpu/optix/aggregate.cpp:1233 ] Optix version 7.7.0 successfully initialized
[ tid 113862 @     3.028s src/pbrt/gpu/optix/aggregate.cpp:1308 ] Warning: use of OPTIX_EXCEPTION_FLAG_DEBUG is deprecated. Use OPTIX_DEVICE_CONTEXT_VALIDATION_MODE_ALL instead.
Info: Pipeline statistics
        module(s)                            :     1
        entry function(s)                    :    19
        trace call(s)                        :     4
        continuation callable call(s)        :     0
        direct callable call(s)              :     0
        basic block(s) in entry functions    :  3560
        instruction(s) in entry functions    : 58388
        non-entry function(s)                :     0
        basic block(s) in non-entry functions:     0
        instruction(s) in non-entry functions:     0
        debug information                    :    no

[ tid 113862 @     3.028s src/pbrt/gpu/optix/aggregate.cpp:1364 ] Finished OptiX initialization
[ tid 113862 @     3.028s src/pbrt/gpu/optix/aggregate.cpp:1376 ] Starting to create shapes and acceleration structures
[ tid 113862 @     3.078s src/pbrt/gpu/optix/aggregate.cpp:1384 ] Starting to read PLY meshes
[ tid 113862 @     3.079s src/pbrt/gpu/optix/aggregate.cpp:1387 ] Finished reading PLY meshes
[ tid 113862 @     3.079s src/pbrt/gpu/optix/aggregate.cpp:1434 ] Starting to create IASes for 0 instance definitions
[ tid 113862 @     3.079s src/pbrt/gpu/optix/aggregate.cpp:1493 ] Finished creating IASes for instance definitions
[ tid 113862 @     3.079s src/pbrt/gpu/optix/aggregate.cpp:1497 ] Starting to create 0 OptixInstances
[ tid 113862 @     3.079s src/pbrt/gpu/optix/aggregate.cpp:1537 ] Starting to consume top-level GAS futures

GDB generated backtrace:

#0  0x00007fd9c2aa2da9 in __futex_abstimed_wait_common64 (private=0, futex_word=0x56112aed49c8, expected=0, op=393, abstime=0x0, cancel=true) at futex-internal.c:57
#1  __futex_abstimed_wait_common (futex_word=futex_word@entry=0x56112aed49c8, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0, cancel=cancel@entry=true)
    at futex-internal.c:87
#2  0x00007fd9c2aa2e2f in __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x56112aed49c8, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0)
    at futex-internal.c:139
#3  0x00007fd9c2aa57f9 in __pthread_cond_wait_common (cond=0x56112aed49a0, mutex=<optimized out>, clockid=0, abstime=0x0) at pthread_cond_wait.c:503
#4  ___pthread_cond_wait (cond=0x56112aed49a0, mutex=<optimized out>) at pthread_cond_wait.c:618
#5  0x00007fd9c2cdd700 in __gthread_cond_wait (__cond=<optimized out>, __mutex=<optimized out>)
    at /usr/src/debug/gcc-14.2.1-3.fc40.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/x86_64-redhat-linux/bits/gthr-default.h:878
#6  std::__condvar::wait (this=<optimized out>, __m=...) at /usr/src/debug/gcc-14.2.1-3.fc40.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/std_mutex.h:171
#7  std::condition_variable::wait (this=<optimized out>, __lock=...) at ../../../../../libstdc++-v3/src/c++11/condition_variable.cc:41
#8  0x00005610ed45442b in pbrt::OptiXAggregate::OptiXAggregate(pbrt::BasicScene const&, pbrt::CUDATrackedMemoryResource*, pbrt::NamedTextures&, std::map<int, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Medium> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::vector<pbrt::Material, std::allocator<pbrt::Material> > const&) ()
#9  0x00005610ed3b9a3c in pbrt::WavefrontPathIntegrator::WavefrontPathIntegrator(pstd::pmr::memory_resource*, pbrt::BasicScene&) ()
#10 0x00005610ed1e268b in pbrt::RenderWavefront(pbrt::BasicScene&) ()
#11 0x00005610ecf00079 in main ()

[plbpietrz]

There is this one thread that hass some Optix related stuff in it:

#0  0x00007fd9c33ae861 in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  #1  0x00007fd9c315c4a3 in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  
#2  0x00007fd9c34be30f in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  #3  0x00007fd9c34be6a5 in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  
#4  0x00007fd9c317168c in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  
#5  0x00007fd9c34bce48 in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  
#6  0x00007fd9c3364305 in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  #7  0x00007fd9c3212ab9 in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  
#8  0x00007fd9c3157f14 in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  
#9  0x00007fd83291f01e in ?? () from /lib64/libnvoptix.so.1                                                                                                                                                               
#10 0x00007fd83291c983 in ?? () from /lib64/libnvoptix.so.1                                                                                                                                                               
#11 0x00007fd83287d4a4 in ?? () from /lib64/libnvoptix.so.1                                                                                                                                                               
#12 0x00007fd83286e3c0 in ?? () from /lib64/libnvoptix.so.1                                                                                                                                                               
#13 0x00005610ed44dffa in pbrt::OptiXAggregate::buildOptixBVH(OptixDeviceContext_t*, std::vector<OptixBuildInput, std::allocator<OptixBuildInput> > const&, pbrt::ThreadLocal<CUstream_st*>&) ()                          
#14 0x00005610ed44feb5 in pbrt::OptiXAggregate::buildBVHForBLPs(std::vector<pbrt::ShapeSceneEntity, std::allocator<pbrt::ShapeSceneEntity> > const&, OptixDeviceContext_t*, OptixProgramGroup_t* const&, OptixProgramGroup
_t* const&, OptixProgramGroup_t* const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::FloatTexture, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, 
std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::FloatTexture> > > const&, std::map<std::__cxx11::basic_string<char, std::c
har_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::vector<pbrt::Material, std::allocator<pbrt::Material> > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::alloc
ator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Medium> > > const&, std::map<int, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr
::polymorphic_allocator<pbrt::Light> >*> > > const&, pbrt::ThreadLocal<pstd::pmr::polymorphic_allocator<std::byte> >&, pbrt::ThreadLocal<CUstream_st*>&) ()                                                               
#15 0x00005610ed450236 in std::_Function_handler<pbrt::OptiXAggregate::OptiXAggregate(pbrt::BasicScene const&, pbrt::CUDATrackedMemoryResource*, pbrt::NamedTextures&, std::map<int, pstd::vector<pbrt::Light, pstd::pmr::
polymorphic_allocator<pbrt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, st
d::char_traits<char>, std::allocator<char> > const, pbrt::Medium> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11::basic_stri
ng<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::vector<pbrt::M
aterial, std::allocator<pbrt::Material> > const&)::GAS (), std::_Bind<pbrt::OptiXAggregate::_Bind(pbrt::BasicScene const&, pbrt::CUDATrackedMemoryResource*, pbrt::NamedTextures&, std::map<int, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Medium> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::vector<pbrt::Material, std::allocator<pbrt::Material> > const&)::{lambda()#4} ()> >::_M_invoke(std::_Any_data const&) ()                                                                                                    
#16 0x00005610ed444a55 in pbrt::AsyncJob<pbrt::OptiXAggregate::OptiXAggregate(pbrt::BasicScene const&, pbrt::CUDATrackedMemoryResource*, pbrt::NamedTextures&, std::map<int, pstd::vector<pbrt::Light, pstd::pmr::polymorp
hic_allocator<pbrt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*> > > const&, std::map<std::__cxx11::basic_string<char, std::
char_traits<char>, std::allocator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_
traits<char>, std::allocator<char> > const, pbrt::Medium> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11::basic_string<char,
 std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::vector<pbrt::Material,
 std::allocator<pbrt::Material> > const&)::GAS>::RunStep(std::unique_lock<std::mutex>*) ()                                                                                                                                
#17 0x00005610ed166c75 in pbrt::ThreadPool::WorkOrWait(std::unique_lock<std::mutex>*, bool) ()                                                                                                                            
#18 0x00005610ed166f3d in pbrt::ThreadPool::Worker() ()                                                                                                                                                                   
#19 0x00007fd9c2ce7564 in std::execute_native_thread_routine (__p=0x56112bb2ff30) at ../../../../../libstdc++-v3/src/c++11/thread.cc:104                                                                                  
#20 0x00007fd9c2aa66d7 in start_thread (arg=<optimized out>) at pthread_create.c:447                                                                                                                                      
#21 0x00007fd9c2b2a60c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

[mmp]

Interesting. What happens if you run pbrt --nthreads 1 --gpu? There's some fancy stuff where it submits multiple BVH builds to the GPU in parallel to improve startup time; it looks like it's stuck in that.

[plbpietrz]

Ok, now it's a little bit different:

[ tid 120073 @     3.865s src/pbrt/gpu/optix/aggregate.cpp:1219 ] Starting OptiX initialization
[ tid 120073 @     3.899s src/pbrt/gpu/optix/aggregate.cpp:1233 ] Optix version 7.7.0 successfully initialized
[ tid 120073 @     3.910s src/pbrt/gpu/optix/aggregate.cpp:1308 ] Warning: use of OPTIX_EXCEPTION_FLAG_DEBUG is deprecated. Use OPTIX_DEVICE_CONTEXT_VALIDATION_MODE_ALL instead.
Info: Pipeline statistics
        module(s)                            :     1
        entry function(s)                    :    19
        trace call(s)                        :     4
        continuation callable call(s)        :     0
        direct callable call(s)              :     0
        basic block(s) in entry functions    :  3560
        instruction(s) in entry functions    : 58388
        non-entry function(s)                :     0
        basic block(s) in non-entry functions:     0
        instruction(s) in non-entry functions:     0
        debug information                    :    no

[ tid 120073 @     3.910s src/pbrt/gpu/optix/aggregate.cpp:1364 ] Finished OptiX initialization
[ tid 120073 @     3.910s src/pbrt/gpu/optix/aggregate.cpp:1376 ] Starting to create shapes and acceleration structures
[ tid 120073 @     3.960s src/pbrt/gpu/optix/aggregate.cpp:1384 ] Starting to read PLY meshes
[ tid 120073 @     3.969s src/pbrt/gpu/optix/aggregate.cpp:1387 ] Finished reading PLY meshes

Backtrace for all threads:

Thread 5 (Thread 0x7f5defc00000 (LWP 120074) "pbrt"):                                                                                                                                                                     
#0  0x00007f5defea2da9 in __futex_abstimed_wait_common64 (private=<optimized out>, futex_word=0x5569b21e1050, expected=0, op=393, abstime=0x0, cancel=true) at futex-internal.c:57                                        
#1  __futex_abstimed_wait_common (futex_word=futex_word@entry=0x5569b21e1050, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>, cancel=cancel@entry=true) at futex-internal.c:87                                                                                                                                                                                                              #2  0x00007f5defea2e2f in __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x5569b21e1050, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>) at futex-internal.c:139                                                                                                                                                                                                      #3  0x00007f5defeae8ef in do_futex_wait (sem=sem@entry=0x5569b21e1050, abstime=0x0, clockid=0) at /usr/src/debug/glibc-2.39-22.fc40.x86_64/nptl/sem_waitcommon.c:111                                                      #4  0x00007f5defeae988 in __new_sem_wait_slow64 (sem=0x5569b21e1050, abstime=0x0, clockid=0) at /usr/src/debug/glibc-2.39-22.fc40.x86_64/nptl/sem_waitcommon.c:183                                                        #5  0x000055696d2e27d2 in IlmThread_2_5::Semaphore::wait() ()                                                                                                                                                             #6  0x000055696d2e1c69 in IlmThread_2_5::(anonymous namespace)::DefaultWorkerThread::run() ()                                                                                                                             
#7  0x00007f5df00e7564 in std::execute_native_thread_routine (__p=0x5569b21dba30) at ../../../../../libstdc++-v3/src/c++11/thread.cc:104                                                                                  #8  0x00007f5defea66d7 in start_thread (arg=<optimized out>) at pthread_create.c:447                                                                                                                                      #9  0x00007f5deff2a60c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78                                                                                                                                                                                                                                                                                                                                                                Thread 4 (Thread 0x7f5def200000 (LWP 120075) "cuda00001400006"):                                                                                                                                                          #0  0x00007f5deff1c87d in __GI___poll (fds=0x5569b21dbb10, nfds=3, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29                                                                                                    #1  0x00007f5df0654e3f in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  #2  0x00007f5df0727fbf in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  #3  0x00007f5df0651113 in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  #4  0x00007f5defea66d7 in start_thread (arg=<optimized out>) at pthread_create.c:447                                                                                                                                      #5  0x00007f5deff2a60c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78                                                                                                                                      
                                                                                                                                                                                                                          Thread 3 (Thread 0x7f5ded600000 (LWP 120085) "pbrt"):                                                                                                                                                                     #0  0x00007f5defea2da9 in __futex_abstimed_wait_common64 (private=0, futex_word=0x5569b2234288, expected=0, op=393, abstime=0x7f5ded5fb800, cancel=true) at futex-internal.c:57                                           #1  __futex_abstimed_wait_common (futex_word=futex_word@entry=0x5569b2234288, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x7f5ded5fb800, private=private@entry=0, cancel=cancel@entry=true) at futex-internal.c:87                                                                                                                                                                                                   #2  0x00007f5defea2e2f in __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x5569b2234288, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x7f5ded5fb800, private=private@e
ntry=0) at futex-internal.c:139                                                                                                                                                                                           #3  0x00007f5defea5b62 in __pthread_cond_wait_common (cond=0x5569b2234260, mutex=<optimized out>, clockid=0, abstime=0x7f5ded5fb800) at pthread_cond_wait.c:503                                                           #4  ___pthread_cond_timedwait64 (cond=0x5569b2234260, mutex=<optimized out>, abstime=0x7f5ded5fb800) at pthread_cond_wait.c:643                                                                                           --Type <RET> for more, q to quit, c to continue without paging--c                                                                                                                                                         #5  0x00007f5df05aebca in ?? () from /lib64/libcuda.so.1                                                                                                                                                                  #6  0x00007f5df0651113 in ?? () from /lib64/libcuda.so.1
#7  0x00007f5defea66d7 in start_thread (arg=<optimized out>) at pthread_create.c:447                                                                                                                                      
#8  0x00007f5deff2a60c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
                                                      
Thread 2 (Thread 0x7f5decc00000 (LWP 120086) "cuda-EvtHandlr"):
#0  0x00007f5deff1c87d in __GI___poll (fds=0x7f5ddc000c20, nfds=10, timeout=100) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007f5df0654e3f in ?? () from /lib64/libcuda.so.1
#2  0x00007f5df0727fbf in ?? () from /lib64/libcuda.so.1
#3  0x00007f5df0651113 in ?? () from /lib64/libcuda.so.1
#4  0x00007f5defea66d7 in start_thread (arg=<optimized out>) at pthread_create.c:447
#5  0x00007f5deff2a60c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 1 (Thread 0x7f5df25a2000 (LWP 120073) "pbrt"):
#0  0x00007f5df07ae901 in ?? () from /lib64/libcuda.so.1
#1  0x00007f5df055c4a3 in ?? () from /lib64/libcuda.so.1
#2  0x00007f5df08be30f in ?? () from /lib64/libcuda.so.1
#3  0x00007f5df08be6a5 in ?? () from /lib64/libcuda.so.1
#4  0x00007f5df057168c in ?? () from /lib64/libcuda.so.1
#5  0x00007f5df08bce48 in ?? () from /lib64/libcuda.so.1
#6  0x00007f5df0764305 in ?? () from /lib64/libcuda.so.1
#7  0x00007f5df0612ab9 in ?? () from /lib64/libcuda.so.1
#8  0x00007f5df0557f14 in ?? () from /lib64/libcuda.so.1
#9  0x00007f5d2691f01e in ?? () from /lib64/libnvoptix.so.1
#10 0x00007f5d2691c983 in ?? () from /lib64/libnvoptix.so.1
#11 0x00007f5d2687d4a4 in ?? () from /lib64/libnvoptix.so.1
#12 0x00007f5d2686e3c0 in ?? () from /lib64/libnvoptix.so.1
#13 0x000055696d204ffa in pbrt::OptiXAggregate::buildOptixBVH(OptixDeviceContext_t*, std::vector<OptixBuildInput, std::allocator<OptixBuildInput> > const&, pbrt::ThreadLocal<CUstream_st*>&) ()
#14 0x000055696d206eb5 in pbrt::OptiXAggregate::buildBVHForBLPs(std::vector<pbrt::ShapeSceneEntity, std::allocator<pbrt::ShapeSceneEntity> > const&, OptixDeviceContext_t*, OptixProgramGroup_t* const&, OptixProgramGroup
_t* const&, OptixProgramGroup_t* const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::FloatTexture, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, 
std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::FloatTexture> > > const&, std::map<std::__cxx11::basic_string<char, std::c
har_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char
_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::vector<pbrt::Material, std::allocator<pbrt::Material> > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::alloc
ator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<
char> > const, pbrt::Medium> > > const&, std::map<int, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr
::polymorphic_allocator<pbrt::Light> >*> > > const&, pbrt::ThreadLocal<pstd::pmr::polymorphic_allocator<std::byte> >&, pbrt::ThreadLocal<CUstream_st*>&) ()
#15 0x000055696d207236 in std::_Function_handler<pbrt::OptiXAggregate::OptiXAggregate(pbrt::BasicScene const&, pbrt::CUDATrackedMemoryResource*, pbrt::NamedTextures&, std::map<int, pstd::vector<pbrt::Light, pstd::pmr::
polymorphic_allocator<pbrt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*> > > const&, std::map<std::__cxx11::basic_string<cha
r, std::char_traits<char>, std::allocator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, st
d::char_traits<char>, std::allocator<char> > const, pbrt::Medium> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11::basic_stri
ng<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::vector<pbrt::M
aterial, std::allocator<pbrt::Material> > const&)::GAS (), std::_Bind<pbrt::OptiXAggregate::_Bind(pbrt::BasicScene const&, pbrt::CUDATrackedMemoryResource*, pbrt::NamedTextures&, std::map<int, pstd::vector<pbrt::Light,
 pstd::pmr::polymorphic_allocator<pbrt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*> > > const&, std::map<std::__cxx11::basi
c_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_str
ing<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Medium> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11
::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::ve
ctor<pbrt::Material, std::allocator<pbrt::Material> > const&)::{lambda()#4} ()> >::_M_invoke(std::_Any_data const&) ()
#16 0x000055696d1feae2 in pbrt::AsyncJob<pbrt::OptiXAggregate::OptiXAggregate(pbrt::BasicScene const&, pbrt::CUDATrackedMemoryResource*, pbrt::NamedTextures&, std::map<int, pstd::vector<pbrt::Light, pstd::pmr::polymorp
hic_allocator<pbrt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*> > > const&, std::map<std::__cxx11::basic_string<char, std::
char_traits<char>, std::allocator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_
traits<char>, std::allocator<char> > const, pbrt::Medium> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11::basic_string<char,
 std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::vector<pbrt::Material,
 std::allocator<pbrt::Material> > const&)::GAS>::DoWork() ()
#17 0x000055696d20bcce in pbrt::OptiXAggregate::OptiXAggregate(pbrt::BasicScene const&, pbrt::CUDATrackedMemoryResource*, pbrt::NamedTextures&, std::map<int, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<p
brt::Light> >*, std::less<int>, std::allocator<std::pair<int const, pstd::vector<pbrt::Light, pstd::pmr::polymorphic_allocator<pbrt::Light> >*> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<cha
r>, std::allocator<char> >, pbrt::Medium, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, s
td::allocator<char> > const, pbrt::Medium> > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pbrt::Material, std::less<std::__cxx11::basic_string<char, std::char_trai
ts<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, pbrt::Material> > > const&, std::vector<pbrt::Material, std::allocator
<pbrt::Material> > const&) ()
#18 0x000055696d170a3c in pbrt::WavefrontPathIntegrator::WavefrontPathIntegrator(pstd::pmr::memory_resource*, pbrt::BasicScene&) ()
#19 0x000055696cf9968b in pbrt::RenderWavefront(pbrt::BasicScene&) ()
#20 0x000055696ccb7079 in main ()

[sasamiyatu]

Just commenting to note that I am having the same issue (getting stuck at the same point) using Windows 11 with RTX 4090 (driver version 561.17), CUDA 12.4 and Optix 8.1.

Edit:
This problem seems to be dependent on the scene. The hair scenes in the scenes repo seem to get stuck (e.g. hair-actual-bsdf.pbrt). The killeroo scenes for example render fine.

[plbpietrz]

@sasamiyatu can confirm that the bunny-fur scene exhibits this 'hang', other scenes that I've testes, render properly. I'm on CUDA 12.1 and Optix 7.5, behavior reproduced on 4090 and a laptop with RTX 2000. This looks like something worth reporting to Nvidia.

[pbrt4bounty]

I'm not sure if this are the cause, but for me, this hair 'curly-hair.pbrt.gz' file seems empty :(
Edited: seems a Windows explorer issue. I'm sorry for the noise

[plbpietrz]

I'll add to my previous reporting that when switched to Windows (previously was on Fedora 41, 565.77 nvidia driver), CUDA 12.4, Optix 7.5, latest driver at the time of writing, scenes with hair (bunny) get stuck, other scenes, work.