Remove managed memory from read_workerflow #82
Conversation
|
These modifications result in a significant increases in the infection rates seen in GPU runs, but there is no change for CPU runs. In versions before this, the CPU and GPU results are almost identical for the same seed, but now they differ significantly. |
src/CensusData.cpp
Outdated
| @@ -736,16 +766,17 @@ void CensusData::assignTeachersAndWorkgroup (AgentContainer& pc /*!< Agent | |||
| auto Ncommunity = demo.Ncommunity; | |||
|
|
|||
| auto np = soa.numParticles(); | |||
| for (int ip = 0; ip < np; ++ip) { | |||
|
|
|||
| amrex::ParallelForRNG( np, | |||
There was a problem hiding this comment.
This change is what causes the difference between the GPU and CPU results. Maybe there are updates in the loop that require atomics for the GPU
There was a problem hiding this comment.
Yes, you're right - I think that instead of running this the Gpu, for now we need to explicitly copy the data off the device, run this loop in serial, then copy the data back. Since this only happens at initialization I think this will be okay.
There was a problem hiding this comment.
Note - with this change, AND with constructing the bins on the GPU, the code runs without crashing on development with amrex.the_arena_is_managed=0.
There was a problem hiding this comment.
If I run unmanaged with random initial cases, I still get a segfault here, on line 961 in CensusData.cpp:
auto inds = bins.permutationPtr();
There was a problem hiding this comment.
This appears to go away if we use the GPU binning policy rather than Serial.
There was a problem hiding this comment.
Binning with BinPolicy::Serial still relies on managed memory, but if you change that to BinPolicy::GPU, it works for me - is that not what you're seeing?
There was a problem hiding this comment.
Yes, that's what i'm seeing. With GPU policy, it works. So we should keep the default managed memory for reproducibility. But I can now check the other branch to see if there is anywhere managed memory is being implicitly used.
|
Superseded by #84 |
No description provided.