Initial implementation of a generalized ERF particle interface

[gardner48]

Retained tracer and hydro particles as specific types
Added uniform distribution of particles as an initialization option

[debog]

@asalmgren @AMLattanzi @gardner48 In this pull request, I had not updated the source file list in BuildERFExe.cmake. So compiling the entire code with CMake caused an error, because CMake was looking for one of the files I had deleted/renamed.

So why didn't the automated checks on github catch this?

[asalmgren]

Because we don't currently have a particle test in the CI!

[gardner48]

Ann beat me to the answer again. The GitHub actions don't set ERF_ENABLE_PARTICLES when configuring with CMake and it looks like the nightly tests use GNUMake.

Also looks like ERF_ENABLE_PARTICLES is missing from the top level CMakeLists.txt. It is set to ON in ParticlesOverWoA but that's after the AMReX options have been set in the top level CMakeLists.txt so I expect that would cause some issues as well.

[asalmgren]

For local cmake testing I do

mkdir build
cd build
cp ../Build/cmake_with_particles.sh .
./cmake_with_particles.sh ..

and that does the local build with particles turned on ...

[debog]

I am not sure why but it seems my default cmake compile has particles turned on.

This is my cmake script:

cmake -DCMAKE_INSTALL_PREFIX:PATH=./install \
      -DCMAKE_CXX_COMPILER:STRING=mpicxx \
      -DCMAKE_C_COMPILER:STRING=mpicc \
      -DCMAKE_Fortran_COMPILER:STRING=mpifort \
      -DMPIEXEC_PREFLAGS:STRING=--oversubscribe \
      -DCMAKE_BUILD_TYPE:STRING=Release \
      -DERF_DIM:STRING=3 \
      -DERF_ENABLE_MPI:BOOL=ON \
      -DERF_ENABLE_TESTS:BOOL=ON \
      -DERF_ENABLE_FCOMPARE:BOOL=ON \
      -DERF_ENABLE_DOCUMENTATION:BOOL=OFF \
      -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=ON \
      ${ERF_HOME} && make -j4

and I see the following lines in the output:

--    AMReX_PARTICLES
--    AMReX_PARTICLES_PRECISION = DOUBLE

I have to specify -DERF_ENABLE_PARTICLES:BOOL=OFF to compile without particles.

Also, in Build/cmake_with_particles.sh, why does one need both -DAMREX_PARTICLES:BOOL=ON and -DERF_ENABLE_PARTICLES:BOOL=ON? Shouldn't the latter enable the former?

[asalmgren]

I will defer to others for cmake questions :-)

[gardner48]

I have to specify -DERF_ENABLE_PARTICLES:BOOL=OFF to compile without particles.

Interesting, that's not what I would expect from looking at the CMake files. I'll take another look, I must have missed something.