Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated ppml #356

Draft
wants to merge 92 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
3e41dd6
Added old MHD Turbulence forcing from 2018
jobordner Aug 21, 2021
a4f8b9e
Updated PPML routines with Alexei Kritusk edits
jobordner Aug 23, 2021
507fd17
Implementing EnzoMethodTurbulenceOU
jobordner Sep 3, 2021
081b7be
Fixing circleci issue: removing -pedantic
jobordner Sep 4, 2021
0fa9c45
Implementing TurbulenceOU: added reductions
jobordner Sep 4, 2021
a8c88b8
Implementing TurbulenceOU: adding Cello fields
jobordner Sep 8, 2021
b571c8e
Fixing single precision compile bug
jobordner Sep 9, 2021
5543464
First draft of EnzoMethodTurbulenceOU complete
jobordner Sep 15, 2021
259982f
Adding MethodPpmlIG (in progress)
jobordner Sep 27, 2021
df73171
Fixes for single precision build
jobordner Sep 27, 2021
cd8421e
Implementing PPML_IG (in progress)
jobordner Oct 8, 2021
80d5726
Debugging PPML_IG (in progress)
jobordner Oct 22, 2021
35e8400
disabled fp trapping in enzo-p.cpp
jobordner Dec 17, 2021
32c3d2c
Fixing precision compile bug and ppml test bug
jobordner Dec 17, 2021
6ad8d12
fixing debug code
jobordner Apr 20, 2022
d9be091
merge enzo-project/main < jobordner/mhd_turbulence
jobordner Apr 23, 2022
d5808ad
Merged jobordner/mhd-turbulence < new-input, etc.
jobordner May 9, 2022
39c1e03
Saved turbulenceOU state for consistency in restart
jobordner May 13, 2022
e9c5692
Debugged saving turbulenceOU state for restart
jobordner May 19, 2022
6129c2d
Added extract_field utility for HDF5 fields
jobordner May 27, 2022
8908bf5
Updating extract_field Usage
jobordner Jul 1, 2022
abb0884
Renaming work fields as (external) accelerations
jobordner Jul 8, 2022
c850760
Added EnzoMethodInferenceArray
jobordner Jul 12, 2022
6ee3e17
Simon says added EnzoMethodInferenceArray
jobordner Jul 12, 2022
4f6a67a
Merge jobordner/inference-array < enzo-project
jobordner Jul 13, 2022
e83e8cb
Fixed extract_field, is_leaf init.; PPML_IG fixes
jobordner Jul 21, 2022
cb0a37e
bug fixes: extract_field, PPML_IG
jobordner Jul 23, 2022
305894f
merging
Jul 23, 2022
8b5f09f
merging
Jul 23, 2022
731c461
removing frontera mods
jobordner Jul 23, 2022
45cf310
Fix div(B) evaluation
jobordner Aug 9, 2022
47ac86f
Implementing inference array method
jobordner Aug 12, 2022
f527e7f
Adding missing code and doc files
jobordner Aug 12, 2022
6e71bcb
Merge jobordner/inferenc-array < enzo-project/main
jobordner Aug 16, 2022
dcb1e63
adding inference documentation
jobordner Aug 16, 2022
44df1f7
Implementing inference array method
jobordner Aug 19, 2022
ea257b0
updating inference array uml image
jobordner Aug 19, 2022
b2a27db
Updates to turbulence_ou
jobordner Aug 31, 2022
e379c19
Copying field stats from turb to turbou(!)
jobordner Sep 7, 2022
20c0911
Added text design draft to inference array docs
jobordner Sep 10, 2022
09b7a3b
Updating documentation
jobordner Sep 14, 2022
c237d86
fixed bugs in extract_field
Sep 15, 2022
70792a1
threading extract_field
jobordner Sep 16, 2022
7ecb01a
Adding "support" for Strang splitting with PPML
jobordner Sep 26, 2022
9e9af6e
bug fix in EnzoMethodTurbulenceOU field name
jobordner Oct 1, 2022
b5eca73
bug fix: added refresh fields in turbulence_ou
jobordner Oct 2, 2022
5a32b50
Merge remote-tracking branch 'refs/remotes/origin/mhd-turbulence' int…
jobordner Oct 2, 2022
c7aeb73
Implementing inference array method
jobordner Oct 7, 2022
4735b3e
Adding missing uml & image files in docs
jobordner Oct 7, 2022
359fc05
Implementing inference array method
jobordner Oct 18, 2022
ac7d10e
Overhauling field stats in EnzoMethodTurbulence
jobordner Oct 26, 2022
530e416
Implementing inference array method
jobordner Oct 28, 2022
b8fe52c
Implementing EnzoMethodInference
jobordner Nov 10, 2022
02b7d0a
Adding missing source file
jobordner Nov 10, 2022
6742e47
Finishing EnzoMethodInference;
jobordner Nov 14, 2022
60c155e
Debugging EnzoMethodInference;
jobordner Nov 16, 2022
1afa2d3
merging enzo-project -> jobordner/inference-array
jobordner Nov 17, 2022
4f9d131
Finishing up EnzoMethodInference; cleaning
jobordner Nov 24, 2022
47a3e4b
fixed botched merge of 'VL+CT test fix' PR
jobordner Nov 24, 2022
234b448
Adapt
jobordner Dec 1, 2022
056cce3
Adding inference method parameter documentation
jobordner Dec 5, 2022
6dadf18
cleaning inference parameter documentation
jobordner Dec 5, 2022
4f6f6b4
Updating inference array design documentation
jobordner Dec 14, 2022
a14d13a
Inference array documentation final draft
jobordner Dec 16, 2022
e0908d1
Finishing up EnzoMethodInference
jobordner Dec 27, 2022
0954a10
Added inference test problems; fixed hang bug
jobordner Dec 29, 2022
f795b3f
Removing unused inference-related code
jobordner Dec 29, 2022
c79204f
Adding missing grackle chemistry pup's
jobordner Dec 30, 2022
dc2bedc
merging inference method
jobordner Dec 31, 2022
63e81ab
merging balance-cello
jobordner Dec 31, 2022
83efc58
Merge remote-tracking branch 'upstream/main'
jobordner Dec 31, 2022
b207a0a
Merge branch 'enzo-project:main' into inference-array
jobordner Dec 31, 2022
027c4af
bug fix: added missing inference array offsets
jobordner Apr 7, 2023
5457219
merge jobordner inference-array < main
jobordner Apr 7, 2023
5284fcf
merge jobordner/main <- enzo-project/main
jobordner Jun 27, 2023
87140cd
Merge remote-tracking branch 'upstream/main' into updated-ppml
jobordner Aug 18, 2023
9614ec6
merge updated-ppml < enzo-project/main
jobordner Aug 31, 2023
019adba
merged updated-ppml-merge < main
jobordner Nov 3, 2023
9f6ed41
merged updated-ppml-merge < main
jobordner Nov 3, 2023
3c432cb
Fixing compile error from previous merge
jobordner Nov 4, 2023
685d1a6
deleting obsolete SConstruct files
jobordner Nov 7, 2023
b3fe750
re-allow empty particle arrays in IO: broke tests
jobordner Nov 9, 2023
a3712b9
Removed inference method code from PPML branch
jobordner Nov 30, 2023
fc282ee
updated circleci config
jobordner Dec 1, 2023
de4c756
Removed inference docs: separate branch
jobordner Dec 1, 2023
ae379db
moving new methods to subdirectories
jobordner Dec 5, 2023
8db464e
merged updated-ppml <- main
jobordner Dec 5, 2023
3601adf
Added missing cmake files
jobordner Dec 5, 2023
5f16c3e
Adding ppml & turbulence_ou docs
jobordner Jan 5, 2024
cfbb20c
Adding Alexei Kritsuk turbulence documentation
jobordner Jan 12, 2024
7223062
merging jobordner/updated-ppml < main
jobordner Jan 26, 2024
ed8ae38
updating turbou to auto-detect PPM[L] fields
jobordner Mar 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion config/linux_gcc.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,13 @@ if(NOT __processedUserDefaults)
set(CMAKE_Fortran_COMPILER gfortran CACHE STRING "")
# Note (12/2021): passing -march=native to gfortran seems to slow down the
# PPM solver
set(CMAKE_Fortran_FLAGS "-ffixed-line-length-132" CACHE STRING "Default Fortran flags")
set(CMAKE_Fortran_FLAGS "-ffixed-line-length-132 -std=legacy " CACHE STRING "Default Fortran flags")

set(__ARCH_C_OPT_FLAGS "-O3 -g -funroll-loops")
set(CMAKE_C_FLAGS_RELEASE "${__ARCH_C_OPT_FLAGS}")
set(CMAKE_C_FLAGS_RELWITHDEBINFO "-g ${__ARCH_C_OPT_FLAGS}")
set(CMAKE_CXX_FLAGS_RELEASE "${__ARCH_C_OPT_FLAGS}")
set(CMAKE_CXX_FLAGS_RELWITHDEBINFO "-g ${__ARCH_C_OPT_FLAGS}")

# these flag(s) are currently only used when using openmp-simd optimizations
# (to specify available/prefered instruction sets).
Expand Down
78 changes: 78 additions & 0 deletions config/linux_gcc_9.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@

import os
from _common_search_paths import charm_path_search, grackle_path_search

is_arch_valid = 1

#
#flags_arch = '-g -fprofile-arcs -ftest-coverage' # gcov
#flags_arch = '-O3 -fopenmp -Wall -g -ffast-math -funroll-loops -fPIC'
flags_arch = '-O3 -fopenmp -Wall -g -ffast-math -funroll-loops -fPIC'
#flags_arch = '-Wall -O1 -g -fPIC -pedantic'
#flags_arch = '-Wall -O0 -g'
#flags_arch = '-O3 -pg -g'
#flags_arch = '-Wall -g -fsanitize=address -fno-omit-frame-pointer'

# rdynamic required for backtraces
#flags_link_charm = '-rdynamic'
#flags_link_charm = '-memory paranoid'
#flags_link_charm = '-fprofile-arcs' # gcov

#optional fortran flag
flags_arch_fortran = '-ffixed-line-length-132 -std=legacy '

cc = 'gcc '
f90 = 'gfortran'

flags_prec_single = ''
flags_prec_double = '-fdefault-real-8 -fdefault-double-8'

libpath_fortran = '/usr/lib/gcc/x86_64-linux-gnu/9'
libs_fortran = ['gfortran']

home = os.getenv('HOME')

# use Charm++ with randomized message queues for debugging and stress-testing
# charm_path = home + '/Charm/charm.random'

charm_path = charm_path_search(home)

# use_papi = 1
# papi_inc = os.getenv('PAPI_INC', '/usr/include')
# papi_lib = os.getenv('PAPI_LIB', '/usr/lib')

hdf5_inc = os.getenv('HDF5_INC')
if hdf5_inc is None:
if os.path.exists('/usr/include/hdf5.h'):
hdf5_inc = '/usr/include'
elif os.path.exists('/usr/include/hdf5/serial/hdf5.h'):
hdf5_inc = '/usr/include/hdf5/serial'
else:
raise Exception('HDF5 include file was not found. Try setting the HDF5_INC environment variable such that $HDF5_INC/hdf5.h exists.')

hdf5_lib = os.getenv('HDF5_LIB')
if hdf5_lib is None:
if os.path.exists('/usr/lib/libhdf5.a'):
hdf5_lib = '/usr/lib'
elif os.path.exists('/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.a'):
hdf5_lib = '/usr/lib/x86_64-linux-gnu/hdf5/serial'
else:
raise Exception('HDF5 lib file was not found. Try setting the HDF5_LIB environment variable such that $HDF5_LIB/libhdf5.a exists.')

png_path = os.getenv('LIBPNG_HOME', '/lib/x86_64-linux-gnu')

boost_inc = os.getenv('BOOST_INC', '/usr/include/boost')
boost_lib = os.getenv('BOOST_LIB', '/usr/lib/x86_64-linux-gnu')

grackle_path = grackle_path_search(home)

cello_var = os.environ.get('CELLO_VAR',"net")

if (cello_var == "net"):
parallel_run = charm_path + "/bin/charmrun +p4 "
parallel_arg = " "
smp = 0
elif (cello_var == "net-smp"):
parallel_run = charm_path + "/bin/charmrun +p4 "
parallel_arg = " ++ppn 4 "
smp = 1
69 changes: 69 additions & 0 deletions config/linux_gnu.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@

import os
from _common_search_paths import charm_path_search, grackle_path_search

is_arch_valid = 1

#
#flags_arch = '-g -fprofile-arcs -ftest-coverage' # gcov
flags_arch = '-O3 -g -ffast-math -funroll-loops -fPIC'
#flags_arch = '-Wall -O3 -g'
#flags_arch = '-Wall -O0 -g'

#flags_arch = '-O3 -pg -g'
#flags_arch = '-fprofile-arcs -ftest-coverage'
#flags_arch = '-Wall -g -fsanitize=address -fno-omit-frame-pointer'
#flags_arch = '-Wall -O3 -pg'

# rdynamic required for backtraces
#flags_link_charm = '-rdynamic'
#flags_link_charm = '-memory paranoid'
#flags_link_charm = '-fprofile-arcs' # gcov

#optional fortran flag
flags_arch_fortran = '-ffixed-line-length-132'

cc = 'gcc '
f90 = 'gfortran'

flags_prec_single = ''
flags_prec_double = '-fdefault-real-8 -fdefault-double-8'

libpath_fortran = '/usr/lib/x86_64-linux-gnu'
libs_fortran = ['gfortran']

home = os.getenv('HOME')

# use Charm++ with randomized message queues for debugging and stress-testing
# charm_path = home + '/Charm/charm.random'

charm_path = charm_path_search(home)

# use_papi = 1
# papi_inc = os.getenv('PAPI_INC', '/usr/include')
# papi_lib = os.getenv('PAPI_LIB', '/usr/lib')

hdf5_inc = os.getenv('HDF5_INC')
if hdf5_inc is None:
if os.path.exists('/usr/include/hdf5.h'):
hdf5_inc = '/usr/include'
elif os.path.exists('/usr/include/hdf5/serial/hdf5.h'):
hdf5_inc = '/usr/include/hdf5/serial'
else:
raise Exception('HDF5 include file was not found. Try setting the HDF5_INC environment variable such that $HDF5_INC/hdf5.h exists.')

hdf5_lib = os.getenv('HDF5_LIB')
if hdf5_lib is None:
if os.path.exists('/usr/lib/libhdf5.a'):
hdf5_lib = '/usr/lib'
elif os.path.exists('/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.a'):
hdf5_lib = '/usr/lib/x86_64-linux-gnu/hdf5/serial'
else:
raise Exception('HDF5 lib file was not found. Try setting the HDF5_LIB environment variable such that $HDF5_LIB/libhdf5.a exists.')

png_path = os.getenv('LIBPNG_HOME', '/lib/x86_64-linux-gnu')

boost_inc = os.getenv('BOOST_INC', '/usr/include/boost')
boost_lib = os.getenv('BOOST_LIB', '/usr/lib/x86_64-linux-gnu')

grackle_path = grackle_path_search(home)
Binary file added doc/source/design/array.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/source/design/infer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/source/design/inference-array.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/source/design/io-read.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/source/design/lowres.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 7 additions & 2 deletions doc/source/param/index.rst
Original file line number Diff line number Diff line change
@@ -1,9 +1,15 @@
.. include:: ../roles.incl

*************************
Enzo-E / Cello Parameters
*************************

.. toctree::
:maxdepth: 1
:glob:
:titlesonly:
:numbered:

This page documents all current parameters implemented in Enzo-E /
Cello. Each parameter is summarized, its type or types are listed,
and the default value (if any) is provided. The scope of the
Expand Down Expand Up @@ -152,7 +158,6 @@ Mesh

.. include:: mesh.incl


------
Method
------
Expand Down
21 changes: 20 additions & 1 deletion doc/source/param/method.incl
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,17 @@
* :t:`"ppml"` :e:`for the PPML ideal MHD solver.` *This may be phased
out in favor of using a more general "mhd" method instead, with a
specific mhd solver specified.*
* :t:`"ppml_it"` :e:`alternate name for "ppml".`
* :t:`"ppml_ig"` :e:`for the adiabatic version of PPML.` *This may be phased
out in favor of using a more general "mhd" method instead, with a
specific mhd solver specified.*
* :t:`"mhd_vlct"` :e:`for the VL + CT (van Leer + Constrained Transport) MHD
solver.`
* :t:`"trace"` :e:`for moving tracer particles.` **This will be phased
out in favor of a more general "move_particles" method.**
* :t:`"turbulence"` :e:`computes random forcing for turbulence
simulations.`

* :t:`"turbulence_ou"` :e:`implements Ornstein-Uhlenbeck driven turbulence.`

:e:`Parameters specific to individual methods are specified in subgroups, e.g.`::

Expand Down Expand Up @@ -461,6 +465,12 @@ ppm
.. include:: method_ppm.incl


ppml
----

.. include:: method_ppml.incl


sink_maker
----------

Expand Down Expand Up @@ -569,6 +579,8 @@ turbulence
:Scope: :z:`Enzo`
:Todo: :o:`write`

:e:`Prescribes kinetic energy injection rate for a forcing method based on random but constant-in-time external x-, y-, z-acceleration fields. The code would normalize the accelerations so that the resulting actual injection rate is constant during the simulation and equal to edot. For details of the method see` `Mac Low (1999) <https://ui.adsabs.harvard.edu/abs/1999ApJ...524..169M/abstract>`_ :e:`The default setting edot=-1 turns this forcing off.`

----

.. par:parameter:: Method:turbulence:mach_number
Expand All @@ -578,3 +590,10 @@ turbulence
:Default: :d:`0.0`
:Scope: :z:`Enzo`
:Todo: :o:`write`

:e:`Specifies the target sonic Mach number for a forced turbulence simulation. If` :p:`mach_number` :e:`is not zero, the energy injection rate will be calculated based on the Mach number.`

turbulence_ou
-------------

.. include:: method_turbulence_ou.incl
15 changes: 15 additions & 0 deletions doc/source/param/method_ppml.incl
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@

:p:`Method:ppml` parameters are used to initialize parameters for
Enzo-E's PPML MHD methods. These include "ppml" (or "ppml_it") for the
idealized solver, and "ppml_ig" for the adiabatic version.

----

.. par:parameter:: Method:ppml:dt_weight

:Summary: :s:`Timestep adjust for Strang splitting`
:Type: :par:typefmt:`float`
:Default: :d:`1.0`
:Scope: :z:`Enzo`

:e:`This parameter is used to implement Strang splitting to preserve the 2nd order of accurace of time integration of PPML when forcing and cooling/heating source terms are present. In that case dt_weight should be set to 0.5 and PPML called twice in the` :par:paramfmt:`Method:list` :e:` parameter list with methods implementing forcing terms between them. The default is not to use Strang splitting so dt_weight = 1.0`
Loading