Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ghost translation #1006

Open
wants to merge 113 commits into
base: dev
Choose a base branch
from
Open

Ghost translation #1006

Show file tree
Hide file tree
Changes from 93 commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
fde9134
Interim commit for removing interim MPI from between translation dire…
markusbattarbee Feb 4, 2022
de7cb92
Reinstated remote neighbour information update and use of updated DCCRG
markusbattarbee Feb 4, 2022
7fa5ae6
Optimized boundary cell pencils and load balance metrics
markusbattarbee Feb 10, 2022
3790398
Store pencil source as cellIDs, not pointers
markusbattarbee Feb 14, 2022
5317947
Added option to only split translation AMR pencils if face neighbour …
markusbattarbee Feb 14, 2022
e8885e6
Merge branch 'less_pencil_splitting' into AMR_translation_optimizations
markusbattarbee Feb 14, 2022
a550e8a
Implemented case defaults, info output to Logger, some optimizations,…
markusbattarbee Mar 30, 2022
46b103b
Path for Mahti makefile to testing version of DCCRG
markusbattarbee Apr 6, 2022
e6b663a
typo fix for logFile output in parameters
markusbattarbee Apr 19, 2022
46ad31d
Interim commit, trying to get one layer of diagonal translation worki…
markusbattarbee Apr 20, 2022
065e9d0
whitespace
markusbattarbee Apr 21, 2022
87948e9
remote list updates
markusbattarbee May 2, 2022
700b7d0
Merge branch 'AMR_translation_optimizations' of github.com:markusbatt…
markusbattarbee May 2, 2022
f83f9f5
Testing neighbourhoods
markusbattarbee May 3, 2022
c3965ee
Merge branch 'dev' into AMR_translation_optimizations_dev
markusbattarbee May 3, 2022
9d4e909
Block list updates
markusbattarbee May 3, 2022
15ddf61
continued local tranlation debugging
markusbattarbee May 4, 2022
a72d486
Merge branch 'dev' into AMR_translation_optimizations_dev
markusbattarbee Oct 27, 2022
93f19c4
Filter out translation source cells with valid pointer but no data
markusbattarbee Oct 27, 2022
9357735
timer initialization
markusbattarbee Oct 27, 2022
477e1f6
Attempting some cleanup of comments and test functions and unnecessar…
markusbattarbee Oct 28, 2022
b77beb9
one missing comment line
markusbattarbee Oct 28, 2022
3243ce6
more cleanup, reverted some communications, still testing
markusbattarbee Oct 28, 2022
035fa47
Even smaller flowthrough test for DCCRG neighbour debugging.
ursg Aug 9, 2023
59af982
Pull Request #609 debugging status
ursg Jun 13, 2023
4def807
Check whether a recieved cell actually exists
ursg Jun 16, 2023
01614cd
More timeclass debugging code
ursg Jun 16, 2023
70c6f0e
Revert some debugging output
ursg Aug 10, 2023
3eae00b
Reduce the neighborhood_length to VLASOV_STENCIL_WIDTH
ursg Oct 24, 2023
fe73a6e
Merge remote-tracking branch 'origin/dev' into 609Test
ursg Oct 31, 2023
f445eff
Disable -DDEBUG again, since it is making runs impractically slow.
ursg Oct 31, 2023
a16bab6
Merge remote-tracking branch 'origin/dev' into 609Test
ursg Jun 6, 2024
673c17d
Fix crash if OMPI_MCA_io is not set in environment.
ursg Jun 6, 2024
4bde2a5
(Re-)remove face neighbor caching.
ursg Jun 18, 2024
83ec043
Point DCCRG submodule to new neighbor definition branch
ursg Jun 18, 2024
ea990b6
Actually properly remove face neighbor caching
ursg Jun 18, 2024
f810036
Merge remote-tracking branch 'origin/dev' into 609Test
ursg Jun 19, 2024
f8a7123
Re-enable translation in all directions.
ursg Jun 19, 2024
aada15d
Revert testpackage cfgs to their before-neighbor-update states
ursg Jun 19, 2024
bf2333a
Remove some commented-out debug statements.
ursg Jun 19, 2024
c74b6d9
Clean up more debug remnants
ursg Jun 19, 2024
d662fe4
Compile with --std=c++20.
ursg Jun 19, 2024
cce3785
Bump dccrg submodule commit id.
ursg Jun 19, 2024
e3e1707
Revert dccrg submodule to fmihpc/dccrg
ursg Jun 20, 2024
580aa19
Purge the no-longer-required addStencilDepth
ursg Jun 20, 2024
4ac6699
Remove debugging nan check leftover
ursg Jun 20, 2024
a8bbca9
Remove more outdated debugging cruft
ursg Jun 20, 2024
021256b
Re-introduce omp parallel in cpu_trans_map
ursg Jun 20, 2024
76b8ae2
Update DCCRG reference, fix one comment.
ursg Jun 26, 2024
faedbca
Update DCCRG submodule version reference
ursg Jun 27, 2024
687d641
Update DCCRG submodule reference
ursg Jun 28, 2024
0d917ec
Bump DCCRG version again.
ursg Jun 28, 2024
e0befdd
Merge branch '609Test' into LocalTranslation
markusbattarbee Jul 1, 2024
178689d
Also include pencils changes
markusbattarbee Jul 1, 2024
24f2c3f
Cleanup, rename local to ghost translation, remove option of face pen…
markusbattarbee Jul 3, 2024
ed8195b
Simplify check_is_written_to()
markusbattarbee Jul 3, 2024
fcb6b28
update dccrg commit
markusbattarbee Jul 4, 2024
047745c
cleanup
markusbattarbee Jul 5, 2024
c4c844e
Add VLASOV_STENCIL_WIDTH+1 ghost neighbourhoods
markusbattarbee Jul 9, 2024
1f5dc23
Add parameter for full or minimal ghost translation
markusbattarbee Jul 9, 2024
2db4ad9
Back to two sets per dimension for ghost translation, including debug…
markusbattarbee Jul 9, 2024
ac22ef9
Remove some debug outputs
markusbattarbee Jul 9, 2024
77cfdd8
Add option to manually set ghost translation extent, defaults to VLAS…
markusbattarbee Jul 9, 2024
75b90fe
Fix parameter for ghost stencil, update dccrg submodule
markusbattarbee Jul 9, 2024
4aa665f
Add extra phiprofs to preparation of ghost lists
markusbattarbee Jul 9, 2024
a23546f
fix timers and compile warnings
markusbattarbee Jul 9, 2024
84b4fde
Add parallel regions to ghost translation cell list preparation
markusbattarbee Aug 8, 2024
07f4b0f
Make remote_cells2 non-const
markusbattarbee Aug 8, 2024
7e68c00
Fix to set parallel loops
markusbattarbee Aug 8, 2024
e3c372a
add valid cell checks
markusbattarbee Aug 8, 2024
aeae374
Lose omp parallel from dccrg call again
markusbattarbee Aug 9, 2024
fdc53e2
Remove parallelism from pencils loop as sets not threadsafe. Re-imple…
markusbattarbee Aug 9, 2024
d9b6618
Debug printouts, have odd std::map out of range error
markusbattarbee Aug 9, 2024
6602bf6
Revert neighbourhood update shenanigans
markusbattarbee Aug 9, 2024
8269f6b
Fix ionosphere mini-app compilation
markusbattarbee Aug 9, 2024
31153ad
Rejig initialization to try and get Magneto/ionospheres to work
markusbattarbee Aug 9, 2024
b6536c8
Rejig order
markusbattarbee Aug 11, 2024
b7c710d
Change to null cell check, turn on only 2 tests to speed up testing
markusbattarbee Aug 11, 2024
7a9524a
Change to null cell check 2nd place
markusbattarbee Aug 11, 2024
fbfcac6
Rejig order more
markusbattarbee Aug 12, 2024
b474319
Better naming, re-introduce all tests, try to drop some communication
markusbattarbee Aug 12, 2024
912450d
Merge branch 'dev' into GhostTranslation
markusbattarbee Aug 12, 2024
2a50b03
Fix issue with MPI in adaptrefinement (was lazy)
markusbattarbee Aug 13, 2024
4c8c8fd
Plenty of cleanup
markusbattarbee Aug 13, 2024
f1e86c0
Purge AMR translation flags as no longer needed
markusbattarbee Aug 13, 2024
7f787f9
Re-introduced larger region block list update as boundaries may also …
markusbattarbee Aug 14, 2024
c743e1f
Added GT timers, update submodule dccrg
markusbattarbee Aug 20, 2024
8694028
Update grid.cpp comments
markusbattarbee Aug 20, 2024
126debf
Cleanup, turn GT off as default, add some debug cerrs
markusbattarbee Aug 21, 2024
9994717
Back to smaller neighbourhood for deallocations if not ghost-translating
markusbattarbee Aug 21, 2024
59da6b1
Re-introduce some remote cells for block gathering in translation (ne…
markusbattarbee Aug 22, 2024
ad7e5d7
actually re-allow non-local cells to add blocks to propagation
markusbattarbee Aug 22, 2024
da53236
nicer logic for skipping translation list building
markusbattarbee Aug 22, 2024
8cdd054
fix comment in parameters
markusbattarbee Aug 23, 2024
5e4e6d4
Fix DCCRG commit
markusbattarbee Aug 23, 2024
fc5d22e
update dccrg
markusbattarbee Aug 30, 2024
1b9fbc8
Re-instate use of cached face neighbors
markusbattarbee Aug 30, 2024
2a776c9
Switch to custom overall ghost neighbourhood for MPI updates (allows …
markusbattarbee Sep 2, 2024
4d0eb35
update dccrg, attempts at adding some threading and debugging to fix …
markusbattarbee Sep 11, 2024
e227bf1
Added lumi 24.03 first makefile using project 358
markusbattarbee Sep 16, 2024
7574a34
Debug printouts as well as gathering GT statistics from all tasks
markusbattarbee Sep 16, 2024
854979a
Add extra timers to ghost translation
markusbattarbee Sep 16, 2024
c00c594
Merge branch 'lumi_2403' into HEAD
markusbattarbee Sep 16, 2024
91bdb15
Different LB weights, more comments on GT cell lists
markusbattarbee Sep 17, 2024
b464bf2
Add extra timers to ghost translation
markusbattarbee Aug 18, 2024
e230da5
Merge branch 'GTb' into GhostTranslation
markusbattarbee Sep 17, 2024
fc37ae6
Better initial weight estimate
markusbattarbee Sep 17, 2024
c65469b
typo fixes
markusbattarbee Sep 17, 2024
31e93a2
Fix LB mistake with incorrect cell list indexing in pencil count
markusbattarbee Sep 17, 2024
4ba7f59
Switch to basic 3x blocks weight in LB
markusbattarbee Oct 2, 2024
13d5ba2
Merge branch 'dev' into GhostTranslation
markusbattarbee Oct 2, 2024
368eacc
Revert neighbourhood testing boost, remove plenty of debug outputs
markusbattarbee Oct 7, 2024
b8544ef
Make parallel loop work with Arriesgado
markusbattarbee Oct 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions MAKE/Makefile.Freezer
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ endif
# MATHFLAGS are for special math etc. flags, these are only applied on solver functions
# LDFLAGS flags for linker

CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -fabi-version=0 -mfma -mavx2 -Wno-unknown-pragmas -Wno-sign-compare
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0 -mno-avx -mno-fma -fno-unsafe-math-optimizations
CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -fabi-version=0 -mfma -mavx2 -Wno-unknown-pragmas -Wno-sign-compare
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0 -mno-avx -mno-fma -fno-unsafe-math-optimizations

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.MN5_gpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ else
endif

# GNU flags:
CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -mavx2
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0 -mavx
CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -mavx2
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0 -mavx

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
2 changes: 1 addition & 1 deletion MAKE/Makefile.appleM1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ CXXFLAGS += -DUSE_JEMALLOC -DJEMALLOC_NO_DEMANGLE
# mpi.h in c++ on Cray

FLAGS =
CXXFLAGS += -I ${GITHUB_WORKSPACE}/libraries-appleM1/include -I/opt/homebrew/opt/llvm/include -I/opt/homebrew/opt/libomp/include -march=native -O3 -std=c++17 -fopenmp -Wall -Wno-unused -g
CXXFLAGS += -I ${GITHUB_WORKSPACE}/libraries-appleM1/include -I/opt/homebrew/opt/llvm/include -I/opt/homebrew/opt/libomp/include -march=native -O3 -std=c++20 -fopenmp -Wall -Wno-unused -g
MATHFLAGS = # -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = # -fno-unsafe-math-optimizations
LDFLAGS = -fopenmp -L/opt/homebrew/opt/llvm/lib -L/opt/homebrew/opt/libomp/lib -L/opt/homebrew/opt/llvm/lib/c++ -g
Expand Down
2 changes: 1 addition & 1 deletion MAKE/Makefile.arriesgado
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ endif

FLAGS =
# note: std was c++11
CXXFLAGS = -O1 -std=c++17 -W -Wall -pedantic -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fopenmp -march=rv64imafdc -isystem /usr/lib/gcc/riscv64-linux-gnu/11/include/
CXXFLAGS = -O1 -std=c++20 -W -Wall -pedantic -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fopenmp -march=rv64imafdc -isystem /usr/lib/gcc/riscv64-linux-gnu/11/include/
MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
LDFLAGS = -fopenmp
Expand Down
2 changes: 1 addition & 1 deletion MAKE/Makefile.bsc_jetsontx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ endif
FLAGS =
# note: std was c++11
# note: testpackage settings missing
CXXFLAGS = -I$(HOME)/include -O3 -std=c++17 -W -Wall -pedantic -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fopenmp -march=native
CXXFLAGS = -I$(HOME)/include -O3 -std=c++20 -W -Wall -pedantic -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fopenmp -march=native

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
6 changes: 3 additions & 3 deletions MAKE/Makefile.carrington_gcc_openmpi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,14 +58,14 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 11.2.0
CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++17 -mavx -march=znver2 #-flto
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++17 -mavx -march=znver2 -DIONOSPHERE_SORTED_SUMS
CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++20 -mavx -march=znver2 #-flto
testpackage: CXXFLAGS = -O2 -g -ggdb -fopenmp -funroll-loops -std=c++20 -mavx -march=znver2 -DIONOSPHERE_SORTED_SUMS
CXXFLAGS += -Wall -Wextra -Wno-unused

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations

LDFLAGS = -lrt -lgfortran -std=c++17 -lgomp
LDFLAGS = -lrt -lgfortran -std=c++20 -lgomp
LIB_MPI = -lgomp -lmpi

#======== PAPI ==========
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.cubbli20
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ PAPI_FLAG =

FLAGS =
# Debug flags:
#CXXFLAGS = -g -funroll-loops -std=c++17 -fopenmp -W -Wall -pedantic -Wno-unused -fabi-version=0 -mavx
CXXFLAGS = -O3 -funroll-loops -std=c++17 -fopenmp -W -Wall -Wno-unused -fabi-version=0 -mavx
#CXXFLAGS = -g -funroll-loops -std=c++20 -fopenmp -W -Wall -pedantic -Wno-unused -fabi-version=0 -mavx
CXXFLAGS = -O3 -funroll-loops -std=c++20 -fopenmp -W -Wall -Wno-unused -fabi-version=0 -mavx

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.docker
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ endif
FLAGS =

#GNU flags:
CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -fabi-version=0 -mavx2
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0 -mavx
CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -fabi-version=0 -mavx2
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0 -mavx

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
2 changes: 1 addition & 1 deletion MAKE/Makefile.github_actions
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ CXXFLAGS += -DUSE_JEMALLOC -DJEMALLOC_NO_DEMANGLE
# mpi.h in c++ on Cray

FLAGS =
CXXFLAGS += -I ${GITHUB_WORKSPACE}/libraries/include -L ${GITHUB_WORKSPACE}/libraries/lib -march=native -O3 -std=c++17 -fopenmp -Wall -Wno-unused -fabi-version=0 -mavx -g -Wno-cast-function-type
CXXFLAGS += -I ${GITHUB_WORKSPACE}/libraries/include -L ${GITHUB_WORKSPACE}/libraries/lib -march=native -O3 -std=c++20 -fopenmp -Wall -Wno-unused -fabi-version=0 -mavx -g -Wno-cast-function-type
MATHFLAGS = # -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = # -fno-unsafe-math-optimizations
LDFLAGS = -L ${GITHUB_WORKSPACE}/libraries/lib -Wl,-rpath=${GITHUB_WORKSPACE}/libraries/lib -g
Expand Down
6 changes: 3 additions & 3 deletions MAKE/Makefile.hawk_gcc_mpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,10 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 9.2.0_bis
CXXFLAGS += -g -fopenmp -O3 -funroll-loops -std=c++17 -W -Wall -Wno-unused -fabi-version=0 -Wall -Wpedantic -mfma -march=native -mavx2
#CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -fabi-version=0 -mavx
CXXFLAGS += -g -fopenmp -O3 -funroll-loops -std=c++20 -W -Wall -Wno-unused -fabi-version=0 -Wall -Wpedantic -mfma -march=native -mavx2
#CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -fabi-version=0 -mavx
not_parallel_tools: CXXFLAGS += -march=native -mno-avx2 -mavx
testpackage: CXXFLAGS = -g -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0
testpackage: CXXFLAGS = -g -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0
not_parallel_tools: CC_BRAND_VERSION = 4.9.2-noavx2

MATHFLAGS = -ffast-math -fno-finite-math-only
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.hawk_gcc_openmpi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 9.2.0
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -fabi-version=0 -Wall -Wpedantic -mfma -march=native -mavx2
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -fabi-version=0 -Wall -Wpedantic -mfma -march=native -mavx2
not_parallel_tools: CXXFLAGS += -march=native -mno-avx2 -mavx
testpackage: CXXFLAGS = -g -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0
testpackage: CXXFLAGS = -g -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0
not_parallel_tools: CC_BRAND_VERSION = 4.9.2-noavx2

MATHFLAGS = -ffast-math -fno-finite-math-only
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.home
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ LNK = mpic++
FLAGS =
# note: std was c++11
# note: testpackage settings missing
CXXFLAGS = -I$(HOME)/include -I/usr/include -L$(HOME)/lib -L/usr/lib -O3 -funroll-loops -std=c++17 -W -Wall -pedantic -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fopenmp
#CXXFLAGS = -I$(HOME)/include -L$(HOME)/lib -O0 -funroll-loops -std=c++17 -W -Wall -pedantic -Wno-unused -Wno-unused-parameter -Wno-missing-braces -g -fopenmp
CXXFLAGS = -I$(HOME)/include -I/usr/include -L$(HOME)/lib -L/usr/lib -O3 -funroll-loops -std=c++20 -W -Wall -pedantic -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fopenmp
#CXXFLAGS = -I$(HOME)/include -L$(HOME)/lib -O0 -funroll-loops -std=c++20 -W -Wall -pedantic -Wno-unused -Wno-unused-parameter -Wno-missing-braces -g -fopenmp
MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
LDFLAGS = -L $(HOME)/lib
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.horakons
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ endif
#======== Flags =========

FLAGS =
CXXFLAGS = -O3 -funroll-loops -std=c++17 -fopenmp -W -Wall -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fabi-version=0 -mavx
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0 -mno-avx -mno-fma -fno-unsafe-math-optimizations
CXXFLAGS = -O3 -funroll-loops -std=c++20 -fopenmp -W -Wall -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fabi-version=0 -mavx
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0 -mno-avx -mno-fma -fno-unsafe-math-optimizations
CXXFLAGS += -DUSE_JEMALLOC -DJEMALLOC_NO_DEMANGLE -DPAPI_MEM
MATHFLAGS = -ffast-math
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.kstppd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ endif
#======== Flags =========

FLAGS =
CXXFLAGS = -O3 -funroll-loops -std=c++17 -fopenmp -W -Wall -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fabi-version=0 -mavx
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0 -mno-avx -mno-fma -fno-unsafe-math-optimizations
CXXFLAGS = -O3 -funroll-loops -std=c++20 -fopenmp -W -Wall -Wno-unused -Wno-unused-parameter -Wno-missing-braces -fabi-version=0 -mavx
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0 -mno-avx -mno-fma -fno-unsafe-math-optimizations
CXXFLAGS += -DUSE_JEMALLOC -DJEMALLOC_NO_DEMANGLE -DPAPI_MEM
MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
2 changes: 1 addition & 1 deletion MAKE/Makefile.lumi_cpeGnu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ endif
FLAGS =

#GNU flags:
CXXFLAGS += -g -ggdb -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -mfma -mavx2 -Wno-unused-parameter
CXXFLAGS += -g -ggdb -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -mfma -mavx2 -Wno-unused-parameter
testpackage: CXXFLAGS = -g -O2 -fopenmp -funroll-loops -std=c++20 -mno-fma -mno-avx2 -mno-avx

MATHFLAGS = -ffast-math -fno-finite-math-only
Expand Down
2 changes: 1 addition & 1 deletion MAKE/Makefile.lumi_cray
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 9.3.0
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -mfma -mavx2
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -mfma -mavx2
testpackage: CXXFLAGS = -g -O2 -fopenmp -funroll-loops -std=c++20 -mno-fma -mno-avx2 -mno-avx

MATHFLAGS = -ffast-math -fno-finite-math-only
Expand Down
2 changes: 1 addition & 1 deletion MAKE/Makefile.lumi_gnu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 9.3.0
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -mfma -mavx2
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -mfma -mavx2
testpackage: CXXFLAGS = -g -O2 -fopenmp -funroll-loops -std=c++20 -mno-fma -mno-avx2 -mno-avx

MATHFLAGS = -ffast-math -fno-finite-math-only
Expand Down
2 changes: 1 addition & 1 deletion MAKE/Makefile.lumi_gnu358
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 9.3.0
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -mfma -mavx2
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -mfma -mavx2
testpackage: CXXFLAGS = -g -O2 -fopenmp -funroll-loops -std=c++20 -mno-fma -mno-avx2 -mno-avx

MATHFLAGS = -ffast-math -fno-finite-math-only
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.mahti_gcc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 9.3.0
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -fabi-version=0 -march=native -mfma -mavx2 -fpermissive
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0 -mfma -fpermissive
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -fabi-version=0 -march=native -mfma -mavx2 -fpermissive
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0 -mfma -fpermissive


MATHFLAGS = -ffast-math -fno-finite-math-only
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.marconi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ endif
CC_BRAND = intel
CC_BRAND_VERSION = 17.0.1
# note: std was c++11
CXXFLAGS += -O2 -g -DMAX_VECTOR_SIZE=512 -xMIC-AVX512 -std=c++17 -qopenmp -ansi-alias
testpackage: CXXFLAGS = -O2 -g -DMAX_VECTOR_SIZE=512 -xMIC-AVX512 -std=c++17 -qopenmp -ansi-alias
CXXFLAGS += -O2 -g -DMAX_VECTOR_SIZE=512 -xMIC-AVX512 -std=c++20 -qopenmp -ansi-alias
testpackage: CXXFLAGS = -O2 -g -DMAX_VECTOR_SIZE=512 -xMIC-AVX512 -std=c++20 -qopenmp -ansi-alias

MATHFLAGS = #-ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = #-fno-unsafe-math-optimizations
Expand Down
2 changes: 1 addition & 1 deletion MAKE/Makefile.perlmutter_gcc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 11.2.0
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -mfma -mavx2
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -mfma -mavx2
testpackage: CXXFLAGS = -g -O2 -fopenmp -funroll-loops -std=c++20 -mno-fma -mno-avx2 -mno-avx

MATHFLAGS = -ffast-math -fno-finite-math-only
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.puhti_gcc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 9.1.0
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -fabi-version=0 -march=native -mfma -mavx512f
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0 -mfma
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -fabi-version=0 -march=native -mfma -mavx512f
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0 -mfma

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
4 changes: 2 additions & 2 deletions MAKE/Makefile.puhti_intel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ FLAGS =
#GNU flags:
CC_BRAND = intel
CC_BRAND_VERSION = 19.0.4
CXXFLAGS += -traceback -g -O3 -qopenmp -std=c++17 -W -Wall -Wno-unused -xHost -ipo -qopt-zmm-usage=high
testpackage: CXXFLAGS = -g -traceback -O2 -qopenmp -std=c++17 -W -Wno-unused -xHost -ipo
CXXFLAGS += -traceback -g -O3 -qopenmp -std=c++20 -W -Wall -Wno-unused -xHost -ipo -qopt-zmm-usage=high
testpackage: CXXFLAGS = -g -traceback -O2 -qopenmp -std=c++20 -W -Wno-unused -xHost -ipo

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
6 changes: 3 additions & 3 deletions MAKE/Makefile.ukko_gcc_openmpi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,13 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 11.2.0
CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++17 -W -Wall -Wno-unused -mavx -march=znver2 #-flto
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++17 -mavx -march=znver2
CXXFLAGS += -O3 -fopenmp -funroll-loops -std=c++20 -W -Wall -Wno-unused -mavx -march=znver2 #-flto
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++20 -mavx -march=znver2

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations

LDFLAGS = -lrt -lgfortran -std=c++17 -lgomp
LDFLAGS = -lrt -lgfortran -std=c++20 -lgomp
LIB_MPI = -lgomp -lmpi

#======== PAPI ==========
Expand Down
6 changes: 3 additions & 3 deletions MAKE/Makefile.vorna_gcc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,13 @@ FLAGS =
#GNU flags:
CC_BRAND = gcc
CC_BRAND_VERSION = 10.2.0
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++17 -mavx
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++17 -mavx
CXXFLAGS += -g -O3 -fopenmp -funroll-loops -std=c++20 -mavx
testpackage: CXXFLAGS = -O2 -fopenmp -funroll-loops -std=c++20 -mavx
CXXFLAGS += -Wall -Wextra -Wno-unused

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
LDFLAGS = -lrt -lgfortran -std=c++17
LDFLAGS = -lrt -lgfortran -std=c++20
LIB_MPI = -lgomp

#======== PAPI ==========
Expand Down
6 changes: 3 additions & 3 deletions MAKE/Makefile.yann
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ LNK = mpic++
PAPI_FLAG =

FLAGS =
#CXXFLAGS = -I $(HOME)/include -L $(HOME)/lib -g -funroll-loops -std=c++0x -fopenmp -W -Wall -pedantic -Wno-unused -fabi-version=0 -mavx
CXXFLAGS += -g3 -ggdb -O3 -funroll-loops -std=c++17 -fopenmp -W -Wall -Wno-unused -fabi-version=0 -mfma -mavx2 -Wno-unknown-pragmas -Wno-sign-compare
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++17 -fabi-version=0 -mno-avx -mno-fma -fno-unsafe-math-optimizations
#CXXFLAGS = -I $(HOME)/include -L $(HOME)/lib -g -funroll-loops -std=c++20 -fopenmp -W -Wall -pedantic -Wno-unused -fabi-version=0 -mavx
CXXFLAGS += -g3 -ggdb -O3 -funroll-loops -std=c++20 -fopenmp -W -Wall -Wno-unused -fabi-version=0 -mfma -mavx2 -Wno-unknown-pragmas -Wno-sign-compare
testpackage: CXXFLAGS = -g -ggdb -O2 -fopenmp -funroll-loops -std=c++20 -fabi-version=0 -mno-avx -mno-fma -fno-unsafe-math-optimizations

MATHFLAGS = -ffast-math -fno-finite-math-only
testpackage: MATHFLAGS = -fno-unsafe-math-optimizations
Expand Down
4 changes: 0 additions & 4 deletions common.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,9 +188,6 @@ namespace CellParams {
FSGRID_BOUNDARYTYPE, /*!< Boundary type of this cell, as stored in the fsGrid */
CELLID, /*! < DCCRG cell index */
REFINEMENT_LEVEL, /*! < Refinement level */
AMR_TRANSLATE_COMM_X, /*! < Flag to include this cell in AMR pre-translate communication */
AMR_TRANSLATE_COMM_Y, /*! < Flag to include this cell in AMR pre-translate communication */
AMR_TRANSLATE_COMM_Z, /*! < Flag to include this cell in AMR pre-translate communication */
CONNECTION, /*!< Magnetic connection. See TracingPointConnectionType for assigned values. */
CONNECTION_FW_X, /*!< Endpoint x (forward-propagated) for the magnetic connection tracer*/
CONNECTION_FW_Y, /*!< Endpoint y (forward-propagated) for the magnetic connection tracer*/
Expand Down Expand Up @@ -502,7 +499,6 @@ struct globalflags {
static bool writeRestart; /*!< Global flag raised to true if a restart writing is needed (without bailout). NOTE: used only by MASTER_RANK in vlasiator.cpp. */
static bool balanceLoad; /*!< Global flag raised to true if a load balancing is needed. NOTE: used only by MASTER_RANK in vlasiator.cpp. */
static bool doRefine; /*!< Global flag raised to true if a re-refine is needed. NOTE: used only by MASTER_RANK in vlasiator.cpp. */
static int AMRstencilWidth; /*!< Global variable used for the extended AMR stencil width */
static bool ionosphereJustSolved; /*!< Flag used to notify that the ionosphere has been freshly solved, used to check whether the Vlasov boundary/bulk forcing need updating. */
};

Expand Down
7 changes: 0 additions & 7 deletions datareduction/datareducer.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,13 +237,6 @@ void initializeDataReducers(DataReducer * outputReducer, DataReducer * diagnosti
continue;
}
}
if(P::systemWriteAllDROs || lowercase == "vg_amr_translate_comm") { // Flag for AMR translation communication
outputReducer->addOperator(new DRO::DataReductionOperatorCellParams("vg_amr_translate_comm",CellParams::AMR_TRANSLATE_COMM_X,3));
outputReducer->addMetadata(outputReducer->size()-1,"","","AMRtranslate","1.0");
if(!P::systemWriteAllDROs) {
continue;
}
}
if(P::systemWriteAllDROs || lowercase == "fg_rhom") { // Overall mass density (summed over all populations)
outputReducer->addOperator(new DRO::DataReductionOperatorFsGrid("fg_rhom",[](
FsGrid< std::array<Real, fsgrids::bfield::N_BFIELD>, FS_STENCIL_WIDTH> & perBGrid,
Expand Down
Loading
Loading