Skip to content
This repository has been archived by the owner on Nov 27, 2024. It is now read-only.

JDBetteridge/spindle cache #729

Draft
wants to merge 38 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
877def6
Remove hash_comm and add per-comm caches
JDBetteridge Aug 8, 2024
767a21f
Update the TestDiskCachedDecorator test for collective
JDBetteridge Aug 8, 2024
e35a589
Refactor disk_cached wrapper and add warning
JDBetteridge Aug 8, 2024
64f5616
Refactor memory/disk caches in order to remove use of id() in GlobalK…
JDBetteridge Aug 8, 2024
9a31c3d
Rethink memory only cache for non-broadcastable values
JDBetteridge Aug 8, 2024
3f030af
New caching implementation, WIP needs tidying and better tests
JDBetteridge Aug 12, 2024
6aa6554
WIP
JDBetteridge Aug 12, 2024
6dec230
Just notes
JDBetteridge Aug 13, 2024
a7451fc
Remove cached(object) and lint
JDBetteridge Aug 14, 2024
c1a48c9
WIP, need to fix remaining cache tests
JDBetteridge Aug 14, 2024
d1bacf0
Round things out and fix remaining tests
JDBetteridge Aug 14, 2024
a7f43d4
CI debugging
JDBetteridge Aug 15, 2024
783ab12
CI debugging, ssh in?
JDBetteridge Aug 15, 2024
3d13c4f
CI debugging, a fix?
JDBetteridge Aug 15, 2024
756e29e
CI debugging, a fix for 3.12
JDBetteridge Aug 15, 2024
f3decc4
.
JDBetteridge Aug 15, 2024
723468b
.
JDBetteridge Aug 15, 2024
474058d
.
JDBetteridge Aug 15, 2024
aab930a
Remove custom implementation of @cached_property, use the Python(>=3.…
JDBetteridge Aug 15, 2024
f512249
Add a sreial cache with the same interface
JDBetteridge Aug 15, 2024
c12194c
Caching fixes and tweaks
JDBetteridge Aug 16, 2024
c74c7bb
Lint
JDBetteridge Aug 16, 2024
f52449c
Add instrumentation to caches
JDBetteridge Aug 19, 2024
0c2b088
Remove get_so from Compiler base class
JDBetteridge Aug 19, 2024
0cf63f9
Remove comm from Compiler class
JDBetteridge Aug 19, 2024
7ca74eb
Use caching.disk_only_cache for make_so
JDBetteridge Aug 19, 2024
91c25b8
A better solution for wrapping make_so in a disk cache
JDBetteridge Aug 20, 2024
919523a
Update pyop2/caching.py
dham Aug 21, 2024
b1ecd92
WIP: may have reintroduced deadlocks
JDBetteridge Aug 22, 2024
c6225f0
WIP: Fixed the deadlock
JDBetteridge Aug 22, 2024
13bc76c
Add event decorators
JDBetteridge Aug 23, 2024
9bdc418
WIP: Fixing and tidying
JDBetteridge Aug 23, 2024
aeddb09
Add additional cache tests
JDBetteridge Aug 23, 2024
f4f4194
Fix stats printing
JDBetteridge Aug 24, 2024
1c69860
Handle multiple VENVs + instrumenting
JDBetteridge Aug 27, 2024
8d0f319
Add PYOP2_SPMD_STRICT environment variable for checking MPI correctness
JDBetteridge Aug 27, 2024
e8f0859
Actually put mismatched kernels in the directory created
JDBetteridge Sep 1, 2024
015cec3
Put all shared libraries i a sincle cache directory
JDBetteridge Sep 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 29 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
# Don't immediately kill all if one Python version fails
fail-fast: false
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11']
python-version: ['3.9', '3.10', '3.11', '3.12']
env:
CC: mpicc
PETSC_DIR: ${{ github.workspace }}/petsc
Expand All @@ -28,6 +28,13 @@ jobs:
timeout-minutes: 60

steps:
- name: Setup tmate session
uses: mxschmitt/action-tmate@v3
with:
detached: true
limit-access-to-actor: true
timeout-minutes: 15

- name: Install system dependencies
shell: bash
run: |
Expand Down Expand Up @@ -58,15 +65,15 @@ jobs:
working-directory: ${{ env.PETSC_DIR }}/src/binding/petsc4py
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade wheel 'cython<3' numpy
python -m pip install --upgrade wheel cython numpy
python -m pip install --no-deps .

- name: Checkout PyOP2
uses: actions/checkout@v2
with:
path: PyOP2

- name: Install PyOP2
- name: Install PyOP2 dependencies
shell: bash
working-directory: PyOP2
run: |
Expand All @@ -76,7 +83,21 @@ jobs:
python -m pip install pulp
python -m pip install -U flake8
python -m pip install -U pytest-timeout
python -m pip install .

- name: Install PyOP2 (Python <3.12)
if: ${{ matrix.python-version != '3.12' }}
shell: bash
working-directory: PyOP2
run: python -m pip install .

# Not sure if this is a bug in setuptools or something PyOP2 is doing wrong
- name: Install PyOP2 (Python == 3.12)
if: ${{ matrix.python-version == '3.12' }}
shell: bash
working-directory: PyOP2
run: |
python -m pip install -U setuptools
python setup.py install

- name: Run linting
shell: bash
Expand All @@ -86,7 +107,10 @@ jobs:
- name: Run tests
shell: bash
working-directory: PyOP2
run: pytest --tb=native --timeout=480 --timeout-method=thread -o faulthandler_timeout=540 -v test
run: |
# Running parallel test cases separately works around a bug in pytest-mpi
pytest -k "not parallel" --tb=native --timeout=480 --timeout-method=thread -o faulthandler_timeout=540 -v test
mpiexec -n 3 pytest -k "parallel[3]" --tb=native --timeout=480 --timeout-method=thread -o faulthandler_timeout=540 -v test
timeout-minutes: 10

- name: Build documentation
Expand Down
Loading
Loading