-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create the CIs for Icy and deepImageJ using JDLL #82
Draft
carlosuc3m
wants to merge
86
commits into
bioimage-io:main
Choose a base branch
from
carlosuc3m:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
86 commits
Select commit
Hold shift + click to select a range
38df8e7
start adding workflow for icy
carlosuc3m fb0477c
update jdll version
carlosuc3m 298bfc1
listen to folder to trigger workflow
carlosuc3m 0ce6e42
add class to download engines, different for Icy and DIJ
carlosuc3m 5aeb5c4
download engines from action
carlosuc3m 4acdc03
install first
carlosuc3m 4dbed35
correct error in syntax
carlosuc3m be38fa1
remove problematic code
carlosuc3m f8145d1
check the directory
carlosuc3m cfccb6b
correct directory
carlosuc3m 0a4b3e4
remove unnecessary classes for logging
carlosuc3m 6ed2651
do not trigger build
carlosuc3m af205f6
avoid triggering it while developping
carlosuc3m fbc24c1
correct name of folder
carlosuc3m 10268b9
correct dir name
carlosuc3m a9b76be
add useful prints
carlosuc3m 0bf128a
ensure the change happens on every subfodler
carlosuc3m 305374e
add job that runs models
carlosuc3m 4bb3619
needs previous run
carlosuc3m 41d53e3
correct where to put needs
carlosuc3m 132dfa7
just a single job
carlosuc3m 239b1d6
update jdll version
carlosuc3m d4aad65
do two directories back
carlosuc3m b9a57fb
check the artifacts that are produced
carlosuc3m ed329cc
add part to upload changesusing python (python script still missing)
carlosuc3m 03c8b8e
start creating the python code to upload dij and icy tests
carlosuc3m f2877b3
keep working towards convergence of CI
carlosuc3m 6a705db
separate the tests for icy and deepimagej
carlosuc3m 82fd7b6
install every engine for Icy
carlosuc3m 41c2c79
do not fail when an engine is not properly installed
carlosuc3m 313145e
remove bioimage.io python deps
carlosuc3m 3d332bf
correct code to make the results depend on software and version
carlosuc3m 39a9fd3
renew the way tests are created
carlosuc3m 9c8380d
keep improving the java part of testing
carlosuc3m 346fbe8
stabilize java part
carlosuc3m 8da01d5
create script to retrieve the dij and icy versions
carlosuc3m 027e1d7
keep iterating
carlosuc3m 2964cb6
add code to understand JDLL and java software versions tested
carlosuc3m 950b1c9
print version so github action can use it
carlosuc3m 632e15e
update actions to use new script retrieving version dict
carlosuc3m 2733da6
add installation of deps
carlosuc3m e4a03d6
corect small details
carlosuc3m e1e904b
add listener to another file
carlosuc3m 7af4a00
try to change something
carlosuc3m 8f4139e
debug
carlosuc3m 72f985a
debug
carlosuc3m a443fa1
debug
carlosuc3m 4ce9640
try new changes
carlosuc3m d7fd1c1
try more changes
carlosuc3m df64bd6
more changes
carlosuc3m 1344fa5
use a list of entries instead of a dict
carlosuc3m 3d29be5
redo map
carlosuc3m 1b016c7
stabilize version management
carlosuc3m f03fff9
che k the dir
carlosuc3m 2904eb6
move to the correct dir to execute java code
carlosuc3m c832bab
orint jarfile
carlosuc3m de04824
correct paths again
carlosuc3m 1fbc127
check if jarfile persits
carlosuc3m 8c4b5ca
preserve jarfile name
carlosuc3m e333e6a
add forgotten id
carlosuc3m 707eb22
need to jobs
carlosuc3m 7a8892c
install deps
carlosuc3m 8a17d95
correct version number
carlosuc3m 4cff69f
update python code
carlosuc3m f397ab5
run as modules to avoid import errors
carlosuc3m 241d6dd
check if this change eas the error
carlosuc3m e49dbce
improve robustness
carlosuc3m 6be07cf
create the collection backoffice plus dependencies
carlosuc3m b521fe1
improving naming of the paths of test summaries
carlosuc3m fc4cb0d
correct installation of backoffice
carlosuc3m 89951e4
Merge branch 'bioimage-io:main' into main
carlosuc3m 0ff8c70
update gitignore again and remove silly print
carlosuc3m d700c34
provide the arguments correctly
carlosuc3m a64cc1d
correct error that was failing passed tests
carlosuc3m b90e825
fail the workflow on purpose
carlosuc3m 8f4ea3e
fail the other workflow on purpose
carlosuc3m 7f2c972
test fail
carlosuc3m 7a04b2e
restore build yaml
carlosuc3m b5a4014
restore default
carlosuc3m 7195192
remove intentional fail
carlosuc3m 06da2c4
remove default path forgotten
carlosuc3m 1ee4bcb
make the details more understandable
carlosuc3m 0fe821e
remove intentional fail
carlosuc3m efd911c
also catch errors
carlosuc3m 8924cae
at the moment only use one version per engine
carlosuc3m 854f356
correct a couple of details of the action
carlosuc3m File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
name: check compatibility deepimagej | ||
|
||
concurrency: deepimagej | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
paths: | ||
- .github/workflows/check_compatibility_deepimagej.yaml | ||
- scripts/get_java_software_versions.py | ||
- scripts/check_compatibility_java_software/**/* | ||
workflow_dispatch: | ||
schedule: | ||
- cron: 0 1 * * * # update compatibility once a day | ||
|
||
jobs: | ||
generate-version-dict: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
matrix: ${{ steps.set-dict.outputs.matrix }} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: Set up Python | ||
uses: actions/setup-python@v4 | ||
with: | ||
python-version: '3.x' | ||
- name: Install deps | ||
run: pip install requests==2.28.2 beautifulsoup4==4.12.3 packaging==23.0 | ||
- name: Generate dict | ||
id: set-dict | ||
run: | | ||
matrix_output=$(python scripts/get_java_software_versions.py deepimagej) | ||
echo "matrix=${matrix_output}" >> $GITHUB_OUTPUT | ||
- name: Versions matrix | ||
run: echo '${{ steps.set-dict.outputs.matrix }}' | ||
|
||
run: | ||
needs: generate-version-dict | ||
runs-on: ubuntu-latest | ||
strategy: | ||
matrix: ${{ fromJson(needs.generate-version-dict.outputs.matrix) }} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: actions/checkout@v3 | ||
with: | ||
repository: bioimage-io/collection-bioimage-io | ||
ref: gh-pages | ||
path: bioimageio-gh-pages | ||
- name: Setup Maven Action | ||
uses: stCarolas/[email protected] | ||
with: | ||
java-version: 11 | ||
maven-version: 3.9.5 | ||
- name: Build with Maven | ||
run: | | ||
cd scripts/check_compatibility_java_software | ||
mvn clean install org.apache.maven.plugins:maven-shade-plugin:3.2.4:shade -Dshade.mainClass=io.bioimage.modelrunner.ci.ContinuousIntegration | ||
- name: Get jar file name | ||
id: get-jar | ||
run: | | ||
cd scripts/check_compatibility_java_software | ||
jarfile=$(mvn -q exec:exec -Dexec.executable=echo -Dexec.args='${project.build.finalName}.jar') | ||
if [ -z "$jarfile" ]; then | ||
echo "Failed to get jar file name" | ||
exit 1 | ||
fi | ||
if [ ! -f "target/$jarfile" ]; then | ||
echo "Jar file not found: target/$jarfile" | ||
exit 1 | ||
fi | ||
echo "Jar file found: target/$jarfile" | ||
echo "jarfile=$jarfile" >> $GITHUB_OUTPUT | ||
- name: Download engines | ||
run: | | ||
java -cp "scripts/check_compatibility_java_software/target/${{ steps.get-jar.outputs.jarfile }}" io.bioimage.modelrunner.ci.DownloadEngines deepimagej | ||
- name: Run models | ||
run: | | ||
java -cp "scripts/check_compatibility_java_software/target/${{ steps.get-jar.outputs.jarfile }}" io.bioimage.modelrunner.ci.ContinuousIntegration deepimagej ${{ matrix.key }} | ||
- name: Upload artifacts | ||
uses: actions/upload-artifact@v3 | ||
with: | ||
path: test_summaries_deepimagej_${{ matrix.key }} | ||
|
||
sendreport: | ||
runs-on: ubuntu-latest | ||
needs: [generate-version-dict, run] | ||
strategy: | ||
matrix: ${{ fromJson(needs.generate-version-dict.outputs.matrix) }} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: Set up Python | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: '3.10' | ||
- name: Get test results | ||
uses: actions/download-artifact@v3 | ||
with: | ||
path: test_summaries_deepimagej_${{ matrix.key }} | ||
- name: Install deps | ||
run: | | ||
pip install . | ||
- name: Send deepimagej ${{ matrix.key }} tests | ||
shell: bash -l {0} | ||
run: python scripts/check_compatibility_java_software.py deepimagej ${{ matrix.key }} --summaries_dir "test_summaries_deepimagej_${{ matrix.key }}/artifact" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
name: check compatibility icy | ||
|
||
concurrency: icy | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
paths: | ||
- .github/workflows/check_compatibility_icy.yaml | ||
- scripts/get_java_software_versions.py | ||
- scripts/check_compatibility_java_software/**/* | ||
workflow_dispatch: | ||
schedule: | ||
- cron: 0 1 * * * # update compatibility once a day | ||
|
||
jobs: | ||
generate-version-dict: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
matrix: ${{ steps.set-dict.outputs.matrix }} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: Set up Python | ||
uses: actions/setup-python@v4 | ||
with: | ||
python-version: '3.x' | ||
- name: Install deps | ||
run: pip install requests==2.28.2 beautifulsoup4==4.12.3 packaging==23.0 | ||
- name: Generate dict | ||
id: set-dict | ||
run: | | ||
matrix_output=$(python scripts/get_java_software_versions.py icy) | ||
echo "matrix=${matrix_output}" >> $GITHUB_OUTPUT | ||
- name: Versions matrix | ||
run: echo '${{ steps.set-dict.outputs.matrix }}' | ||
|
||
run: | ||
needs: generate-version-dict | ||
runs-on: ubuntu-latest | ||
strategy: | ||
matrix: ${{ fromJson(needs.generate-version-dict.outputs.matrix) }} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: actions/checkout@v3 | ||
with: | ||
repository: bioimage-io/collection-bioimage-io | ||
ref: gh-pages | ||
path: bioimageio-gh-pages | ||
- name: Setup Maven Action | ||
uses: stCarolas/[email protected] | ||
with: | ||
java-version: 11 | ||
maven-version: 3.9.5 | ||
- name: Build with Maven | ||
run: | | ||
cd scripts/check_compatibility_java_software | ||
mvn clean install org.apache.maven.plugins:maven-shade-plugin:3.2.4:shade -Dshade.mainClass=io.bioimage.modelrunner.ci.ContinuousIntegration | ||
- name: Get jar file name | ||
id: get-jar | ||
run: | | ||
cd scripts/check_compatibility_java_software | ||
jarfile=$(mvn -q exec:exec -Dexec.executable=echo -Dexec.args='${project.build.finalName}.jar') | ||
if [ -z "$jarfile" ]; then | ||
echo "Failed to get jar file name" | ||
exit 1 | ||
fi | ||
if [ ! -f "target/$jarfile" ]; then | ||
echo "Jar file not found: target/$jarfile" | ||
exit 1 | ||
fi | ||
echo "Jar file found: target/$jarfile" | ||
echo "jarfile=$jarfile" >> $GITHUB_OUTPUT | ||
- name: Download engines | ||
run: | | ||
java -cp "scripts/check_compatibility_java_software/target/${{ steps.get-jar.outputs.jarfile }}" io.bioimage.modelrunner.ci.DownloadEngines icy | ||
- name: Run models | ||
run: | | ||
java -cp "scripts/check_compatibility_java_software/target/${{ steps.get-jar.outputs.jarfile }}" io.bioimage.modelrunner.ci.ContinuousIntegration icy ${{ matrix.key }} | ||
- name: Upload artifacts | ||
uses: actions/upload-artifact@v3 | ||
with: | ||
path: test_summaries_icy_${{ matrix.key }} | ||
|
||
sendreport: | ||
runs-on: ubuntu-latest | ||
needs: [generate-version-dict, run] | ||
strategy: | ||
matrix: ${{ fromJson(needs.generate-version-dict.outputs.matrix) }} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: Set up Python | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: '3.10' | ||
- name: Get test results | ||
uses: actions/download-artifact@v3 | ||
with: | ||
path: test_summaries_icy_${{ matrix.key }} | ||
- name: Install deps | ||
run: | | ||
pip install . | ||
- name: Send icy ${{ matrix.key }} tests | ||
shell: bash -l {0} | ||
run: python scripts/check_compatibility_java_software.py icy ${{ matrix.key }} --summaries_dir "test_summaries_icy_${{ matrix.key }}/artifact" | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could you maybe reuse https://github.com/bioimage-io/collection/blob/main/scripts/upload_reports.py here instead?