Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preparations for singularity support #167

Open
wants to merge 45 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
dcc1c27
First step to get away from the clumsy handling of tool scripts, code…
vinjana Sep 30, 2023
285410e
Dropped BEJob toolID parameter. Should be taken from job. Other refac…
vinjana Sep 30, 2023
8d91ecd
Some refactorings.
vinjana Sep 30, 2023
3336935
Some progress.
vinjana Sep 30, 2023
eadc97c
Some changes in response to review.
vinjana Oct 31, 2023
91e6c5a
Some simplifications. Fixed last tests in ApptainerCommandBuilder.
vinjana Oct 31, 2023
05e8c8b
Added linting option to gradle.
vinjana Oct 31, 2023
3224a3b
Rolled back the Command->BECommand renaming.
vinjana Oct 31, 2023
c6e59a9
Some improvements of mount handling for apptainer.
vinjana Nov 7, 2023
ab03208
Add setting of instance-level and call-level image name.
vinjana Nov 8, 2023
d315da9
Refactored ApptainerCommandBuilder to use fluent API. Added some fiel…
vinjana Nov 10, 2023
331d6a2
Smarter command quoting.
vinjana Nov 10, 2023
13ac4ad
Added explicitly set variables, needed by Roddy.
vinjana Nov 15, 2023
f2f6f74
Answer to @coderabbitai.
vinjana Nov 16, 2023
f4779e2
Some improvements and tests.
vinjana Nov 17, 2023
2ef8b71
Fix reoccurring test problem with hard-coded years.
vinjana Jan 11, 2024
01f19ec
Fixed forgotten null tests
vinjana Jan 11, 2024
0e9549b
Fix date bug in LSF date parser and fixed the test itself.
vinjana Jan 26, 2024
e3b2bed
Fixed hashCode and equals bug.
vinjana Jan 26, 2024
3607a77
Better test documentation.
vinjana Jan 26, 2024
d0595ab
Changed some Precondition.checkArgument(x != null) to Precondition.ch…
vinjana Jan 29, 2024
ccb79f7
Some refactorings and a change to allow SLURM code submission.
vinjana Feb 16, 2024
c7e79ec
Added tests for code-submission via LSF and SLURM. Fixed environment …
vinjana Feb 16, 2024
6473ec9
Bumped some Apache library versions.
vinjana Feb 19, 2024
199941c
Fixed script generation to pipe into SLURM and LSF commands.
vinjana Feb 21, 2024
a719755
Introduced EscapableString and adapted Apptainer and other code to us…
vinjana Feb 26, 2024
6e2200b
Turn on pmd checker in CI
vinjana Feb 26, 2024
ff5f7bc
Some fine tuning. Renamed test files.
vinjana Feb 27, 2024
5ace7e7
Added test of dependency parameters. In the course, removed unused de…
vinjana Feb 27, 2024
f276f3e
By default, copied envvars should not be quoted in ApptainerCommandBu…
vinjana Feb 27, 2024
ad6cdb9
Some fixes.
vinjana Mar 13, 2024
d621348
Change sbatch-parameter --propagate=none to --propagate=NONE.
vinjana Mar 13, 2024
424f771
Bugfix --propagate=NONE for SLURM
vinjana Mar 13, 2024
ced9cc6
Added a test and some documentation.
vinjana Mar 15, 2024
b429ab8
Small README edits.
vinjana Mar 15, 2024
e609b85
Type fix in readme
vinjana Mar 15, 2024
8f6f34d
Update gradle to 8.3, usable by GitHub CodeQL.
vinjana Apr 5, 2024
c30694c
Added cyclonedx tasks for SBOM generation.
vinjana Apr 17, 2024
61227d6
Use java.toolchain in Gradle.
vinjana Apr 18, 2024
3a0423f
Added cyclonedx SBOM and test-repots as tarballs to github release
vinjana Apr 25, 2024
4276492
Moved AnyEscapableString hierarchy into RoddyToolLib.
vinjana Apr 25, 2024
7282481
Fix bug in SlurmSubmissionCommand (missing temp-dependencies). Rename…
vinjana Jul 4, 2024
99b9d03
Bumped guava to 33.2.1-jre.
vinjana Jul 5, 2024
ce4979d
Added test for SlurmSubmissionCommand.assemblyDependencyParameter(<no…
vinjana Jul 8, 2024
69b1e7a
Code simplification after review.
vinjana Jul 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ version: 2.1
executors:
docker-executor:
docker:
- image: cimg/openjdk:8.0.322
- image: cimg/openjdk:8.0.322 # Version for Gradle 5.1 should be 11.0.18?
vinjana marked this conversation as resolved.
Show resolved Hide resolved
Gordi marked this conversation as resolved.
Show resolved Hide resolved
jobs: # a collection of steps
build-and-test:
environment:
Expand Down
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,11 @@ To start the integration tests, please fill in host and user settings (password

# Change Logs

* 0.2.0
- major: CommandI classes to model commands and code
- minor: ApptainerCommmandBuilder to create a wrapping command to run code in containers
- patch: Library updates

* 0.1.4
- patch: Changed the conversion of EpochSeconds to ZonedDateTime for the value "0".
* For eligibleTime it is set to null
Expand Down
51 changes: 28 additions & 23 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ apply plugin: 'maven'
group = "com.github.theroddywms"

mainClassName = 'de.dkfz.roddy.BEIntegrationTestStarter'
sourceCompatibility = '1.8'
targetCompatibility = '1.8'
sourceCompatibility = JavaVersion.VERSION_1_8
targetCompatibility = JavaVersion.VERSION_1_8

/*
* Gets the version name from the current Git tag. If the current commit is not tagged,
Expand All @@ -30,10 +30,12 @@ def getVersionName() {

rootProject.version = getVersionName()

compileJava.options.compilerArgs = ["-release", targetCompatibility]

// Set -Pchecked on the command line to change this compiler parameter.
if (!project.hasProperty("checked") || !project.checked) {
compileJava.options.compilerArgs = ["-Xlint:unchecked"]
compileGroovy.options.compilerArgs = ["-Xlint:unchecked"]
if (project.hasProperty("checked") && !project.checked) {
compileJava.options.compilerArgs += ["-Xlint:unchecked", "-Xlint:deprecation"]
compileGroovy.options.compilerArgs += ["-Xlint:unchecked", "-Xlint:deprecation"]
vinjana marked this conversation as resolved.
Show resolved Hide resolved
}

repositories {
Expand All @@ -46,24 +48,27 @@ configurations.all {
}

dependencies {
compile 'com.fasterxml.jackson.core:jackson-core:2.9.7'
compile 'com.fasterxml.jackson.core:jackson-annotations:2.9.7'
compile 'com.fasterxml.jackson.core:jackson-databind:2.9.7'
compile 'org.codehaus.groovy:groovy-all:2.4.9'
testCompile 'junit:junit:4.12'
testCompile 'org.spockframework:spock-core:1.1-groovy-2.4'
testRuntime "net.bytebuddy:byte-buddy:1.6.5"
testRuntime "org.objenesis:objenesis:2.5.1"
testCompile 'com.google.guava:guava:23.0'
testCompile 'org.slf4j:slf4j-nop:1.7.25'
compile 'org.slf4j:slf4j-api:1.7.25'
compile group: 'org.apache.commons', name: 'commons-io', version: '1.3.2'
compile group: 'commons-codec', name: 'commons-codec', version: '1.9'
compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.3'
compile group: 'commons-cli', name: 'commons-cli', version: '1.2'
compile group: 'org.apache.commons', name: 'commons-text', version: '1.1'
compile 'com.google.guava:guava:23.0'
compile 'com.github.theroddywms:RoddyToolLib:2.4.0'
implementation 'com.fasterxml.jackson.core:jackson-core:2.9.7'
implementation 'com.fasterxml.jackson.core:jackson-annotations:2.9.7'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.12.7.1'
implementation 'org.codehaus.groovy:groovy-all:2.4.21'
testImplementation 'junit:junit:4.13.1'
testImplementation 'org.spockframework:spock-core:1.1-groovy-2.4'
testRuntimeOnly "net.bytebuddy:byte-buddy:1.6.5"
testRuntimeOnly "org.objenesis:objenesis:2.5.1"
testImplementation 'com.google.guava:guava:30.0-android'
testImplementation 'org.slf4j:slf4j-nop:1.7.25'
implementation 'org.slf4j:slf4j-api:1.7.25'
implementation group: 'org.apache.commons', name: 'commons-io', version: '1.3.2'
implementation group: 'commons-codec', name: 'commons-codec', version: '1.13'
implementation group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.13'
implementation group: 'commons-cli', name: 'commons-cli', version: '1.2'
implementation group: 'org.apache.commons', name: 'commons-text', version: '1.1'
implementation 'com.google.guava:guava:30.0-android'
implementation 'com.github.theroddywms:RoddyToolLib:2.4.0'

// Runtime retention of @NonNull and annotations.
implementation 'com.github.bbottema:jetbrains-runtime-annotations:1.0.1'
}

task writePom {
Expand Down
2 changes: 1 addition & 1 deletion src/main/groovy/de/dkfz/roddy/TestExecutionService.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ package de.dkfz.roddy

import de.dkfz.roddy.execution.BEExecutionService
import de.dkfz.roddy.execution.io.LocalExecutionHelper
import de.dkfz.roddy.execution.jobs.Command
import de.dkfz.roddy.execution.io.ExecutionResult
import de.dkfz.roddy.execution.jobs.Command
import de.dkfz.roddy.tools.BashUtils
import groovy.transform.CompileStatic

Expand Down
25 changes: 25 additions & 0 deletions src/main/groovy/de/dkfz/roddy/config/EmptyResourceSet.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
* Copyright (c) 2023 German Cancer Research Center (Deutsches Krebsforschungszentrum, DKFZ).
*
* Distributed under the MIT License (license terms are at https://www.github.com/TheRoddyWMS/Roddy/LICENSE.txt).
*/

package de.dkfz.roddy.config


import de.dkfz.roddy.tools.TimeUnit
import groovy.transform.CompileStatic

@CompileStatic
class EmptyResourceSet extends ResourceSet {
EmptyResourceSet() {
super(null,
null,
null,
null,
null as TimeUnit,
null,
null,
null)
}
}
Loading