Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test out the preview version of the next GKL (DO NOT MERGE) #8712

Open
wants to merge 23 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
df9d19a
Add in the preview GKL build as a dependency
droazen Mar 4, 2024
5e6bd62
Added the java code to wrap the IntelPDPairHMM implementation for DRA…
jamesemery Mar 6, 2024
fe4a60d
Switch to gkl-0.8.11-3-g26de538-SNAPSHOT.jar
droazen Apr 24, 2024
1952912
Switch to gkl-2024_05_31, incorporate Intel patch to VectorLoglessPai…
droazen May 31, 2024
b1d7994
Temporarily revert Intel patch to VectorLoglessPairPDHMM.java, for te…
droazen Jul 19, 2024
4bd2d71
Restore one-line Intel patch to VectorLoglessPairPDHMM
droazen Jul 22, 2024
5997b30
added some debugging for figuring out what is going on with the gkl m…
jamesemery Jul 22, 2024
4ee399b
error
jamesemery Jul 22, 2024
cacf20d
adding quick remote test
jamesemery Jul 23, 2024
62bc0ce
;
jamesemery Jul 23, 2024
0217486
of course i can't run this anywhere
jamesemery Jul 23, 2024
e5dc0a4
piggybacking off of one that may have worked past?
jamesemery Jul 23, 2024
54dff49
more permissions
jamesemery Jul 23, 2024
fecf938
why didn't that last one work?
jamesemery Jul 23, 2024
bc68361
group
jamesemery Jul 23, 2024
c366995
restart? hopefully transient
jamesemery Jul 24, 2024
f2fb477
Refactored the interface to the new GKL JNI to batch submissions to a…
jamesemery Jul 25, 2024
a2d32bd
two tweeks and one bug
jamesemery Jul 25, 2024
545907f
yet more guardrails
jamesemery Jul 25, 2024
a67d8b3
assert failure
jamesemery Jul 26, 2024
cbd4bde
testing limits for batches
jamesemery Jul 29, 2024
a1a431b
Switch to latest GKL build (2024-08-08)
droazen Aug 8, 2024
7874846
update to test new jar
lbergelson Sep 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ funcotator_tmp
#Eclipse files
/bin/
.project
*.jar
#*.jar

#JENV local file
.java-version
Expand Down
12 changes: 8 additions & 4 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,10 @@ dependencies {
// Library for configuration:
implementation 'org.aeonbits.owner:owner:1.0.9'

implementation 'com.github.broadinstitute:picard:' + picardVersion
implementation ('com.github.broadinstitute:picard:' + picardVersion) {
exclude module: 'gkl'
}

externalSourceConfiguration 'com.github.broadinstitute:picard:' + picardVersion + ':sources'
implementation ('org.genomicsdb:genomicsdb:' + genomicsdbVersion) {
exclude module: 'log4j-api'
Expand Down Expand Up @@ -329,9 +332,10 @@ dependencies {
implementation('com.github.fommil.netlib:netlib-native_system-linux-x86_64:1.1:natives')
implementation('com.github.fommil.netlib:netlib-native_system-osx-x86_64:1.1:natives')

implementation('com.intel.gkl:gkl:0.8.11') {
exclude module: 'htsjdk'
}
// implementation('com.intel.gkl:gkl:0.8.11') {
// exclude module: 'htsjdk'
// }
implementation files('libs/gkl-2024_08_13.jar')

implementation 'org.broadinstitute:gatk-bwamem-jni:1.0.4'
implementation 'org.broadinstitute:gatk-fermilite-jni:1.2.0'
Expand Down
Binary file added libs/gkl-2024_08_13.jar
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,7 @@
import org.broadinstitute.hellbender.utils.genotyper.*;
import org.broadinstitute.hellbender.utils.haplotype.Haplotype;
import org.broadinstitute.hellbender.utils.haplotype.PartiallyDeterminedHaplotype;
import org.broadinstitute.hellbender.utils.pairhmm.LoglessPDPairHMM;
import org.broadinstitute.hellbender.utils.pairhmm.PDPairHMM;
import org.broadinstitute.hellbender.utils.pairhmm.PairHMM;
import org.broadinstitute.hellbender.utils.pairhmm.PairHMMInputScoreImputator;
import org.broadinstitute.hellbender.utils.pairhmm.*;
import org.broadinstitute.hellbender.utils.read.GATKRead;
import org.broadinstitute.hellbender.utils.read.ReadUtils;

Expand Down Expand Up @@ -114,7 +111,7 @@ public PDPairHMMLikelihoodCalculationEngine(final byte constantGCP,
this.log10globalReadMismappingRate = log10globalReadMismappingRate;
this.pcrErrorModel = this.dragstrParams == null ? pcrErrorModel : PairHMMLikelihoodCalculationEngine.PCRErrorModel.NONE;
// TODO later we probably need a LOG and LOGLESS version for parsimony with DRAGEN
this.pdPairHMM = new LoglessPDPairHMM();
this.pdPairHMM = new VectorLoglessPairPDHMM(VectorLoglessPairPDHMM.Implementation.OMP , new PairHMMNativeArguments());
if (resultsFile != null) {
pdPairHMM.setAndInitializeDebugOutputStream(new OutputStreamWriter(resultsFile.getOutputStream()));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,12 @@ public enum Implementation {
logger.info("Using the non-hardware-accelerated Java LOGLESS_CACHING PairHMM implementation");
return hmm;
}),
/* Optimized version of the PairHMM which caches per-read computations and operations in real space to avoid costly sums of log10'ed likelihoods */
AVX_LOGLESS_CACHING(args -> {
final PDPairHMM hmm = new VectorLoglessPairPDHMM(VectorLoglessPairPDHMM.Implementation.OMP, args);
logger.info("Using the non-hardware-accelerated Java LOGLESS_CACHING PairHMM implementation");
return hmm;
}),
FASTEST_AVAILABLE(args -> {
return new LoglessPDPairHMM();
});
Expand Down Expand Up @@ -120,7 +126,7 @@ public void initialize( final List<PartiallyDeterminedHaplotype> haplotypes, fin
initialize(readMaxLength, haplotypeMaxLength);
}

private static int findMaxAlleleLength(final List<? extends Allele> alleles) {
static int findMaxAlleleLength(final List<? extends Allele> alleles) {
int max = 0;
for (final Allele allele : alleles) {
final int alleleLength = allele.length();
Expand Down
Loading
Loading