Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native build does not link libzstd-jni correctly #24

Open
heuermh opened this issue Mar 3, 2020 · 4 comments
Open

Native build does not link libzstd-jni correctly #24

heuermh opened this issue Mar 3, 2020 · 4 comments
Milestone

Comments

@heuermh
Copy link
Owner

heuermh commented Mar 3, 2020

$ ./target/dsh-bio-native compress-bed \
  -i GIAB_SmallVariant_Benchmark_v4beta_GRCh37_HG002.bed.gz \
  -o GIAB_SmallVariant_Benchmark_v4beta_GRCh37_HG002.bed.zst

java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.dishevelled.bio.tools.Tools.call(Tools.java:79)
	at org.dishevelled.bio.tools.Tools.main(Tools.java:276)
Caused by: java.lang.UnsatisfiedLinkError: no zstd-jni in java.library.path
Unsupported OS/arch, cannot find /darwin/x86_64/libzstd-jni.dylib or load zstd-jni from system libraries. Please try building from source the jar or providing libzstd-jni in your system.
	at com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibrary(NativeLibrarySupport.java:131)
	at java.lang.ClassLoader.loadLibrary(Target_java_lang_ClassLoader.java:126)
	at java.lang.Runtime.loadLibrary0(Runtime.java:870)
	at java.lang.Runtime.loadLibrary(Runtime.java:235)
	at java.lang.System.loadLibrary(System.java:360)
	at com.github.luben.zstd.util.Native.load(Native.java:68)
	at com.github.luben.zstd.util.Native.load(Native.java:54)
	at com.github.luben.zstd.ZstdOutputStream.<clinit>(ZstdOutputStream.java:15)
	at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350)
	at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270)
	at java.lang.Class.ensureInitialized(DynamicHub.java:475)
	at org.apache.commons.compress.compressors.zstandard.ZstdCompressorOutputStream.<init>(ZstdCompressorOutputStream.java:83)
	at org.dishevelled.compress.Writers.writer(Writers.java:155)
	at org.dishevelled.compress.Writers.writer(Writers.java:134)
	at org.dishevelled.bio.tools.CompressBed.call(CompressBed.java:77)
	at org.dishevelled.bio.tools.CompressBed.main(CompressBed.java:140)
	... 3 more
@heuermh heuermh added this to the 1.4 milestone Mar 3, 2020
@heuermh
Copy link
Owner Author

heuermh commented Mar 3, 2020

Added jni-config.json in commit e407d93, which required a compile time dependency on org.brotli.dec:0.1.2. Build succeeds but same result at runtime.

@heuermh
Copy link
Owner Author

heuermh commented Mar 4, 2020

native-image-agent provides the following

$ JAVA_HOME=.../graalvm-ce-java8-20.0.0/Contents/Home java \
  -agentlib:native-image-agent=config-output-dir=`pwd` \
  -classpath "../tools/target/appassembler/lib/*" \
  org.dishevelled.bio.tools.Tools compress-bed \
  -i GIAB_SmallVariant_Benchmark_v4beta_GRCh37_HG002.bed.gz \
  -o GIAB_SmallVariant_Benchmark_v4beta_GRCh37_HG002.bed.2.zst

$ cat jni-config.json
[
{
  "name":"com.github.luben.zstd.ZstdOutputStream",
  "fields":[
    {"name":"dstPos"},
    {"name":"srcPos"}
  ]
},
{
  "name":"org.dishevelled.bio.tools.Tools",
  "methods":[{"name":"main","parameterTypes":["java.lang.String[]"] }]
}
]

$ cat proxy-config.json
[
]

$ cat reflect-config.json
[
{
  "name":"org.dishevelled.bio.tools.CompressBed",
  "methods":[{"name":"main","parameterTypes":["java.lang.String[]"] }]
}
]

$ cat resource-config.json
{
  "resources":[
    {"pattern":"\\QMETA-INF/services/jdk.vm.ci.hotspot.HotSpotJVMCIBackendFactory\\E"},
    {"pattern":"\\QMETA-INF/services/jdk.vm.ci.services.JVMCIServiceLocator\\E"},
    {"pattern":"\\Qdarwin/x86_64/libzstd-jni.dylib\\E"}
  ],
  "bundles":[]
}

Using this would seem to make the native-image build platform specific.

@heuermh heuermh modified the milestones: 1.4, 2.0 Jul 16, 2020
@heuermh heuermh modified the milestones: 2.0, 2.1 Jan 8, 2021
@heuermh
Copy link
Owner Author

heuermh commented Sep 11, 2021

Still an issue

$ ./target/dsh-bio-native compress-bed \
    -i GIAB_SmallVariant_Benchmark_v4beta_GRCh37_HG002.bed.gz \
    -o GIAB_SmallVariant_Benchmark_v4beta_GRCh37_HG002.bed.zst

java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Method.invoke(Method.java:566)
	at org.dishevelled.bio.tools.Tools.call(Tools.java:79)
	at org.dishevelled.bio.tools.Tools.main(Tools.java:294)
Caused by: java.lang.UnsatisfiedLinkError: no zstd-jni-1.5.0-4 in java.library.path
Unsupported OS/arch, cannot find /darwin/amd64/libzstd-jni-1.5.0-4.dylib or load zstd-jni-1.5.0-4 from system libraries. Please try building from source the jar or providing libzstd-jni-1.5.0-4 in your system.
	at com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibraryRelative(NativeLibrarySupport.java:132)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:197)
	at java.lang.Runtime.loadLibrary0(Runtime.java:830)
	at java.lang.System.loadLibrary(System.java:1871)
	at com.github.luben.zstd.util.Native.load(Native.java:87)
	at com.github.luben.zstd.util.Native.load(Native.java:55)
	at com.github.luben.zstd.ZstdOutputStreamNoFinalizer.<clinit>(ZstdOutputStreamNoFinalizer.java:18)
	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:375)
	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:295)
	at com.github.luben.zstd.ZstdOutputStream.<init>(ZstdOutputStream.java:73)
	at com.github.luben.zstd.ZstdOutputStream.<init>(ZstdOutputStream.java:53)
	at org.apache.commons.compress.compressors.zstandard.ZstdCompressorOutputStream.<init>(ZstdCompressorOutputStream.java:86)
	at org.dishevelled.compress.Writers.writer(Writers.java:155)
	at org.dishevelled.compress.Writers.writer(Writers.java:134)
	at org.dishevelled.bio.tools.CompressBed.call(CompressBed.java:77)
	at org.dishevelled.bio.tools.CompressBed.main(CompressBed.java:140)
	... 3 more

@heuermh heuermh modified the milestones: 2.1, 3.0 Sep 7, 2022
@heuermh
Copy link
Owner Author

heuermh commented Jun 15, 2023

Note upstream issue was closed as fixed, try again with updated metadata?

@heuermh heuermh modified the milestones: 3.0, 2.2.2 Jun 20, 2023
@heuermh heuermh modified the milestones: 2.2.2, 2.3 Jun 27, 2023
@heuermh heuermh modified the milestones: 2.3, 3.0 Sep 16, 2023
@heuermh heuermh modified the milestones: 3.0, 4.0 Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant