Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document and fail fast if tmp directory is noexec #252

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,29 @@ If using features that rely on OpenCV (see the [Downsampling type](#downsampling

__NOTE:__ If you are setting `jna.library.path` via the `JAVA_OPTS` environment variable, make sure the path is to the folder __containing__ the library not path to the library itself.

Temporary directory usage
=========================

Beginning with 0.10.0, if the default temporary directory (usually `/tmp/`) is mounted as `noexec`, conversion will fail immediately by default.
[CIS security benchmarks](https://www.cisecurity.org/benchmark) recommend that `/tmp/` be mounted as `noexec`; these standards are increasingly
being adopted by major Linux distributions. For certain types of input data (e.g. NDPI, JPEG-XR compression), bioformats2raw needs
to extract a native library from one or more jars to a temporary location. In these cases, the extracted native library must be executable in order
to read image data.

The recommended solution is to choose a different temporary directory by adding `-Djava.io.tmpdir=/path/to/alternate/tmp` to `JAVA_OPTS`.
If multiple properties need to be set via `JAVA_OPTS`, separate them with a space, e.g. `JAVA_OPTS="-Djava.io.tmpdir/path/to/alternate/tmp -Djna.library.path=/path/to/blosc"`.

If you know this issue does not affect your input data and wish to warn instead of immediately stopping conversion, the `--warn-no-exec` option can be used.
For input data that relies on native library loading (e.g. NDPI, JPEG-XR compression), using `--warn-no-exec` instead of specifying an alternate
temporary directory will simply cause the conversion to fail at a later point.

For additional information, please see:

* https://github.com/glencoesoftware/bioformats2raw/pull/252
* https://github.com/scijava/native-lib-loader/issues/41
* https://forum.image.sc/t/after-omero-server-upgrade-hamamatsu-ndpi-files-display-only-in-low-resolution/81868
* https://forum.image.sc/t/bioformats-unable-to-read-czi-files-with-jpegxr-compression-on-almalinux-os-java-lang-unsatisfiedlinkerror-ome-jxrlib-jxrjni-new-decodecontext-j/82646

Installation
============

Expand Down
58 changes: 58 additions & 0 deletions src/main/java/com/glencoesoftware/bioformats2raw/Converter.java
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,8 @@ public class Converter implements Callable<Integer> {
private volatile Path inputPath;
private volatile String outputLocation;

private volatile boolean warnNoExec = false;

private Map<String, String> outputOptions;
private volatile Integer pyramidResolutions;
private volatile List<Integer> seriesList;
Expand Down Expand Up @@ -409,6 +411,23 @@ public void setProgressBars(boolean useProgressBars) {
progressBars = useProgressBars;
}

/**
* Configure whether to warn instead of throwing an exception
* if files created in the temporary directory
* ("java.io.tmpdir" system property) will not be executable.
*
* @param warnOnly true if a warning should be logged instead of an exception
*/
@Option(
names = {"--warn-no-exec"},
description = "Warn instead of throwing an exception if " +
"java.io.tmpdir is not executable",
defaultValue = "false"
)
public void setWarnOnNoExec(boolean warnOnly) {
warnNoExec = warnOnly;
}

/**
* Configure whether to print version information and exit
* without converting.
Expand Down Expand Up @@ -944,6 +963,14 @@ public boolean getProgressBars() {
return progressBars;
}

/**
* @return true if a warning is logged instead of an exception when the tmp
* directory is noexec
*/
public boolean getWarnNoExec() {
return warnNoExec;
}

/**
* @return true if only version info is displayed
*/
Expand Down Expand Up @@ -1172,6 +1199,37 @@ public Integer call() throws Exception {
tileWidth, tileHeight);
}

// if the tmpdir is mounted as noexec, then any native library
// loading (OpenCV, turbojpeg, etc.) is expected to fail, which
// can cause conversion to fail with an exception that is not user friendly
//
// try to detect this case early and fail before the conversion begins,
// with a more informative message

// a temp file is created and set as executable
// in the noexec case, setting as executable is expected to silently fail
File tmpdirCheck = File.createTempFile("noexec-test", ".txt");
try {
// expect 'success' to be true in the noexec case, even though
// the file will not actually be executable
boolean success = tmpdirCheck.setExecutable(true);
if (!success || !tmpdirCheck.canExecute()) {
String msg = System.getProperty("java.io.tmpdir") +
" is noexec; fix it or specify a different java.io.tmpdir." +
" See https://github.com/glencoesoftware/bioformats2raw/" +
"blob/master/README.md#temporary-directory-usage for details";
if (getWarnNoExec()) {
LOGGER.warn(msg);
}
else {
throw new RuntimeException(msg);
}
}
}
finally {
tmpdirCheck.delete();
}

OpenCVTools.loadOpenCV();

if (progressBars) {
Expand Down