-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Files in "tmp" should not be too large resp. removed #122
Comments
This is no log file. It's result file of a workflow. But I don't understand why this isn't deleted after returning the result to the client. I will have a look what is going on there. |
The files are not deleted at the moment. @dr0i and I discussed offline, that we first need a mechanism to delete these files after a successful processed workflow and a configurable limit how big a file may be. |
The size of these result files depends on the size of the input, right? |
Regarding the input file not being deleted: Possible fix: diff --git metafacture-runner/src/main/java/org/metafacture/runner/Flux.java metafacture-runner/src/main/java/org/metafacture/runner/Flux.java
index 9d889b45..5acdab6f 100644
--- metafacture-runner/src/main/java/org/metafacture/runner/Flux.java
+++ metafacture-runner/src/main/java/org/metafacture/runner/Flux.java
@@ -25,6 +25,7 @@
import java.io.File;
import java.io.IOException;
+import java.io.InputStream;
import java.util.HashMap;
import java.util.Map;
import java.util.regex.Matcher;
@@ -82,7 +83,9 @@ public static void main(final String[] args) throws IOException, RecognitionExce
}
// run parser and builder
- FluxCompiler.compile(ResourceUtil.getStream(fluxFile), vars).start();
+ try (InputStream inputStream = ResourceUtil.getStream(fluxFile)) {
+ FluxCompiler.compile(inputStream, vars).start();
+ }
}
} |
Hm, but that stream is for the Flux file, not the input file, right? Still looks like a good change fixing a resource leak. |
D'oh :) Sorry, you're absolutely right! |
Then it's the diff --git metafacture-io/src/main/java/org/metafacture/io/FileOpener.java metafacture-io/src/main/java/org/metafacture/io/FileOpener.java
index b88b9aab..a2a8eeac 100644
--- metafacture-io/src/main/java/org/metafacture/io/FileOpener.java
+++ metafacture-io/src/main/java/org/metafacture/io/FileOpener.java
@@ -154,8 +154,8 @@ public Reader open(final InputStream stream) throws IOException {
@Override
public void process(final String file) {
- try {
- getReceiver().process(open(file));
+ try (Reader reader = open(file)) {
+ getReceiver().process(reader);
}
catch (final IOException e) {
throw new MetafactureException(e); (Breaks some tests, though, since they try to read from the reader a second time in order to verify its contents.) |
Oh, yes, that looks good! (Seems the |
I move this ticket back to backlog and it becomes ready, when the problem with the FileInputStream is fixed in Fix |
But can you confirm that this is indeed the issue here? We've identified two unclosed resources, one of which still allows deleting the file ( |
I think I can confirm it (or not), when these unclosed resources are closed. I think the probability is very high that one of these two is the reason that I can't delete the input file, isn't it? |
Well, let me put it this way: If you could confirm beforehand that properly closing the So would it be possible for you to apply the fix locally and observe its effect in the Playground? |
I'll try 👍 |
Great, thank you for confirming! Then we'll have to see how we can fix it (in -core, not -fix) while still preserving testability. |
I've opened metafacture/metafacture-core#514 to address the unclosed resources. |
Resources are now closed in metafacture-core; should be released with version 6.0 (see metafacture/metafacture-core#515). |
I would like to split this issue in two:
|
Subissues are opend, so this issue can be closed.. |
There is one really big file in tmp:
We should somehow restrict log files to be too big.
The text was updated successfully, but these errors were encountered: