Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exit after read does not work if all files have already been read #288

Open
max-frank opened this issue Mar 26, 2021 · 0 comments · May be fixed by #289
Open

Exit after read does not work if all files have already been read #288

max-frank opened this issue Mar 26, 2021 · 0 comments · May be fixed by #289

Comments

@max-frank
Copy link

Description

In read mode if exit_after_read is set to true and file_completed_action to log (i.e., the files are not removed from their path after being fully read) logstash will not properly halt execution if all files have already been fully read during a previous run.

This is because files which are fully read (according to sincedb) are marked as in-active and are thus skipped in the Processor.process_active file loop and left in a watched state thus the pipeline never becomes empty.
Calling common_detach_when_allread before skipping in-active files, when exit_after_read is set fixes this issue.

Information

  • Version: 4.2.4
  • Operating System: Ubuntu Bionic (18.04)
  • Config File:
# pipeline config
input {
  file {
    path => "/tmp/test.log"
    mode => "read"
    sincedb_path => "/tmp/sincedb"
    file_completed_action => "log"
    file_completed_log_path => "/tmp/done.log"
    exit_after_read => true
  }
}

output {
  stdout {}
}
  • Sample Data:
    /tmp/test.log
Line 1
Line 2
Line 3
Line 4
Line 5
  • Steps to Reproduce:
    1. Run the pipeline the first time (this will create the sincedb file)
    2. See that the logstash process correctly stops after reading "Line 5"
    3. Run the pipeline again
    4. See that the logstash process correctly recognizes test.log as fully read, but does not stop execution.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant