Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 prefix with timestamp #193

Open
tezaaaa opened this issue Nov 19, 2019 · 0 comments
Open

S3 prefix with timestamp #193

tezaaaa opened this issue Nov 19, 2019 · 0 comments

Comments

@tezaaaa
Copy link

tezaaaa commented Nov 19, 2019

We are using logstash s3 input plugin to parse data of an S3 bucket that has millions of records. All the logfiles are prefixed with timestamp (eg:- yyyy-mm-dd-) i am unable to find a way to specify the pattern to match just the logs for the day. Any help is much appreciated.

The problem here is it has to cycle through all the files to pick the ones for today. i can use hard coded prefix (2019-11-) to match this month but multiple updates to this pipeline is not a feasible solution.

  • 6.2.0:
  • Red Hat Linux 7 :
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant