Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to notify Sentry only after the last retry on resque #2087

Conversation

GlauberrBatista
Copy link
Contributor

sentry-sidekiq and sentry-delayed_job have the option to report the failure only after the retry process has finished.

With sentry-resque we can do the same when using the resque-retry gem. The implementation is not as clean as Sidekiq and DelayedJob, which expose the attempts in a better way, but still, I was able to make it work.
I noticed the configuration option was there but not implemented. I added a conditional inside the exception block to check if the job is an instance of Resque::Plugins::Retry and if so, it checks the retry key.

An odd behavior: resque-retry cleans the key on the last try, before being caught on the exception block. Thus, when the key no longer exists on Redis, it means the retries were exhausted.

I understand this change implies using the resque-retry gem, but I'm unaware of a similar gem widely used as resque-retry.

@GlauberrBatista GlauberrBatista changed the title feat: add option to report failure only on the last retry Add option to notify Sentry only after the last retry on resque Aug 7, 2023
@codecov
Copy link

codecov bot commented Aug 20, 2023

Codecov Report

All modified lines are covered by tests ✅

Files Coverage Δ
sentry-resque/lib/sentry/resque.rb 97.36% <100.00%> (+0.14%) ⬆️

... and 50 files with indirect coverage changes

📢 Thoughts on this report? Let us know!.

@GlauberrBatista
Copy link
Contributor Author

Hello, any update on this?

@andreynering
Copy link

/cc @sl0thentr0py

@sl0thentr0py sl0thentr0py force-pushed the feature/resque_retry_report_after branch from 36f916f to 27beb99 Compare September 21, 2023 11:14
Copy link
Member

@sl0thentr0py sl0thentr0py left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey sorry for the delay on this but I don't think this solution is something we can put in the SDK (see comment)

If there's another possibility to fix the logic, I can take that otherwise I'll dig into resque later.


if Sentry.configuration.resque.report_after_job_retries && defined?(::Resque::Plugins::Retry) == 'constant' && klass.is_a?(::Resque::Plugins::Retry)
retry_key = klass.redis_retry_key(payload['args'])
retry_attempt = ::Resque.redis.get(retry_key)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the SDK should never do its own database calls by any chance, that's a side effect that breaks all sorts of contracts.
Is there no way of knowing the retry status from the stuff available to this function?

Copy link
Contributor Author

@GlauberrBatista GlauberrBatista Sep 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sl0thentr0py In fact, there is. I didn't realize it, but we are catching an exception here (so the processed job context is available) and when we use constantize against the class name, resque-retry will capture the job context, so methods retry_limit_reached? and retry_limit_attempt will expose the current "status" of the job.

I just pushed a new commit that fixes it. Hopefully it will match the requirements

@sl0thentr0py sl0thentr0py force-pushed the feature/resque_retry_report_after branch from 76fa859 to 2b1ab7d Compare October 10, 2023 13:23
Copy link
Member

@sl0thentr0py sl0thentr0py left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for the delay again and ty @GlauberrBatista !

@sl0thentr0py sl0thentr0py merged commit 71ec4f4 into getsentry:master Oct 10, 2023
95 checks passed
@GlauberrBatista
Copy link
Contributor Author

@sl0thentr0py not a problem and thank you 🙌

@GlauberrBatista GlauberrBatista deleted the feature/resque_retry_report_after branch October 10, 2023 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants