Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: augment predicate status returned by GET/LIST endpoints #397

Merged
merged 52 commits into from
Sep 14, 2023
Merged

feat: augment predicate status returned by GET/LIST endpoints #397

merged 52 commits into from
Sep 14, 2023

Conversation

MicaiahReid
Copy link
Contributor

@MicaiahReid MicaiahReid commented Aug 24, 2023 •
edited
Loading

Description

This PR accomplishes a few things:

  • renames/adds/removes predicate statuses according to the flowchart in docs/images/predicate-status-flowchart/PredicateStatusFlowchart.png
  • adds more context to errors returned in Interrupted status
  • adds status data to GET /v1/chainhooks endpoint
  • fixes bug in chainhook service where block streaming continued past end_block

Fixes #396, fixes #324

Also:
Improves how we handle a restart of chainhook service while predicates are scanning/streaming. Here are the cases we now handle:

  1. Predicates that were in scanning status when Chainhook was terminated will resume scanning starting from their last_evaluated_block_height. Note: because we only save predicate status every 10 scans, we could end up re-emiting matches on a resetart
  2. Predicates that were in new status when Chainhook was terminated will start scanning at the predicate's start_block
  3. Predicates that were in streaming status will return to a scanning status, starting at last_evaluated_block_height to catch up on the missed blocks. Note, the number_of_blocks_to_scan is set to 0 for this temporary catch-up, as it's difficult to compute the number of remaining blocks in the context of this change
  4. If predicates were passed in at startup, we also register those to begin scanning, which previously didn't happen
  5. We now allow passing in a predicate at startup and registering additional predicates with the predicate registration server. This means that if you use the same startup predicate repeatedly, it will already be saved in redis and not be reloaded.

Fixes: #298, fixes #390, fixes #402, fixes #403

Breaking change?

The rename of ScanningData's number_of_blocks_sent field could technically be considered breaking, let's discuss.

Checklist

  • All tests pass
  • Tests added in this PR (if applicable)
    Test coverage before: 23.2%
    Test coverage after: 37.72%

@MicaiahReid MicaiahReid marked this pull request as ready for review August 24, 2023 19:19
@MicaiahReid MicaiahReid changed the title feat: augment predicate status returned by GET /v1/chainhooks/<predicate-uuid> feat: augment predicate status returned by GET/LIST endpoints Aug 24, 2023
@MicaiahReid MicaiahReid mentioned this pull request Aug 24, 2023
Closed
@MicaiahReid MicaiahReid removed the request for review from lgalabru August 28, 2023 18:04
@MicaiahReid MicaiahReid marked this pull request as draft August 28, 2023 18:04
@MicaiahReid MicaiahReid mentioned this pull request Sep 13, 2023
Open
lgalabru
lgalabru approved these changes Sep 13, 2023
View reviewed changes
Copy link
Contributor

@lgalabru lgalabru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MicaiahReid!

@MicaiahReid MicaiahReid merged commit a100263 into develop Sep 14, 2023
1 check passed
@MicaiahReid MicaiahReid deleted the augment-get-chainhooks branch September 14, 2023 15:40
@smcclellan smcclellan added this to the Q3-2023 milestone Sep 14, 2023
@MicaiahReid MicaiahReid mentioned this pull request Oct 6, 2023
Closed
github-actions bot pushed a commit that referenced this pull request Oct 10, 2023
@semantic-release-bot
chore(release): 1.1.0 [skip ci]
8ca9afa 
## [1.1.0](v1.0.0...v1.1.0) (2023-10-10)

### Features

* allow matching with regex for stacks print_event ([#380](#380)) ([131809e](131809e)), closes [#348](#348)
* augment predicate status returned by GET/LIST endpoints ([#397](#397)) ([a100263](a100263)), closes [#396](#396) [#324](#324) [#390](#390) [#402](#402) [#403](#403)
* introduce "data_handler_tx" ([ee486f3](ee486f3))

### Bug Fixes

* build error ([85d4d91](85d4d91))
* build errors ([b9ff1aa](b9ff1aa))
* build errro ([be0c229](be0c229))
* bump retries and delays ([aff3690](aff3690))
* chainhook not being registered ([5a809c6](5a809c6))
* ensure that the parent block was previously received. else, fetch it ([2755266](2755266))
* migrate to finer zmq lib ([4eb5a07](4eb5a07))
* prevent panic when scanning from genesis block ([#408](#408)) ([1868a06](1868a06))
* remove event_handlers ([6fecfd2](6fecfd2))
* retrieve blocks until tip ([5213f5f](5213f5f))
* revisit approach ([67a34dc](67a34dc))
* use crossbeam channels ([ea33553](ea33553))
* workflow ([d434c93](d434c93))
@github-actions
Copy link

🎉 This PR is included in version 1.1.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

MicaiahReid pushed a commit that referenced this pull request Oct 10, 2023
@semantic-release-bot @MicaiahReid
chore(release): 1.1.0 [skip ci]
09b68ca 
## [1.1.0](v1.0.0...v1.1.0) (2023-10-10)

### Features

* allow matching with regex for stacks print_event ([#380](#380)) ([131809e](131809e)), closes [#348](#348)
* augment predicate status returned by GET/LIST endpoints ([#397](#397)) ([a100263](a100263)), closes [#396](#396) [#324](#324) [#390](#390) [#402](#402) [#403](#403)
* introduce "data_handler_tx" ([ee486f3](ee486f3))

### Bug Fixes

* build error ([85d4d91](85d4d91))
* build errors ([b9ff1aa](b9ff1aa))
* build errro ([be0c229](be0c229))
* bump retries and delays ([aff3690](aff3690))
* chainhook not being registered ([5a809c6](5a809c6))
* ensure that the parent block was previously received. else, fetch it ([2755266](2755266))
* migrate to finer zmq lib ([4eb5a07](4eb5a07))
* prevent panic when scanning from genesis block ([#408](#408)) ([1868a06](1868a06))
* remove event_handlers ([6fecfd2](6fecfd2))
* retrieve blocks until tip ([5213f5f](5213f5f))
* revisit approach ([67a34dc](67a34dc))
* use crossbeam channels ([ea33553](ea33553))
* workflow ([d434c93](d434c93))
vabanaerytk added a commit to vabanaerytk/chainhook that referenced this pull request Aug 7, 2024
@vabanaerytk @MicaiahReid
chore(release): 1.1.0 [skip ci]
e4fc545 
## [1.1.0](hirosystems/chainhook@v1.0.0...v1.1.0) (2023-10-10)

### Features

* allow matching with regex for stacks print_event ([#380](hirosystems/chainhook#380)) ([af0fcad](hirosystems/chainhook@af0fcad)), closes [#348](hirosystems/chainhook#348)
* augment predicate status returned by GET/LIST endpoints ([#397](hirosystems/chainhook#397)) ([0f32704](hirosystems/chainhook@0f32704)), closes [#396](hirosystems/chainhook#396) [#324](hirosystems/chainhook#324) [#390](hirosystems/chainhook#390) [#402](hirosystems/chainhook#402) [#403](hirosystems/chainhook#403)
* introduce "data_handler_tx" ([a7704d8](hirosystems/chainhook@a7704d8))

### Bug Fixes

* build error ([9d47f87](hirosystems/chainhook@9d47f87))
* build errors ([27b0ae4](hirosystems/chainhook@27b0ae4))
* build errro ([6363fad](hirosystems/chainhook@6363fad))
* bump retries and delays ([483663c](hirosystems/chainhook@483663c))
* chainhook not being registered ([7d2b156](hirosystems/chainhook@7d2b156))
* ensure that the parent block was previously received. else, fetch it ([79561c8](hirosystems/chainhook@79561c8))
* migrate to finer zmq lib ([2e55a20](hirosystems/chainhook@2e55a20))
* prevent panic when scanning from genesis block ([#408](hirosystems/chainhook#408)) ([4ab8dd8](hirosystems/chainhook@4ab8dd8))
* remove event_handlers ([e3664d2](hirosystems/chainhook@e3664d2))
* retrieve blocks until tip ([d21cacc](hirosystems/chainhook@d21cacc))
* revisit approach ([47d2ef6](hirosystems/chainhook@47d2ef6))
* use crossbeam channels ([50595ab](hirosystems/chainhook@50595ab))
* workflow ([84c4ec9](hirosystems/chainhook@84c4ec9))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lgalabru lgalabru lgalabru approved these changes

Assignees

@MicaiahReid MicaiahReid

Labels
released
Projects
DevTools
Archived in project
Milestone
Q3-2023
3 participants
@MicaiahReid @lgalabru @smcclellan