Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gracefully exit with and on GoAway messages #54

Merged
merged 6 commits into from
Sep 22, 2023

Conversation

stephenc-pace
Copy link
Collaborator

Summary

Http2 supports the concept of a GOAWAY frame
https://www.rfc-editor.org/rfc/rfc7540#section-6.8
which our underlying library (h2) doesn't handle correctly (python-hyper/h2#1181)

This is patches in support to h2 and then we handle it correctly from our side.

@stephenc-pace
Copy link
Collaborator Author

@mattbennett
We've had this in production (with the other PRs) for a number of months and it has resolved all the issues we saw with GoAway errors.
Can we look at merging along with other fixes so we can cut a new release?

Copy link
Member

@mattbennett mattbennett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice. A few questions and cosmetic changes but I'm otherwise happy for this to get merged.

nameko_grpc/connection.py Outdated Show resolved Hide resolved
nameko_grpc/connection.py Show resolved Hide resolved

When a stream is closed we append the STREAM_END item or a GrpcError, so an
exhausted stream will possibly still have 1 item left in the queue, so we
must check for that.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem to be reflected in the implementation?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're correct, the comment is not needed.

@@ -166,7 +170,7 @@ def headers_to_send(self, defer_until_data=True):
if self.headers_sent or len(self.headers) == 0:
return False

if defer_until_data and self.queue.empty():
if defer_until_data and self.queue.empty() and self.buffer.empty():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this a preexisting bug, or did the implementation need to change as a result of the termination handling?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this as It doesn't seem like it was needed and I don't see why I added it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually I've added it back.
Previously, we wouldn't send headers until there was data to send (which sat on the queue as it wasn't flushed until the headers had been sent. But now, as a result of calling flush_queue_to_buffer indiscriminately, that data might be on the buffer rather than the queue.
We could possibly remove the check on the queue empty but it feels safer to keep it.

@timbu timbu self-requested a review September 21, 2023 15:27
@timbu
Copy link
Contributor

timbu commented Sep 21, 2023

Hi @mattbennett - we're planning on merging this as we think it's working well in production now. Shout if you have any issue 😄

@timbu timbu merged commit 16b9b18 into nameko:master Sep 22, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants