Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mock_uss] Mitigate #28 #172

Merged
merged 2 commits into from
Aug 18, 2023
Merged

Conversation

BenjaminPelletier
Copy link
Member

This PR takes a few measures to attempt to mitigate the effects of #28, but it does not attempt to address the root problem as that is still unknown.

  • Stop mock_uss from querying itself for its operational intent details
  • Stop mock_uss from notifying itself when an operational intent is changed
  • Reduce query timeouts to 5 seconds (from 25) to have problems show up closer to when they happen (and all our queries seem to be well under 5 seconds in general -- most seem under 0.1 seconds)
  • Reduce deadlock timeouts to 5 seconds (from 60) to have problems show up sooner
  • Add logging, especially with regard to which process things are happening on, to attempt to isolate the problem more in the future

These changes should not resolve the underlying issue. The underlying issue seems to be that some requests are simply not received by their intended recipient and the sender times out waiting for the receiver to handle the request (sender doesn't seem to know recipient hasn't even started their handler). When this happens, all timeouts are set to about the same length, so if the handler for call 1 depends on making call 2 and this happens with call 2, the top-level caller for call 1 is guaranteed to abort call 1 and send the same request again, and this sometimes causes problems.

@BenjaminPelletier BenjaminPelletier marked this pull request as ready for review August 18, 2023 04:49
@barroco barroco merged commit 2c3a704 into interuss:main Aug 18, 2023
@BenjaminPelletier BenjaminPelletier deleted the feature/mitigate-28 branch August 18, 2023 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants