Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add to check the liveness for connection with controller #920

Merged
merged 2 commits into from
Aug 13, 2024

Conversation

dofmind
Copy link
Contributor

@dofmind dofmind commented Aug 1, 2024

This PR is that agent can detect disconnection with controller faster, like controller.
Also includes an integration test to test this feature.

This adds a new heartbeat signal for internal DBus to controller, also
adds a new option to agent: ControllerHeartbeatThreshold.

The controller emits the heartbeat signal like agent. The agent writes
last seen timestamp for connection with controller if it receives the
heartbeat signal from controller, and the agent periodically checks the
last seen timestamp for connection with controller, and if it was sent
before ControllerHeartbeatThreshold, the agent treats it as
disconnected.

If ControllerHeartbeatThreshold value is 0, it is disabled to check the
liveness for connection with controller, and the default value is 0.

@coveralls
Copy link

coveralls commented Aug 1, 2024

Coverage Status

coverage: 84.791% (-0.05%) from 84.837%
when pulling 19ba4a4 on dofmind:check-liveness
into de46dbd on eclipse-bluechi:main.

@dofmind dofmind force-pushed the check-liveness branch 4 times, most recently from 576d0f9 to 28a82bf Compare August 2, 2024 05:09
@engelmi
Copy link
Member

engelmi commented Aug 6, 2024

Hi @dofmind,
Thanks for your contribution!

This PR comprises of a new feature to be added to BlueChi. It is highly recommended to file a new issue before creating a PR. This way we can discuss it upfront and keep the review of the PR(s) concise.

About the feature itself:
The agents are sending a recurring Heartbeat signal (based on HeartbeatInterval) to the controller. This should already be sufficient to detect disconnects to the controller within the HeartbeatInterval. Do you have a case where the Heartbeat doesn't enable the agent to detect a disconnect from the controller? @dofmind

@dofmind
Copy link
Contributor Author

dofmind commented Aug 7, 2024

@engelmi Sorry for posting PR before creating an issue. I submitted a new issue - #921. I created this PR because my bluechi-agent still takes around 60 seconds to detect a disconnect from bluechi-controller.

src/controller/controller.c Outdated Show resolved Hide resolved
src/controller/controller.c Outdated Show resolved Hide resolved
src/agent/agent.h Outdated Show resolved Hide resolved
src/agent/agent.h Outdated Show resolved Hide resolved
src/agent/agent.c Outdated Show resolved Hide resolved
src/agent/agent.c Outdated Show resolved Hide resolved
src/agent/agent.c Outdated Show resolved Hide resolved
src/controller/controller.c Outdated Show resolved Hide resolved
This adds a new heartbeat signal for internal DBus to controller, also
adds a new option to agent: ControllerHeartbeatThreshold.

The controller emits the heartbeat signal like agent. The agent writes
last seen timestamp for connection with controller if it receives the
heartbeat signal from controller, and the agent periodically checks the
last seen timestamp for connection with controller, and if it was sent
before ControllerHeartbeatThreshold, the agent treats it as
disconnected.

If ControllerHeartbeatThreshold value is 0, it is disabled to check the
liveness for connection with controller, and the default value is 0.

Signed-off-by: Joonyoung Shim <[email protected]>
An integration test is to verify if the agent gets disconnected when did
not receive heartbeat since threshold from controller.

Signed-off-by: Joonyoung Shim <[email protected]>
Copy link
Member

@engelmi engelmi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@engelmi engelmi merged commit e5e4681 into eclipse-bluechi:main Aug 13, 2024
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add to check the liveness for connection with controller
3 participants