Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logs for ScheduledTellMsg DeadLetter are not specific enough #7264

Open
Arkatufus opened this issue Jun 24, 2024 · 0 comments
Open

Logs for ScheduledTellMsg DeadLetter are not specific enough #7264

Arkatufus opened this issue Jun 24, 2024 · 0 comments

Comments

@Arkatufus
Copy link
Contributor

Arkatufus commented Jun 24, 2024

Version Information

Version of Akka.NET? dev branch
Which Akka.NET Modules? Core

Describe the bug

Dead letters containing ScheduledTellMsg from the Scheduler are hiding the underlying payload type it is wrapping, making it hard to debug/inspect via telemetry

Here's an example snippet log dump from one of the internal Akka.Cluster.Tools.Tests.Multinode.ClusterClient.ClusterClientDiscoerySpec test:

[WARNING][06/24/2024 14:51:03.026Z][Thread 0015][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.120Z][Thread 0017][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.136Z][Thread 0019][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.246Z][Thread 0020][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.356Z][Thread 0016][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.464Z][Thread 0017][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.571Z][Thread 0022][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.679Z][Thread 0018][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.773Z][Thread 0020][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.897Z][Thread 0019][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:04.004Z][Thread 0018][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:04.112Z][Thread 0021][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:04.112Z][Thread 0021][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>

Here a message sent from Self to Self via ScheduleTell (either once or repeated) was not being handled correctly, but we could not see which message was being sent, thus we could never tell which timer were at fault.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant