[BUG] Polluting log entry regarding destination migration

[spapadop]

What is the bug?
There is a specific log that keeps on appearing quite frequently on all OpenSearch clusters (even fresh installations). While it does not somehow harm the cluster, its consistent presence is quite annoying:

[INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [my-node-name] Detected cluster change event for destination migration

I tracked this message on DestinationMigrationCoordinator.kt. I am not sure what is exactly wrong there, but something is.

There are couple of related threads already on the forums, like here and here.

And if you google "Detected cluster change event for destination migration", you'll see that it appears on many logs, for unrelated issues, so my guess is that it's just polluting logs.

How can one reproduce the bug?
Steps to reproduce the behavior:
Just bootstrap a new cluster and check the output logs.

What is the expected behavior?
Not to produce such a polluting log.

What is your host/environment?

• OpenSearch: v2.9.0
• OS: AlmaLinux 9.2
• Plugins: all that come by default

[IAmFQQ]

Yeah, I met the same issue, such logs are very annoying.

[2023-10-09T04:13:20,219][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [s-masterlvs-lvs-1] Detected cluster change event for destination migration
[2023-10-09T04:13:21,500][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [s-masterlvs-lvs-1] Detected cluster change event for destination migration
[2023-10-09T04:13:24,036][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [s-masterlvs-lvs-1] Detected cluster change event for destination migration

There are logs like this almost every few seconds, confused why alerting logs appeared in Opensearch...

[spapadop]

still there in v2.11

[riysaxen-amzn]

closing in favor of #1356

[uranru]

version 2.13
the problem persists

[bagipriyank]

i am seeing this is version 2.16.0

[Its-Ankush]

Can confirm i see this all the time [in OS_2.16]

[2024-09-20T09:23:18,647][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [data_node_1] Cancelling the migration process.
[2024-09-20T09:23:42,440][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [data_node_1] Cancelling the migration process.
[2024-09-20T09:23:43,819][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [data_node_1] Cancelling the migration process.
[2024-09-20T09:23:43,920][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [data_node_1] Cancelling the migration process.

[mousemaxx]

OS 2.17, messages still in log

[spapadop]

@riysaxen-amzn could you please reopen this issue? Or shall we create a new one?
The original reported problem is not fixed by #1356

btw, there is a suggestion there: #1356 (comment)

[tallyoh]

Can report that it still happens in 2.16 - Is there a new issue for this?

[jkrishnarao2003]

we upgraded our cluster from 2.10.0 to 2.18.0 last week. We see bunch of logs every often with message "Cancelling the migration process." Is this a bug or issue with our upgrade?