[Optimization] [Remote state] With every election full state shouldn't get uploaded to remote

[shwetathareja]

Describe the bug
Every time there is election, it bumps up the election term and full cluster state is uploaded to remote. This ensures during version upgrades and other cases, the leader uploads the full state and shouldn't face any compatibility issues. But, the downside is that it will result in uploading a large payload to remote store if the cluster state is in MBs and would result in elections taking longer.

OpenSearch/server/src/main/java/org/opensearch/gateway/GatewayMetaState.java

Line 737 in 9cc0e7d

|| lastAcceptedState.term() != clusterState.term()

Expected behavior
There can be optimization where a nodeId with new ephemeralId uploads the full state and in other cases we can skip uploading the full state and just upload the manifest with new term and version.

[kiranprakash154]

Hi, are we on track for this to be released in 2.12 ?

[soosinha]

Removing 2.12 tag. This will be picked up later.