Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Allocation and Rebalance Constraints of WeightFunction are incorrectly reset to default if indexBalanceFactor, shardBalanceFactor or preferPrimaryShardRebalanceBuffer is updated #13429

Open
KunjueYu opened this issue Apr 28, 2024 · 2 comments
Assignees

Comments

@KunjueYu
Copy link

KunjueYu commented Apr 28, 2024

Describe the bug

In BalancedShardsAllocator, the update consumers of PREFER_PRIMARY_SHARD_REBALANCE and PREFER_PRIMARY_SHARD_BALANCE directly modifies the constraint config of existing weightFunction. However, if indexBalanceFactor, shardBalanceFactor or preferPrimaryShardRebalanceBuffer is updated, a new weightFunction is constructed, which will discard the preferPrimary Constraint configs of previous weightFunction.

  /**
     * When primary shards balance is desired, enable primary shard balancing constraints
     * @param preferPrimaryShardBalance boolean to prefer balancing by primary shard
     */
    private void setPreferPrimaryShardBalance(boolean preferPrimaryShardBalance) {
        this.preferPrimaryShardBalance = preferPrimaryShardBalance;
        this.weightFunction.updateAllocationConstraint(INDEX_PRIMARY_SHARD_BALANCE_CONSTRAINT_ID, preferPrimaryShardBalance);
        this.weightFunction.updateAllocationConstraint(CLUSTER_PRIMARY_SHARD_BALANCE_CONSTRAINT_ID, preferPrimaryShardBalance);
        this.weightFunction.updateRebalanceConstraint(INDEX_PRIMARY_SHARD_BALANCE_CONSTRAINT_ID, preferPrimaryShardBalance);
    }

    private void setPreferPrimaryShardRebalance(boolean preferPrimaryShardRebalance) {
        this.preferPrimaryShardRebalance = preferPrimaryShardRebalance;
        this.weightFunction.updateRebalanceConstraint(CLUSTER_PRIMARY_SHARD_REBALANCE_CONSTRAINT_ID, preferPrimaryShardRebalance);
    }
 private void updateIndexBalanceFactor(float indexBalanceFactor) {
        this.indexBalanceFactor = indexBalanceFactor;
        updateWeightFunction();
    }

    private void updateShardBalanceFactor(float shardBalanceFactor) {
        this.shardBalanceFactor = shardBalanceFactor;
        updateWeightFunction();
    }

    private void updatePreferPrimaryShardBalanceBuffer(float preferPrimaryShardBalanceBuffer) {
        this.preferPrimaryShardRebalanceBuffer = preferPrimaryShardBalanceBuffer;
        updateWeightFunction();
    }

    private void updateWeightFunction() {
        weightFunction = new WeightFunction(this.indexBalanceFactor, this.shardBalanceFactor, this.preferPrimaryShardRebalanceBuffer);
    }

Related component

Cluster Manager

To Reproduce

  1. set setting "PREFER_PRIMARY_SHARD_BALANCE" as true
  2. set setting "SHARD_BALANCE_FACTOR_SETTING" as 0.5f (arbitrary value which differs from default value 0.45f)
  3. the preferPrimary constraints of weight function will be incorrectly disabled.

Expected behavior

Updating indexBalanceFactor, shardBalanceFactor or preferPrimaryShardRebalanceBuffer will not reset preferPrimary constraints of weight function to default.

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@peternied
Copy link
Member

[Triage - attendees 1 2 3 4 5 6 7 8]
@KunjueYu Thanks for creating this issue, we'd welcome a pull request to address this bug

@patelsmit32123
Copy link

@peternied I would like to take this up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🆕 New
Status: 🆕 New
Development

No branches or pull requests

3 participants