Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test: Take test host/runner specs into account for VDiff diff duration test #14868

Merged
merged 1 commit into from
Dec 28, 2023
+9 −1
Merged

Test: Take test host/runner specs into account for VDiff diff duration test #14868

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion go/test/endtoend/vreplication/vdiff2_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ package vreplication

import (
"fmt"
"math"
"runtime"
"strings"
"testing"
"time"
Expand All @@ -29,6 +31,7 @@ import (
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/proto"

"vitess.io/vitess/go/vt/log"
"vitess.io/vitess/go/vt/sqlparser"
"vitess.io/vitess/go/vt/vttablet"

Expand Down Expand Up @@ -214,8 +217,13 @@ func testWorkflow(t *testing.T, vc *VitessCluster, tc *testCase, tks *Keyspace,
require.NoError(t, err, "could not parse --max-diff-duration %q: %v", diffDuration, err)
seconds := int64(dur.Seconds())
chunkSize := int64(100000)
perSecondCount := int64(1000000)
// Take the test host/runner vCPU count into account when generating rows.
perVCpuCount := int64(100000)
// Cap it at 1M rows per second so that we will create betweeen 100,000 and 1,000,000
// rows for each second in the diff duration, depending on the test host vCPU count.
perSecondCount := int64(math.Min(float64(perVCpuCount*int64(runtime.NumCPU())), 1000000))
totalRowsToCreate := seconds * perSecondCount
log.Infof("Test host has %d vCPUs. Generating %d rows in the customer table to test --max-diff-duration", runtime.NumCPU(), totalRowsToCreate)
for i := int64(0); i < totalRowsToCreate; i += chunkSize {
generateMoreCustomers(t, sourceKs, chunkSize)
}
Expand Down
Loading