Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Throttler: return app name in check result, synthesize "why throttled" explanation from result #16416

Merged
merged 41 commits into from
Jul 28, 2024
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
5fb9a6e
rename variable mysqlMetricThresholds to metricThresholds
shlomi-noach Jul 11, 2024
c6d7047
rename function mysqlMetricThresholdsSnapshot to metricThresholdsSnap…
shlomi-noach Jul 11, 2024
9eb59b3
rename *Interval variables away from 'mysql'
shlomi-noach Jul 11, 2024
9f2716a
rename *Interval variables away from 'mysql'
shlomi-noach Jul 11, 2024
685441e
rename aggregateMySQLProbes -> aggregateMetricResults. Cleanup tests
shlomi-noach Jul 11, 2024
dd11de1
rename MySQLThrottleMetrics -> ThrottleMetrics. Rename mysqlThrottleM…
shlomi-noach Jul 11, 2024
c9ac150
renamed files
shlomi-noach Jul 11, 2024
dd3868f
rename file
shlomi-noach Jul 11, 2024
2886bf3
eliminatye throttle/mysql package; move everything under throttle/base
shlomi-noach Jul 11, 2024
f6e8a55
Remove unused type
shlomi-noach Jul 11, 2024
1ea1b51
remove ctx argument
shlomi-noach Jul 11, 2024
1bddcc0
rename aggregateMetricResults -> aggregateTabletMetricResults
shlomi-noach Jul 11, 2024
ec62bbe
shuffle code around, moving aggregateTabletMetricResults into 'base' …
shlomi-noach Jul 11, 2024
c82ea9a
remove ctx where unused
shlomi-noach Jul 11, 2024
3423780
terminology: remove 'MySQL' where irrelevant. Add where relevant
shlomi-noach Jul 11, 2024
1f584a1
terminology: remove 'MySQL' where irrelevant
shlomi-noach Jul 11, 2024
85994ed
renamed getMySQLStoreMetric -> getScopedMetric
shlomi-noach Jul 11, 2024
2492d6d
comment wording
shlomi-noach Jul 11, 2024
d7c63c8
tests: explicitly disabling throttler so as to ensure it does not re-…
shlomi-noach Jul 11, 2024
114405f
CheckThrottlerResponse: include the matched app name
shlomi-noach Jul 11, 2024
692f7d5
CheckThrottler: return matched app name
shlomi-noach Jul 11, 2024
090f96d
adding missing files
shlomi-noach Jul 11, 2024
8c20483
adding missing files
shlomi-noach Jul 11, 2024
d913132
Merge branch 'main' into throttler-cleanup-types-and-names
shlomi-noach Jul 11, 2024
a8eeea7
update copyright year
shlomi-noach Jul 15, 2024
f6fda40
resolved conflict
shlomi-noach Jul 15, 2024
ca9b878
Merge branch 'main' into throttler-response-app
shlomi-noach Jul 17, 2024
ff04023
Merge branch 'main' into throttler-response-app
shlomi-noach Jul 18, 2024
eb0635d
remove unused httpClient
shlomi-noach Jul 18, 2024
032c87a
adding Summary in CheckThrottlerResponse
shlomi-noach Jul 18, 2024
3527745
evaluate and populate metric result Summary
shlomi-noach Jul 18, 2024
32d1f3c
apply matchedApp on indirect throttling. Add testing
shlomi-noach Jul 18, 2024
0f48902
proto: add ThrottledReason in VStreamRowsResponse and in VEvent
shlomi-noach Jul 18, 2024
af99924
ThrottlerClient returns CheckResult. Used in vreplication to extract …
shlomi-noach Jul 18, 2024
b31a552
copy _vt.vreplication.reason_throttled to _vt.schema_migrations.reaso…
shlomi-noach Jul 18, 2024
313675c
test 'reason_throttled' in endtoend
shlomi-noach Jul 18, 2024
5a3aea7
fix expected message
shlomi-noach Jul 18, 2024
f66da96
Merge branch 'main' into throttler-response-app
shlomi-noach Jul 24, 2024
2cd72f1
string format
shlomi-noach Jul 28, 2024
d6e3f45
remove extraneous comment indicator.
shlomi-noach Jul 28, 2024
0eef755
formalize expected check result summary based on app name
shlomi-noach Jul 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions go/test/endtoend/onlineddl/vrepl/onlineddl_vrepl_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -397,6 +397,8 @@ func TestSchemaChange(t *testing.T) {
assert.GreaterOrEqual(t, lastThrottledTimestamp, startedTimestamp)
component := row.AsString("component_throttled", "")
assert.Contains(t, []string{throttlerapp.VCopierName.String(), throttlerapp.VPlayerName.String()}, component)
reason := row.AsString("reason_throttled", "")
assert.Contains(t, reason, "is explicitly denied access")

// unthrottle
onlineddl.UnthrottleAllMigrations(t, &vtParams)
Expand Down
4 changes: 2 additions & 2 deletions go/vt/binlog/binlogplayer/binlog_player.go
Original file line number Diff line number Diff line change
Expand Up @@ -669,11 +669,11 @@ func GenerateUpdateHeartbeat(uid int32, timeUpdated int64) (string, error) {
}

// GenerateUpdateTimeThrottled returns a statement to record the latest throttle time in the _vt.vreplication table.
func GenerateUpdateTimeThrottled(uid int32, timeThrottledUnix int64, componentThrottled string) (string, error) {
func GenerateUpdateTimeThrottled(uid int32, timeThrottledUnix int64, componentThrottled string, reasonThrottled string) (string, error) {
if timeThrottledUnix == 0 {
return "", fmt.Errorf("timeUpdated cannot be zero")
}
return fmt.Sprintf("update _vt.vreplication set time_updated=%v, time_throttled=%v, component_throttled='%v' where id=%v", timeThrottledUnix, timeThrottledUnix, componentThrottled, uid), nil
return fmt.Sprintf("update _vt.vreplication set time_updated=%v, time_throttled=%v, component_throttled='%v', reason_throttled=%v where id=%v", timeThrottledUnix, timeThrottledUnix, componentThrottled, encodeString(MessageTruncate(reasonThrottled)), uid), nil
}

// StartVReplicationUntil returns a statement to start the replication with a stop position.
Expand Down
35 changes: 35 additions & 0 deletions go/vt/binlog/binlogplayer/binlog_player_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ import (
"testing"
"time"

"github.com/stretchr/testify/assert"

"vitess.io/vitess/go/mysql/replication"
"vitess.io/vitess/go/mysql/sqlerror"
querypb "vitess.io/vitess/go/vt/proto/query"
Expand Down Expand Up @@ -454,3 +456,36 @@ func TestReadVReplicationStatus(t *testing.T) {
t.Errorf("ReadVReplicationStatus(482821) = %#v, want %#v", got, want)
}
}

func TestEncodeString(t *testing.T) {
tcases := []struct {
in, out string
}{
{
in: "",
out: "''",
},
{
in: "a",
out: "'a'",
},
{
in: "here's",
out: "'here\\'s'",
},
{
in: "online-ddl is denied access due to lag metric value 94.821447 exceeding threshold 5",
out: "'online-ddl is denied access due to lag metric value 94.821447 exceeding threshold 5'",
},
{
in: "'a','b','c'",
out: "'\\'a\\',\\'b\\',\\'c\\''",
},
}
for _, tcase := range tcases {
t.Run(tcase.in, func(t *testing.T) {
out := encodeString(tcase.in)
assert.Equal(t, tcase.out, out)
})
}
}
374 changes: 199 additions & 175 deletions go/vt/proto/binlogdata/binlogdata.pb.go

Large diffs are not rendered by default.

126 changes: 108 additions & 18 deletions go/vt/proto/binlogdata/binlogdata_vtproto.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading