Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

faster status updates #2615

Merged
merged 9 commits into from
Aug 1, 2024
Merged

faster status updates #2615

merged 9 commits into from
Aug 1, 2024

Conversation

max-hoffman
Copy link
Contributor

@max-hoffman max-hoffman commented Aug 1, 2024

System variables can be session, global, or both. sql.IncrementStatusVariable is a helper method that primarily helps the "both" category increment the global and session counters for certain variables. Threads_running is a global only variable that is incremented/decremented every begin/end query, and gets a lot of traffic. The old code used sql.IncrementStatusVariable to increment Threads_running, which was a particularly expensive way to increment a global var because (1) we'd make a new error for every call to the session updater, and (2) the extra map lookup is unnecessary. We don't do the extra map lookup now, and we weren't using the error return so I removed the return variable.

Note: this also refactors status variables to be explicitly initializated in the engine

bump/perf here: dolthub/dolt#8189

@max-hoffman max-hoffman requested a review from jycor August 1, 2024 17:56
Copy link
Contributor

@jycor jycor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

sql/core.go Outdated Show resolved Hide resolved
Copy link
Member

@zachmu zachmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind of crazy how much perf this unlocks, good find

sql/core.go Outdated
@@ -713,7 +713,7 @@ type StatusVariableRegistry interface {
// SetGlobal sets the global value of the status variable with the given name, returns an error if the variable is SessionOnly scope
SetGlobal(name string, val interface{}) error
// IncrementGlobal increments the value of the status variable by the given integer value
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should document failure behavior, kind of important

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added note about session-only nooping

v, ok := g.varVals[name]
if !ok || v.Variable().GetScope() == sql.StatusVariableScope_Session {
return sql.ErrUnknownSystemVariable.New(name)
return
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Appropriate to panic here? Basically always means a bug right?

Copy link
Contributor Author

@max-hoffman max-hoffman Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The globals list excludes sessionOnly*, so it's only ~297/678 of the total list. So we want to ideally not call this on session only, but noop if we do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The query type counters have both scopes unfortunately, so I had to leave those for now

sql/base_session.go Outdated Show resolved Hide resolved
@max-hoffman max-hoffman merged commit 5dac189 into main Aug 1, 2024
8 checks passed
@max-hoffman max-hoffman deleted the max/sync-status-updates branch August 1, 2024 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants