Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

clusterversion: create 25.1 version #134750

Merged
merged 2 commits into from
Nov 15, 2024
Merged

clusterversion: create 25.1 version #134750

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
519a3de
clusterversion: create 25.1 version
RaduBerinde Nov 15, 2024
9f8b923
schemachanger: switch check to V25_1
RaduBerinde Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/generated/settings/settings-for-tenants.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -402,4 +402,4 @@ trace.span_registry.enabled boolean true if set, ongoing traces can be seen at h
trace.zipkin.collector string the address of a Zipkin instance to receive traces, as <host>:<port>. If no port is specified, 9411 will be used. application
ui.database_locality_metadata.enabled boolean true if enabled shows extended locality data about databases and tables in DB Console which can be expensive to compute application
ui.display_timezone enumeration etc/utc the timezone used to format timestamps in the ui [etc/utc = 0, america/new_york = 1] application
version version 1000024.2-upgrading-to-1000024.3-step-022 set the active cluster version in the format '<major>.<minor>' application
version version 1000024.3-upgrading-to-1000025.1-step-002 set the active cluster version in the format '<major>.<minor>' application
2 changes: 1 addition & 1 deletion docs/generated/settings/settings.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,6 @@
<tr><td><div id="setting-trace-zipkin-collector" class="anchored"><code>trace.zipkin.collector</code></div></td><td>string</td><td><code></code></td><td>the address of a Zipkin instance to receive traces, as &lt;host&gt;:&lt;port&gt;. If no port is specified, 9411 will be used.</td><td>Serverless/Dedicated/Self-Hosted</td></tr>
<tr><td><div id="setting-ui-database-locality-metadata-enabled" class="anchored"><code>ui.database_locality_metadata.enabled</code></div></td><td>boolean</td><td><code>true</code></td><td>if enabled shows extended locality data about databases and tables in DB Console which can be expensive to compute</td><td>Serverless/Dedicated/Self-Hosted</td></tr>
<tr><td><div id="setting-ui-display-timezone" class="anchored"><code>ui.display_timezone</code></div></td><td>enumeration</td><td><code>etc/utc</code></td><td>the timezone used to format timestamps in the ui [etc/utc = 0, america/new_york = 1]</td><td>Serverless/Dedicated/Self-Hosted</td></tr>
<tr><td><div id="setting-version" class="anchored"><code>version</code></div></td><td>version</td><td><code>1000024.2-upgrading-to-1000024.3-step-022</code></td><td>set the active cluster version in the format &#39;&lt;major&gt;.&lt;minor&gt;&#39;</td><td>Serverless/Dedicated/Self-Hosted</td></tr>
<tr><td><div id="setting-version" class="anchored"><code>version</code></div></td><td>version</td><td><code>1000024.3-upgrading-to-1000025.1-step-002</code></td><td>set the active cluster version in the format &#39;&lt;major&gt;.&lt;minor&gt;&#39;</td><td>Serverless/Dedicated/Self-Hosted</td></tr>
</tbody>
</table>
2 changes: 1 addition & 1 deletion pkg/build/version.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v24.3.0-alpha.3
v25.1.0-alpha.00000000
4 changes: 2 additions & 2 deletions pkg/ccl/logictestccl/testdata/logic_test/crdb_internal_tenant
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,7 +413,7 @@ select crdb_internal.get_vmodule()
query T
SELECT crdb_internal.release_series(crdb_internal.node_executable_version())
----
24.3
25.1

query ITTT colnames,rowsort
select node_id, component, field, regexp_replace(regexp_replace(value, '^\d+$', '<port>'), e':\\d+', ':<port>') as value from crdb_internal.node_runtime_info
Expand Down Expand Up @@ -501,7 +501,7 @@ select * from crdb_internal.gossip_alerts
query T
SELECT crdb_internal.release_series(crdb_internal.node_executable_version())
----
24.3
25.1

user root

Expand Down
2 changes: 1 addition & 1 deletion pkg/cli/testdata/declarative-rules/invalid_version
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@ invalid_version
debug declarative-print-rules 1.1 op
unsupported version number, the supported versions are:
latest
latest
1000024.3
1000024.2
1000024.1
13 changes: 7 additions & 6 deletions pkg/clusterversion/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,10 @@ avoid large changes on `master` which might cause merge conflicts for backports.

- [ ] Update `PreviousRelease` constant

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for keeping the runbook up-to-date!

- [ ] Add start version (e.g. `V24.2Start` with version `24.1-2`)
- [ ] Add start version (e.g. `V24.2Start` with version `24.1-2`) and add a new
first upgrade for it (in `upgrades/upgrades.go`).

- [ ] Update `roachpb.successorSeries` map and update `TestReleaseSeriesSuccessor`

- [ ] Update `pkg/build/version.txt` to the new version (e.g. `v24.2.0-alpha.00000000`)

Expand All @@ -208,13 +211,11 @@ avoid large changes on `master` which might cause merge conflicts for backports.
no longer have that code around. The data is obtained from the release branch
(e.g. `release-24.1`) using the `sql-bootstrap-data` utility:
```
./dev build sql-bootstrap-data
./bin/sql-bootstrap-data
./dev build sql-bootstrap-data && bin/sql-bootstrap-data
```
This will create a pair of files that need to be copied to
This will create a set of files that need to be copied to
`pkg/sql/catalog/bootstrap/data` on the `master` branch; it will also output
what code modifications need to be performed. Note that the BUILD.bazel will
- also need to be modified to include the new embedded files under `embedsrcs`.
what code modifications need to be performed.

- [ ] Update releases file:
```
Expand Down
24 changes: 16 additions & 8 deletions pkg/clusterversion/cockroach_versions.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,11 @@ const (
// to the system.table_metadata table
V24_3_AddTableMetadataCols

// V24_3 is CockroachDB v24.3. It's used for all v24.3.x patch releases.
V24_3

V25_1_Start

// *************************************************
// Step (1) Add new versions above this comment.
// Do not add new versions to a patch release.
Expand Down Expand Up @@ -314,6 +319,11 @@ var versionTable = [numKeys]roachpb.Version{
V24_3_UseRACV2Full: {Major: 24, Minor: 2, Internal: 20},
V24_3_AddTableMetadataCols: {Major: 24, Minor: 2, Internal: 22},

V24_3: {Major: 24, Minor: 3, Internal: 0},

// v25.1 versions. Internal versions must be even.
V25_1_Start: {Major: 24, Minor: 3, Internal: 2},

// *************************************************
// Step (2): Add new versions above this comment.
// Do not add new versions to a patch release.
Expand All @@ -327,19 +337,17 @@ const Latest Key = numKeys - 1
// MinSupported is the minimum logical cluster version supported by this branch.
const MinSupported Key = V24_2

// PreviousRelease is the logical cluster version of the previous release.
//
// Note: this is always the last element of SupportedPreviousReleases(); it is
// also provided as a constant for convenience.
// PreviousRelease is the logical cluster version of the previous release (which must
// have at least an RC build published).
const PreviousRelease Key = V24_2

// V24_3 is a placeholder that will eventually be replaced by the actual 24.3
// V25_1 is a placeholder that will eventually be replaced by the actual 25.1
// version Key, but in the meantime it points to the latest Key. The placeholder
// is defined so that it can be referenced in code that simply wants to check if
// a cluster is running 24.3 and has completed all associated migrations; most
// version gates can use this instead of defining their own version key if they
// only need to check that the cluster has upgraded to 24.3.
const V24_3 = Latest
// only need to check that the cluster has upgraded to 25.1.
const V25_1 = Latest

// DevelopmentBranch must be true on the main development branch but should be
// set to false on a release branch once the set of versions becomes append-only
Expand Down Expand Up @@ -398,7 +406,7 @@ func (k Key) String() string {
// cluster).
func SupportedPreviousReleases() []Key {
res := make([]Key, 0, 2)
for k := MinSupported; k < Latest; k++ {
for k := MinSupported; k <= PreviousRelease; k++ {
if k.IsFinal() {
res = append(res, k)
}
Expand Down
11 changes: 6 additions & 5 deletions pkg/cmd/release/update_releases.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,10 +126,10 @@ func processReleaseData(data []Release) map[string]release.Series {
continue
}

// For the purposes of the cockroach_releases file, we are only
// interested in beta and rc pre-releases, as we do not support
// upgrades from alpha releases.
if pre := v.PreRelease(); pre != "" && !strings.HasPrefix(pre, "rc") && !strings.HasPrefix(pre, "beta") {
// For the purposes of the cockroach_releases file, we are only interested
// in rc pre-releases, as we do not support upgrades from alpha or beta
// releases.
if pre := v.PreRelease(); pre != "" && !strings.HasPrefix(pre, "rc") {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice fix/solution 👍🏼

continue
}
// Skip cloud-only releases, because the binaries are not yet publicly available.
Expand Down Expand Up @@ -322,7 +322,7 @@ func releaseName(name string) string {
}

func generateRepositoriesFile(versions ...string) error {
client := httputil.NewClientWithTimeout(15 * time.Second)
client := httputil.NewClientWithTimeout(45 * time.Second)
cfgKeys := map[string]string{
"CONFIG_LINUX_AMD64": "linux-amd64",
"CONFIG_LINUX_ARM64": "linux-arm64",
Expand All @@ -334,6 +334,7 @@ func generateRepositoriesFile(versions ...string) error {
versionToCfgToHash[v] = make(map[string]string)
for cfgKey, cfg := range cfgKeys {
url := fmt.Sprintf("https://binaries.cockroachdb.com/cockroach-v%s.%s.tgz", v, cfg)
fmt.Printf("getting %s\n", url)
resp, err := client.Get(context.Background(), url)
if err != nil {
return fmt.Errorf("could not download cockroach release: %w", err)
Expand Down
4 changes: 0 additions & 4 deletions pkg/cmd/release/update_releases_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,6 @@ func Test_processReleaseData(t *testing.T) {
Predecessor: "22.2",
Withdrawn: []string{"23.1.0"},
},
"23.2": {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to confirm, this removal/cleanup is more related to advancing MinSupported to 24.3 (than to creating 24.3 and 25.1 start version gates, right?)

(mostly to clarify for when we refer to this for the next release cycle/task)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a test for the updated releases code, which we updated to ignore beta builds. This test runs against fixed data, the versions should be considered fictitious.

Latest: "23.2.0-beta.1",
Predecessor: "23.1",
},
}
require.Equal(t, expectedReleaseData, processReleaseData(data))
}
Expand Down
3 changes: 2 additions & 1 deletion pkg/cmd/roachtest/roachtestutil/mixedversion/mixedversion_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ var testPredecessorMapping = map[string]*clusterupgrade.Version{
"24.1": clusterupgrade.MustParseVersion("v23.2.4"),
"24.2": clusterupgrade.MustParseVersion("v24.1.1"),
"24.3": clusterupgrade.MustParseVersion("v24.2.2"),
"25.1": clusterupgrade.MustParseVersion("v24.3.0"),
}

//go:embed testdata/test_releases.yaml
Expand Down Expand Up @@ -244,7 +245,7 @@ func Test_choosePreviousReleases(t *testing.T) {
arch: vm.ArchAMD64,
numUpgrades: 3,
enableSkipVersion: true,
expectedReleases: []string{"23.2.4", "24.1.1", "24.2.2"},
expectedReleases: []string{"23.1.17", "23.2.4", "24.1.1"},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

x-referencing note from MinSupport bump comment (for when we do this task for the next cycle), that this change is a result of the randomized sequence of versions that the test decides on with that seed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, and it's all based on the testPredecessorMapping (not the actual release data)

},
}

Expand Down
72 changes: 36 additions & 36 deletions pkg/cmd/roachtest/roachtestutil/mixedversion/testdata/planner/skip_version_upgrades
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,72 +28,72 @@ ok
plan
----
Seed: 12345
Upgrades: v23.2.0 → v24.1.1 → v24.2.0 → <current>
Upgrades: v23.1.4 → v23.2.0 → v24.1.1 → <current>
Deployment mode: system-only
Plan:
├── install fixtures for version "v23.2.0" (1)
├── start cluster at version "v23.2.0" (2)
├── wait for all nodes (:1-4) to acknowledge cluster version '23.2' on system tenant (3)
├── install fixtures for version "v23.1.4" (1)
├── start cluster at version "v23.1.4" (2)
├── wait for all nodes (:1-4) to acknowledge cluster version '23.1' on system tenant (3)
├── run "initialize bank workload" (4)
├── start background hooks concurrently
│ ├── run "bank workload", after 3m0s delay (5)
│ └── run "csv server", after 5s delay (6)
├── upgrade cluster from "v23.2.0" to "v24.1.1"
├── upgrade cluster from "v23.1.4" to "v23.2.0"
│ ├── prevent auto-upgrades on system tenant by setting `preserve_downgrade_option` (7)
│ ├── upgrade nodes :1-4 from "v23.2.0" to "v24.1.1"
│ │ ├── restart node 4 with binary version v24.1.1 (8)
│ │ ├── restart node 1 with binary version v24.1.1 (9)
│ ├── upgrade nodes :1-4 from "v23.1.4" to "v23.2.0"
│ │ ├── restart node 4 with binary version v23.2.0 (8)
│ │ ├── restart node 1 with binary version v23.2.0 (9)
│ │ ├── run mixed-version hooks concurrently
│ │ │ ├── run "mixed-version 1", after 500ms delay (10)
│ │ │ └── run "mixed-version 2", after 5s delay (11)
│ │ ├── restart node 2 with binary version v24.1.1 (12)
│ │ └── restart node 3 with binary version v24.1.1 (13)
│ │ ├── restart node 2 with binary version v23.2.0 (12)
│ │ └── restart node 3 with binary version v23.2.0 (13)
│ ├── allow upgrade to happen on system tenant by resetting `preserve_downgrade_option` (14)
│ ├── run mixed-version hooks concurrently
│ │ ├── run "mixed-version 1", after 100ms delay (15)
│ │ └── run "mixed-version 2", after 500ms delay (16)
│ ├── wait for all nodes (:1-4) to acknowledge cluster version '24.1' on system tenant (17)
│ ├── wait for all nodes (:1-4) to acknowledge cluster version '23.2' on system tenant (17)
│ └── run "validate upgrade" (18)
├── upgrade cluster from "v24.1.1" to "v24.2.0"
├── upgrade cluster from "v23.2.0" to "v24.1.1"
│ ├── prevent auto-upgrades on system tenant by setting `preserve_downgrade_option` (19)
│ ├── upgrade nodes :1-4 from "v24.1.1" to "v24.2.0"
│ │ ├── restart node 1 with binary version v24.2.0 (20)
│ │ ├── restart node 3 with binary version v24.2.0 (21)
│ │ ├── restart node 4 with binary version v24.2.0 (22)
│ ├── upgrade nodes :1-4 from "v23.2.0" to "v24.1.1"
│ │ ├── restart node 1 with binary version v24.1.1 (20)
│ │ ├── restart node 3 with binary version v24.1.1 (21)
│ │ ├── restart node 4 with binary version v24.1.1 (22)
│ │ ├── run "mixed-version 1" (23)
│ │ ├── restart node 2 with binary version v24.2.0 (24)
│ │ ├── restart node 2 with binary version v24.1.1 (24)
│ │ └── run "mixed-version 2" (25)
│ ├── downgrade nodes :1-4 from "v24.2.0" to "v24.1.1"
│ │ ├── restart node 4 with binary version v24.1.1 (26)
│ │ ├── restart node 3 with binary version v24.1.1 (27)
│ │ ├── restart node 2 with binary version v24.1.1 (28)
│ │ └── restart node 1 with binary version v24.1.1 (29)
│ ├── upgrade nodes :1-4 from "v24.1.1" to "v24.2.0"
│ │ ├── restart node 3 with binary version v24.2.0 (30)
│ ├── downgrade nodes :1-4 from "v24.1.1" to "v23.2.0"
│ │ ├── restart node 4 with binary version v23.2.0 (26)
│ │ ├── restart node 3 with binary version v23.2.0 (27)
│ │ ├── restart node 2 with binary version v23.2.0 (28)
│ │ └── restart node 1 with binary version v23.2.0 (29)
│ ├── upgrade nodes :1-4 from "v23.2.0" to "v24.1.1"
│ │ ├── restart node 3 with binary version v24.1.1 (30)
│ │ ├── run "mixed-version 1" (31)
│ │ ├── restart node 2 with binary version v24.2.0 (32)
│ │ ├── restart node 1 with binary version v24.2.0 (33)
│ │ ├── restart node 2 with binary version v24.1.1 (32)
│ │ ├── restart node 1 with binary version v24.1.1 (33)
│ │ ├── run "mixed-version 2" (34)
│ │ └── restart node 4 with binary version v24.2.0 (35)
│ │ └── restart node 4 with binary version v24.1.1 (35)
│ ├── allow upgrade to happen on system tenant by resetting `preserve_downgrade_option` (36)
│ ├── wait for all nodes (:1-4) to acknowledge cluster version '24.2' on system tenant (37)
│ ├── wait for all nodes (:1-4) to acknowledge cluster version '24.1' on system tenant (37)
│ └── run "validate upgrade" (38)
└── upgrade cluster from "v24.2.0" to "<current>"
└── upgrade cluster from "v24.1.1" to "<current>"
├── prevent auto-upgrades on system tenant by setting `preserve_downgrade_option` (39)
├── upgrade nodes :1-4 from "v24.2.0" to "<current>"
├── upgrade nodes :1-4 from "v24.1.1" to "<current>"
│ ├── restart node 3 with binary version <current> (40)
│ ├── run "mixed-version 1" (41)
│ ├── restart node 2 with binary version <current> (42)
│ ├── run "mixed-version 2" (43)
│ ├── restart node 4 with binary version <current> (44)
│ └── restart node 1 with binary version <current> (45)
├── downgrade nodes :1-4 from "<current>" to "v24.2.0"
│ ├── restart node 2 with binary version v24.2.0 (46)
│ ├── restart node 3 with binary version v24.2.0 (47)
│ ├── restart node 1 with binary version v24.2.0 (48)
├── downgrade nodes :1-4 from "<current>" to "v24.1.1"
│ ├── restart node 2 with binary version v24.1.1 (46)
│ ├── restart node 3 with binary version v24.1.1 (47)
│ ├── restart node 1 with binary version v24.1.1 (48)
│ ├── run "mixed-version 1" (49)
│ └── restart node 4 with binary version v24.2.0 (50)
├── upgrade nodes :1-4 from "v24.2.0" to "<current>"
│ └── restart node 4 with binary version v24.1.1 (50)
├── upgrade nodes :1-4 from "v24.1.1" to "<current>"
│ ├── restart node 1 with binary version <current> (51)
│ ├── restart node 3 with binary version <current> (52)
│ ├── run "mixed-version 2" (53)
Expand Down
12 changes: 10 additions & 2 deletions pkg/cmd/sql-bootstrap-data/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ func main() {
tenantKeys, tenantHash := bootstrap.GetAndHashInitialValuesToString(12345)

fmt.Printf("\n// Commit the generated files and " +
" add the following declarations to the end of initial_values.go:\n")
"add the following declarations to the end of initial_values.go:\n")
writeDataFile(version, "system.keys", systemKeys)
writeDataFile(version, "system.sha256", systemHash)
writeDataFile(version, "tenant.keys", tenantKeys)
Expand All @@ -45,7 +45,15 @@ func main() {
systemHash: v%[1]d_%[2]d_system_sha256,
nonSystem: v%[1]d_%[2]d_tenant_keys,
nonSystemHash: v%[1]d_%[2]d_tenant_sha256,
}.build
}.build,
`, version.Major, version.Minor)

fmt.Printf("\n\n// Add these to BUILD.bazel, under embedsrcs:\n")
fmt.Printf(`
"data/%[1]d_%[2]d_system.keys",
"data/%[1]d_%[2]d_system.sha256",
"data/%[1]d_%[2]d_tenant.keys",
"data/%[1]d_%[2]d_tenant.sha256",
`, version.Major, version.Minor)
}

Expand Down
1 change: 1 addition & 0 deletions pkg/roachpb/version.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,7 @@ var successorSeries = map[ReleaseSeries]ReleaseSeries{
{23, 2}: {24, 1},
{24, 1}: {24, 2},
{24, 2}: {24, 3},
{24, 3}: {25, 1},
}

// ReleaseSeries obtains the release series for the given version. Specifically:
Expand Down
2 changes: 1 addition & 1 deletion pkg/roachpb/version_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ func TestReleaseSeriesSuccessor(t *testing.T) {
for ok := true; ok; r, ok = r.Successor() {
seq = append(seq, r.String())
}
expected := "20.1, 20.2, 21.1, 21.2, 22.1, 22.2, 23.1, 23.2, 24.1, 24.2, 24.3"
expected := "20.1, 20.2, 21.1, 21.2, 22.1, 22.2, 23.1, 23.2, 24.1, 24.2, 24.3, 25.1"
require.Equal(t, expected, strings.Join(seq, ", "))
}

Expand Down
Loading