1.41.2

Merged PRs

dolt

• 8093: go/libraries/doltcore/remotestorage: internal/reliable: Recv: Fix a race where a completed state machine run and a canceled parent context could return a nil response message with a nil error.
• 8090: dolt sql shell slash redux
• 8087: Bug fix: Use correct log position in Dolt to MySQL replication heartbeats
  Ensure heartbeat events sent from a Dolt primary to a MySQL replica have the latest nextLogPosition populated, otherwise the MySQL replica will close the binlog event stream.
• 8086: define schema for dolt_schemas table
  dolthub/doltgresql#454 depends on this PR.
• 8082: /docker/{docker-entrypoint.sh,serverDockerfile}: change image to pass all args to dolt sql-server command
  This PR fixes #8079. Now when running dolthub/dolt-sql-server if the args dolt sql-server are passed to the image, it will error. This will also prevent accidentally starting two Dolt servers in the container.
• 8081: Fixed keyless secondary indexing for Doltgres
  • Companion PR: dolthub/doltgresql#452
    This PR fixes two issues with creating secondary indexes for Doltgres types. The first deals with handlers, as we were not adding a nil handler for the additional hash type, which would cause a panic as the counts were not equal (all non-Extended types should have a matching nil handler).
    The second issue was due to the reuse of an ExtendedTupleComparator. When creating a new ExtendedTupleComparator, we pass in the previous TupleTypeHandler to handle all non-Extended types. If the previous TupleTypeHandler was ExtendedTupleComparator and the new one was also ExtendedTupleComparator, then we could end up with a misinterpretation of data that could lead to incorrect results, as the handler assumed a different type than the actual type. This has been changed so that ExtendedTupleComparator will always use the inner comparator of a previous ExtendedTupleComparator. For now this will always be the default comparator, but if we ever add another one, then this should properly handle that change.
• 8080: Re-enable doltgres sysbench scripts
• 8078: Archive index rework to make loading faster
  The initial impl of archive indexes over optimized for space. This resulted in being 10x slower to load the index of archives than noms table files. To address this:
  • Dropped the end to end compression of the index
  • Dropped the use of var ints for offset deltas and chunk refs
  • Altered the use of byte span offsets, and instead used a end-offset approach which requires no delta processing on load.
  • Used only slices of primitive types in the index memory. Constant time read path with a little more complexity, but allows us to read directly off disk into memory.
    Testing indicates that on a 41 Gb archive file, this returned load performance to match classic table files, and the size of the index increased by about 350Mb (total ~ 1Gb)
• 8077: /go/libraries/doltcore/remotestorage/chunk_fetcher.go: fix nil pointer
  We observe dolthubapi can crash with the following nil pointer error:

  panic: runtime error: invalid memory address or nil pointer dereference
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x29e14d1]
  goroutine 399548427 [running]:
  github.com/dolthub/dolt/go/libraries/doltcore/remotestorage.fetcherRPCDownloadLocsThread.func3()
  external/com_github_dolthub_dolt_go/libraries/doltcore/remotestorage/chunk_fetcher.go:266 +0xf1
  golang.org/x/sync/errgroup.(*Group).Go.func1()
  external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
  created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 399548420
  external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
  

  This pr aims to prevent this.
• 8076: Bump golang.org/x/image from 0.10.0 to 0.18.0 in /go
  Bumps golang.org/x/image from 0.10.0 to 0.18.0.
  Commits

  • 3bbf4a6 tiff: Validate palette indices when parsing palette-color images
  • 6c5fa46 go.mod: update golang.org/x dependencies
  • 55c4ab6 go.mod: update golang.org/x dependencies
  • 0057a93 tiff: fix function name in comment
  • 9e190ae webp: disallow multiple VP8X chunks
  • 445ab0e go.mod: update golang.org/x dependencies
  • 240a51a font/sfnt: support early version 0 OS/2 tables
  • c20bbc3 draw: simplify some calls to fmt.Fprintf
  • 491771c draw: merge draw_go117.go into draw.go
  • 4aa0222 go.mod: update go directive to 1.18
  • Additional commits viewable in compare view
  
  [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
  Dependabot commands and options

  
  You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/dolt/network/alerts).
• 8073: Added schema to index creation
  In Doltgres, whenever we would create an index, we would use the empty schema as the destination (the default value for the schema name). This meant that the updated table with an index was saved into the empty schema, which is incorrect since Doltgres always has a schema. This adds the schema to index creation, along with several other locations that it should be in.

go-mysql-server

• 2583: [stats] Disable histogram bucket merging for now because it mutated shared memory
  Merging buckets in the current format is unsafe:
  • we collect statistics for an index where two buckets have overlapping values
  • we execute a join using the index with overlapping values, and use a merge algorithm to combine those buckets. The merged bucket is synthetic, but the statistics used for the join is also synthetic, so this all works as expected.
  • a future indexscan selects the compressed range from before, accessing one of the synthetic buckets created by the join
  • we error invalid bucket type: *stats.Bucket at the end of the indexscan when adding the filtered histogram with a synthetic back to the implementor-type statistic
    Edited mergeOverlappingBuckets to not share memory, but also I'm not sure if merging buckets is a common performance win in most cases, so disabling for now
• 2581: [stats] populate types for nil zeroing
• 2577: calling JSON_EXTRACT and JSON_VALUE with a path that has an out-of-bounds array access should return SQL NULL, not an error.
  The jsonpath module returns an error when performing a lookup with an out-of-bounds array index. We need to capture that error and return nil for the lookup operation instead.
• 2576: fix type and precision for unix_timestamp
  builds off of: dolthub/go-mysql-server#2573
• 2572: fix for table_catalog for information_schema.tables

vitess

• 355: New functions to create PreviousGtids events, and to update event checksum

Closed Issues

• 8079: dolthub/dolt-sql-server image doesnt work correctly when regular dolt commands are supplied to it as arguments
• 8054: Checked out a branch but data is still obtained from the main branch
• 8051: Flyway and dolt: missing performance_schema
• 8052: select from a subquery with information_schema: command denied to user 'restadmin'@'%'

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	2.07	2.91	1.4
groupby_scan	13.22	17.01	1.3
index_join	1.34	5.37	4.0
index_join_scan	1.27	2.57	2.0
index_scan	34.33	53.85	1.6
oltp_point_select	0.18	0.46	2.6
oltp_read_only	3.49	7.56	2.2
select_random_points	0.34	0.75	2.2
select_random_ranges	0.39	0.89	2.3
table_scan	34.33	54.83	1.6
types_table_scan	74.46	142.39	1.9
reads_mean_multiplier			2.1

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.13	6.09	0.7
oltp_insert	3.75	3.02	0.8
oltp_read_write	8.58	13.95	1.6
oltp_update_index	3.89	3.07	0.8
oltp_update_non_index	3.89	3.02	0.8
oltp_write_only	5.37	6.43	1.2
types_delete_insert	7.7	6.67	0.9
writes_mean_multiplier			1.0

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	99.28	32.71	3.0
tpcc_tps_multiplier			3.0

Overall Mean Multiple	2.03