doltgres 0.4.0

[Porkepix]

Created by brew bump

---

Created with brew bump-formula-pr.

release notes

# Merged PRs
doltgresql

125: Added UUID type

This adds the UUID type. Ignore the serialization portion for now, I'll figure out a different serialization scheme later that should work for all types (including user-defined types). While this works, we need to forward the type information through to the connector so it can use the correct OID. For now though, this should be enough to unblock.
123: Fixed command docs generation

Sometimes, command docs would generate duplicate tests. This was due to the RNG selecting the same number multiple times. This fixes this issue, along with fixing all of the affected tests. There are still duplicate tests, but that is due to the synopses defining the duplicates, rather than the generation, which I think is fine.
122: support syntax parsing for INDEX statements

This PR adds syntax parsing support for:

CREATE INDEX
CREATE TABLE AS

It also splits postgres/parser/sem/tree/create.go file into create_*.go files for each specific statements.


121: Referencing columns within arrays

This makes use of an addition to Vitess and GMS that allows for name resolution to occur for injected expressions. The new ARRAY[...] smoke test demonstrates this.

Related PRs:

Allow InjectedExpr to handle name resolution dolthub/vitess#312
Allow InjectedExpr to handle name resolution dolthub/go-mysql-server#2322


120: Added BOOLEAN and BOOLEAN[] types

This adds rough support for both BOOLEAN and BOOLEAN[] types. This is primarily to show a working implementation of the new ExtendedType interface that was added to GMS to support these new types, demonstrating that we can now add standard PostgreSQL types, as well as array types (along with accompanying functionality, such as ARRAY[] support).

Related PRs:

Added a way for Doltgres to inject expressions dolthub/vitess#309
Added the ExtendedType interface dolthub/go-mysql-server#2312
Added storage capabilities for GMS Extended Types dolthub/dolt#7457


119: support parsing create extension and function statements

Added parsing support to following statements:

CREATE EXTENSION
CREATE FUNCTION
COMMENT ON EXTENSION
DROP EXTENSION
DROP FUNCTION


118: [auto-update-readme] by coffeegoddd
117: [auto-update-readme] by coffeegoddd
115: Update get-doltgres-correctness-job-json.sh
114: /testing/logictest/harness/doltgres_server_harness.go: catch error
113: /README.md: changes for automated perf updates
112: /.github/{scripts,workflows}: add script for updating readme, fix other workflows
109: /.github/workflows: enable nightly correctness, add latency on pull comment
108: /.github/{scripts,workflows}: add correctness workflows
107: /.github/scripts/performance-benchmarking/get-{postgres,doltgres}-doltgres-job-json.sh: set email template
106: support ddl for schema and database

This PR adds missing postgres syntax for:

CREATE SCHEMA
CREATE DATABASE
DROP DATABASE


105: /.github/workflows: skip tpcc, add nightly benchmarks and benchmarks on release
104: /.github/workflows/email-report.yaml: fix email report workflow
102: Db/fix ci
100: /.github/{actions,scripts,workflows}: add benchmarking workflows to ci
99: Better error handling

Server no longer always forcefully closes a connection when catching a panic, which is still very common because of unsupported features that panic, as well as unintended panics due to bugs.

This change also refactors the connection handling logic to extract a new ConnectionHandler type.

Also fixes a problem where a errors during a Query execution would hang a connection while it waited for a Sync message that never came. Added waitForSync flag in ConnectionHandler to manage this bit of state.
96: Updated contribution guide
95: Better prepared statement support

Now handles type casts for placeholders, as well as matching a larger variety of expressions.

Companion PR:

[no-release-notes] BinaryExpression refactor dolthub/go-mysql-server#2272
93: PostgreSQL function PR feedback
91: Added more PostgreSQL functions

This only includes quite a few functions, but doesn't include their tests. I was planning on including those too, but it's turning into a "more work than expected" type of situation, and if I'm going to solve those problems, I'd rather do it with all of the rest of the functions too, so that I can just get all the functions done for all the tests as once. This implements every function that I think we can implement for now based on our currently-supported types (without ripping out the implementation as soon as we get proper type support).

In the meantime, we can get these functions in, and get the tests in later.
89: add utility for building doltgres binaries
87: Prepared statement support

This PR implements postgres prepared statements with support for $1 style parameters in query strings. If this approach is reasonable, the next step would be to add support for additional types and lots more tests.

Also missing is support for the client manually specifying the types of parameters in a Parse message, which is required for certain statements where the type information cannot be determined based on referenced columns.

Companion PRs:

Support for doltgres prepared statements dolthub/go-mysql-server#2239

Define an ExtendedHandler interface to handle postgres specific interactions dolthub/vitess#299
86: support postgres syntaxes - 2

This PR includes some postgres-specific syntax support.

ALTER DATABASE
GRANT
REVOKE
ALTER DEFAULT PRIVILEGES


85: support some postgres syntaxes - 1

This PR includes some postgres-specific syntax support.

ABORT
ALTER AGGREGATE
ALTER COLLATION
ALTER CONVERSION


84: Release v0.3.1

Created by the Release workflow to update DoltgreSQL's version
83: starting doltgres server creates db in defined directory

Currently, doltgres uses dolt init when starting doltgres server with data dir set to empty directory. When there is no dolt or doltgres config user.name and user.email, it fails to create a new database with error message from dolt init, which suggests to set dolt user info. Current fix will not return error; instead, it will use user name, postgres and email, [email protected] if it's not set.
82: Added a framework for creating PostgreSQL functions

This is the 2nd version of my attempt at implementing functions.

The first iteration hijacked *tree.FuncExpr and special-cased names of functions that matched the built-in ones for Postgres. Then it returned a new node that approximated the functionality. In some cases this was as simple as returning a different *vitess.FuncExpr node that pointed to a different function, however some cases did not have direct analogues in MySQL. Originally I was going to ignore those, but since I was working on functions anyway, I decided to tackle them to get it over with, similar to my approach with the entire AST conversion. Well that's when I started running into two key issues:

No straightforward conversion for some functions
Different behavior for similar functions

The second was born from the first, as I needed to extend my tests to make sure my massive nodes actually worked for most cases. However, extending the tests showed some issues that are somewhat fundamentally different to how MySQL approaches functions, with a big one being overloading.

The first was a major issue though. I'll post an example in another comment, but some of the nodes became almost unreadable, and they also took forever to create (multiple hours per function, hence the continual delays). The only real maintainable alternative would be to skip the AST conversion pipeline and jump straight to the GMS expressions, but that posed its own issues. Not only is there a lot of boilerplate for expressions, but some of the more sophisticated aspects (such as the origin of values, which I'm calling their Source) are specific to PostgreSQL, So I instead decided to modify the entire set of functions by replacing all of the built-ins (Postgres doesn't want MySQL's built-ins anyway) with a custom "compiled" function structure.

So now, this mimics a bit how overloaded stored procedures work. We define a set of functions under a single name, and those functions become overloads for that name. Whenever a function is called, we evaluate all of the parameters, and use their types to look for an exact match. If one is not found, then we allow some types to cast to others, and check while rotating type casts. If we still don't find anything, then we assume that the function does not exist. In addition, some functions allow for some casts and others don't, so we also store the original type before the cast. This way, a function only needs to worry about its own inputs, and the rest is handled automatically.

This does create the consequence of needing to use specific types (Integer, Float, etc.) for parameters and the return value. I considered allowing native Go types and using a context to host the additional information, but that made it a bit more difficult to handle the overloading and automatic casting, but I could be convinced that it's a better alternative (and probably easier to upgrade too as we add more functions).

On the note of functions, this only adds a few as a proof-of-concept. It's astronomically quicker to write these compared to the AST approach, and I'm going to use that test generation framework to get better testing coverage for functions, since the testing scope is orders of magnitude smaller compared to permutations of every statement, and the think-write-eval loop of coming up with tests manually takes quite a bit of time.

Last thing, all of the files that are related to the function compilation are prefixed with 0_. I wanted to prefix them with a single underscore instead, but apparently Go ignores files that start with an underscore. I didn't want to just dump them together with the functions without any special indication, because they'll end up lost in the sea of files. For example, which files in server/ast aren't nodes? Only way to know is to scroll through all 141 files, so I wanted to avoid that. I also tried separating packages, but then that required either a lone file that would get lost, or an init() in every file that adds functions to the catalog, and both of those seemed worse.



Closed Issues

46: Add Homebrew formula
81: cannot run doltgres with data dir set to empty directory

[github-actions]

🤖 An automated task has requested bottles to be published to this PR.