Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepared statement support #87

Merged
merged 34 commits into from
Jan 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
19fbeb5
Added comPrepare
zachmu Dec 19, 2023
0688042
Define PortalData
zachmu Dec 19, 2023
1c779bf
Another caching change
zachmu Dec 21, 2023
648cadf
Checkpoint
zachmu Dec 21, 2023
e43c495
Close to first draft
zachmu Dec 28, 2023
a52a092
Prepared statement tests
zachmu Dec 28, 2023
b1ba5ab
Convert placeholder nodes
zachmu Dec 28, 2023
15d81a8
Adding object ids to prepare responses
zachmu Dec 28, 2023
37be300
Some fake scaffolding to make more inroads before fixing interfaces
zachmu Jan 2, 2024
e93d747
Bug fixes for bind varsD
zachmu Jan 3, 2024
0a313b2
Merge branch 'main' into zachmu/prepared
zachmu Jan 3, 2024
46fb598
refactoring: split out query / execute
zachmu Jan 3, 2024
f7623c0
Getting closer
zachmu Jan 3, 2024
a1782fd
Destroy the unnamed statement on query
zachmu Jan 3, 2024
70f5701
mising err handling
zachmu Jan 3, 2024
8b24280
Cleanup, added support for the Close message
zachmu Jan 3, 2024
f88dcb3
More prepared statement tests
zachmu Jan 4, 2024
fa8c9b2
Moar tests
zachmu Jan 4, 2024
cf63ea0
Moar tests
zachmu Jan 4, 2024
ce35cbd
Int64 support
zachmu Jan 5, 2024
c88bdad
Added skipped tests for varchar columns with no length
zachmu Jan 5, 2024
b684aa1
Better tests
zachmu Jan 5, 2024
9492d9c
Merge main
zachmu Jan 5, 2024
b02e649
Sorted oid consts
zachmu Jan 10, 2024
483e2cf
Refactoring for redundant testing infra
zachmu Jan 10, 2024
5f3d081
Added skip for broken text type
zachmu Jan 10, 2024
d98b275
Skip a couple more tests
zachmu Jan 10, 2024
46576b6
Development branch for vitess / gms
zachmu Jan 10, 2024
fd60949
Upgrade dolt
zachmu Jan 10, 2024
50442b6
Merge main
zachmu Jan 10, 2024
d2c8503
Bug fix for empty statement
zachmu Jan 10, 2024
4665fbb
Refactored the handleMessage method
zachmu Jan 10, 2024
f6a6159
Formatting
zachmu Jan 10, 2024
fcdaaba
Typo, upgraded deps
zachmu Jan 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ require (
github.com/PuerkitoBio/goquery v1.8.1
github.com/cockroachdb/apd/v2 v2.0.3-0.20200518165714-d020e156310a
github.com/cockroachdb/errors v1.7.5
github.com/dolthub/dolt/go v0.40.5-0.20240105180317-61c234610835
github.com/dolthub/dolt/go v0.40.5-0.20240110011351-84b9180295cc
github.com/dolthub/dolt/go/gen/proto/dolt/services/eventsapi v0.0.0-20231213233028-64c353bf920f
github.com/dolthub/go-mysql-server v0.17.1-0.20240104231423-dcf9acb9f61f
github.com/dolthub/go-mysql-server v0.17.1-0.20240110020052-1eabd6054d96
github.com/dolthub/sqllogictest/go v0.0.0-20201107003712-816f3ae12d81
github.com/dolthub/vitess v0.0.0-20240104220048-4b296d3a3d8b
github.com/dolthub/vitess v0.0.0-20240110003421-4030c3dac015
github.com/fatih/color v1.13.0
github.com/gogo/protobuf v1.3.2
github.com/golang/geo v0.0.0-20200730024412-e86565bf3f35
Expand All @@ -29,7 +29,6 @@ require (
github.com/twpayne/go-geom v1.3.6
golang.org/x/exp v0.0.0-20230522175609-2e198f4a06a1
golang.org/x/net v0.17.0
golang.org/x/sync v0.3.0
golang.org/x/sys v0.15.0
golang.org/x/text v0.14.0
)
Expand Down Expand Up @@ -139,6 +138,7 @@ require (
golang.org/x/crypto v0.17.0 // indirect
golang.org/x/mod v0.12.0 // indirect
golang.org/x/oauth2 v0.8.0 // indirect
golang.org/x/sync v0.3.0 // indirect
golang.org/x/term v0.15.0 // indirect
golang.org/x/time v0.1.0 // indirect
golang.org/x/tools v0.13.0 // indirect
Expand Down
12 changes: 6 additions & 6 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -214,8 +214,8 @@ github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZm
github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/dolthub/dolt/go v0.40.5-0.20240105180317-61c234610835 h1:mA/InHIYvvB/21Bmc6JsJVyX1tJNbqRrEfGxTq8rVX8=
github.com/dolthub/dolt/go v0.40.5-0.20240105180317-61c234610835/go.mod h1:TPOSgjcJDlGuqetwh2baQCs/c/QN7wgxp6IMEzoADuM=
github.com/dolthub/dolt/go v0.40.5-0.20240110011351-84b9180295cc h1:7C97S8tm3cKL4tZIKaudt4BTBOBgwdZ3ceSExwb+bNo=
github.com/dolthub/dolt/go v0.40.5-0.20240110011351-84b9180295cc/go.mod h1:+oni3DE3qkT79htI/fVogLu00bRTfdu15fL4A3KPr24=
github.com/dolthub/dolt/go/gen/proto/dolt/services/eventsapi v0.0.0-20231213233028-64c353bf920f h1:f250FTgZ/OaCql9G6WJt46l9VOIBF1mI81hW9cnmBNM=
github.com/dolthub/dolt/go/gen/proto/dolt/services/eventsapi v0.0.0-20231213233028-64c353bf920f/go.mod h1:gHeHIDGU7em40EhFTliq62pExFcc1hxDTIZ9g5UqXYM=
github.com/dolthub/flatbuffers/v23 v23.3.3-dh.2 h1:u3PMzfF8RkKd3lB9pZ2bfn0qEG+1Gms9599cr0REMww=
Expand All @@ -224,8 +224,8 @@ github.com/dolthub/fslock v0.0.3 h1:iLMpUIvJKMKm92+N1fmHVdxJP5NdyDK5bK7z7Ba2s2U=
github.com/dolthub/fslock v0.0.3/go.mod h1:QWql+P17oAAMLnL4HGB5tiovtDuAjdDTPbuqx7bYfa0=
github.com/dolthub/go-icu-regex v0.0.0-20230524105445-af7e7991c97e h1:kPsT4a47cw1+y/N5SSCkma7FhAPw7KeGmD6c9PBZW9Y=
github.com/dolthub/go-icu-regex v0.0.0-20230524105445-af7e7991c97e/go.mod h1:KPUcpx070QOfJK1gNe0zx4pA5sicIK1GMikIGLKC168=
github.com/dolthub/go-mysql-server v0.17.1-0.20240104231423-dcf9acb9f61f h1:nIJGTmtDxVgmnaou1FGNJwZs13RO9WyLT5GXN2ZCml4=
github.com/dolthub/go-mysql-server v0.17.1-0.20240104231423-dcf9acb9f61f/go.mod h1:XVhlCn7TOZvALss7hO4CKaJsydzi4p6zoKTX/pIvDH0=
github.com/dolthub/go-mysql-server v0.17.1-0.20240110020052-1eabd6054d96 h1:FDMByaljXrMExow4qE3qwQoyRbXku6GBy6jnqPjx4zg=
github.com/dolthub/go-mysql-server v0.17.1-0.20240110020052-1eabd6054d96/go.mod h1:z98pba7qbSvXiceU3NlUbJaYwITxc1Am06YjK6hexXA=
github.com/dolthub/ishell v0.0.0-20221214210346-d7db0b066488 h1:0HHu0GWJH0N6a6keStrHhUAK5/o9LVfkh44pvsV4514=
github.com/dolthub/ishell v0.0.0-20221214210346-d7db0b066488/go.mod h1:ehexgi1mPxRTk0Mok/pADALuHbvATulTh6gzr7NzZto=
github.com/dolthub/jsonpath v0.0.2-0.20230525180605-8dc13778fd72 h1:NfWmngMi1CYUWU4Ix8wM+USEhjc+mhPlT9JUR/anvbQ=
Expand All @@ -236,8 +236,8 @@ github.com/dolthub/sqllogictest/go v0.0.0-20201107003712-816f3ae12d81 h1:7/v8q9X
github.com/dolthub/sqllogictest/go v0.0.0-20201107003712-816f3ae12d81/go.mod h1:siLfyv2c92W1eN/R4QqG/+RjjX5W2+gCTRjZxBjI3TY=
github.com/dolthub/swiss v0.1.0 h1:EaGQct3AqeP/MjASHLiH6i4TAmgbG/c4rA6a1bzCOPc=
github.com/dolthub/swiss v0.1.0/go.mod h1:BeucyB08Vb1G9tumVN3Vp/pyY4AMUnr9p7Rz7wJ7kAQ=
github.com/dolthub/vitess v0.0.0-20240104220048-4b296d3a3d8b h1:isS4RQQIxNGku8NV/SrVGSyBoHtrgpYt0fd/zv53ix4=
github.com/dolthub/vitess v0.0.0-20240104220048-4b296d3a3d8b/go.mod h1:IwjNXSQPymrja5pVqmfnYdcy7Uv7eNJNBPK/MEh9OOw=
github.com/dolthub/vitess v0.0.0-20240110003421-4030c3dac015 h1:n45HAYH+kmlvZ+lZPKtJoserQJNwgQkyVWZAL7kJpn0=
github.com/dolthub/vitess v0.0.0-20240110003421-4030c3dac015/go.mod h1:IwjNXSQPymrja5pVqmfnYdcy7Uv7eNJNBPK/MEh9OOw=
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
Expand Down
123 changes: 105 additions & 18 deletions postgres/messages/row_description.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,85 @@ import (
"github.com/dolthub/doltgresql/postgres/connection"
)

const (
OidBool = 16
OidBytea = 17
OidChar = 18
OidName = 19
OidInt8 = 20
OidInt2 = 21
OidInt2Vector = 22
OidInt4 = 23
OidRegproc = 24
OidText = 25
OidOid = 26
OidTid = 27
OidXid = 28
OidCid = 29
OidOidVector = 30
OidPgType = 71
OidPgAttribute = 75
OidPgProc = 81
OidPgClass = 83
OidJson = 114
OidXml = 142
OidXmlArray = 143
OidPgNodeTree = 194
OidPgNodeTreeArray = 195
OidJsonArray = 199
OidSmgr = 210
OidIndexAm = 261
OidPoint = 600
OidLseg = 601
OidPath = 602
OidBox = 603
OidPolygon = 604
OidLine = 628
OidCidr = 650
OidCidrArray = 651
OidFloat4 = 700
OidFloat8 = 701
OidAbstime = 702
OidReltime = 703
OidTinterval = 704
OidUnknown = 705
OidCircle = 718
OidCash = 790
OidMacaddr = 829
OidInet = 869
OidByteaArray = 1001
OidInt2Array = 1005
OidInt4Array = 1007
OidTextArray = 1009
OidVarcharArray = 1015
OidInt8Array = 1016
OidPointArray = 1017
OidFloat4Array = 1021
OidFloat8Array = 1022
OidAclitem = 1033
OidAclitemArray = 1034
OidInetArray = 1041
OidVarchar = 1043
OidDate = 1082
OidTime = 1083
OidTimestamp = 1114
OidTimestampArray = 1115
OidDateArray = 1182
OidTimeArray = 1183
OidNumeric = 1700
OidRefcursor = 1790
OidRegprocedure = 2202
OidRegoper = 2203
OidRegoperator = 2204
OidRegclass = 2205
OidRegtype = 2206
OidRegrole = 4096
OidRegnamespace = 4097
OidRegnamespaceArray = 4098
OidRegclassArray = 4099
OidRegRoleArray = 4090
)

func init() {
connection.InitializeDefaultMessage(RowDescription{})
}
Expand Down Expand Up @@ -134,50 +213,58 @@ func (m RowDescription) DefaultMessage() *connection.MessageFormat {
return &rowDescriptionDefault
}

// VitessFieldToDataTypeObjectID returns a type, as defined by Vitess, into a type as defined by Postgres.
// VitessFieldToDataTypeObjectID returns the type of a vitess Field into a type as defined by Postgres.
// OIDs can be obtained with the following query: `SELECT oid, typname FROM pg_type ORDER BY 1;`
func VitessFieldToDataTypeObjectID(field *query.Field) (int32, error) {
switch field.Type {
return VitessTypeToObjectID(field.Type)
}

// VitessFieldToDataTypeObjectID returns a type, as defined by Vitess, into a type as defined by Postgres.
// OIDs can be obtained with the following query: `SELECT oid, typname FROM pg_type ORDER BY 1;`
func VitessTypeToObjectID(typ query.Type) (int32, error) {
switch typ {
case query.Type_INT8:
// Postgres doesn't make use of a small integer type for integer returns, which presents a bit of a conundrum.
// GMS defines boolean operations as the smallest integer type, while Postgres has an explicit bool type.
// We can't always assume that `INT8` means bool, since it could just be a small integer. As a result, we'll
// always return this as though it's an `INT32`, which also means that we can't support bools right now.
// OIDs 16 (bool) and 18 (char, ASCII only?) are the only single-byte types as far as I'm aware.
return 23, nil
return OidInt4, nil
case query.Type_INT16:
// The technically correct OID is 21 (2-byte integer), however it seems like some clients don't actually expect
// this, so I'm not sure when it's actually used by Postgres. Because of this, we'll just pretend it's an `INT32`.
return 23, nil
return OidInt4, nil
case query.Type_INT24:
// Postgres doesn't have a 3-byte integer type, so just pretend it's `INT32`.
return 23, nil
return OidInt4, nil
case query.Type_INT32:
return 23, nil
return OidInt4, nil
case query.Type_INT64:
return 20, nil
return OidInt8, nil
case query.Type_FLOAT32:
return 700, nil
return OidFloat4, nil
case query.Type_FLOAT64:
return 701, nil
return OidFloat8, nil
case query.Type_DECIMAL:
return 1700, nil
return OidNumeric, nil
case query.Type_CHAR:
return 1042, nil
return OidChar, nil
case query.Type_VARCHAR:
return 1043, nil
return OidVarchar, nil
case query.Type_TEXT:
return 25, nil
return OidText, nil
case query.Type_JSON:
return 114, nil
return OidJson, nil
case query.Type_TIMESTAMP, query.Type_DATETIME:
return 1114, nil
const OidTimestamp = 1114
return OidTimestamp, nil
case query.Type_DATE:
return 1082, nil
const OidDate = 1082
return OidDate, nil
case query.Type_NULL_TYPE:
return 25, nil // NULL is treated as TEXT on the wire
return OidText, nil // NULL is treated as TEXT on the wire
default:
return 0, fmt.Errorf("unsupported type returned from engine: %s", field.Type)
return 0, fmt.Errorf("unsupported type: %s", typ)
}
}

Expand Down
5 changes: 3 additions & 2 deletions server/ast/expr.go
Original file line number Diff line number Diff line change
Expand Up @@ -457,8 +457,9 @@ func nodeExpr(node tree.Expr) (vitess.Expr, error) {
case *tree.PartitionMinVal:
return nil, fmt.Errorf("MINVALUE is not yet supported")
case *tree.Placeholder:
//TODO: figure out if I can delete this
panic("this should probably be deleted (internal error, Placeholder)")
// TODO: deal with type annotation
zachmu marked this conversation as resolved.
Show resolved Hide resolved
mysqlBindVarIdx := node.Idx + 1
return vitess.NewValArg([]byte(fmt.Sprintf(":v%d", mysqlBindVarIdx))), nil
case *tree.RangeCond:
operator := vitess.BetweenStr
if node.Not {
Expand Down
19 changes: 18 additions & 1 deletion server/converted_query.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,11 @@

package server

import vitess "github.com/dolthub/vitess/go/vt/sqlparser"
import (
"github.com/dolthub/go-mysql-server/sql"
querypb "github.com/dolthub/vitess/go/vt/proto/query"
vitess "github.com/dolthub/vitess/go/vt/sqlparser"
)

// ConvertedQuery represents a query that has been converted from the Postgres representation to the Vitess
// representation. String may contain the string version of the converted query. AST will contain the tree
Expand All @@ -24,3 +28,16 @@ type ConvertedQuery struct {
String string
AST vitess.Statement
}

type PreparedStatementData struct {
Query ConvertedQuery
ReturnFields []*querypb.Field
BindVarTypes []int32
}

type PortalData struct {
Query ConvertedQuery
IsEmptyQuery bool
Fields []*querypb.Field
BoundPlan sql.Node
}
66 changes: 0 additions & 66 deletions server/implicit_commit.go

This file was deleted.

Loading