Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support neon's sql-over-websocket protocol #955

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

nicksrandall
Copy link
Contributor

@nicksrandall nicksrandall commented Oct 3, 2024

Neon is a popular postgres provider that is an open source alternative to AWS Aurora. In order to support connecting to their database in more runtimes (like cloudflare workers), they have implemented a "sql-over-websocket" protocol that these runtimes can use to connect to their database with less "cold-start" latency than other approaches (like hyperdrive).

If you're open to landing this change in this repo, that would be awesome! That said, I understand that this use-case might be niche enough that you may not want to sign up for the ongoing maintenance. If that is the case, I will likely just publish a fork to our own npm namespace to support this use-case.

Notes

  • Most of this is just directly copied from the "cf" module. I'll do my best to highlight the things that were actually changed.

Potential future work

  • Neon also supports sql-over-http which has the lowest startup latency but there is some limitations (like it doesn't support interactive transactions). It would be really nice to bake in some logic that this library could use the http protocol for simple one-shot queries and use the ws protocol for everything else?

tcp.readyState = "opening";
const rootURL = host + "/v2" + "?address=" + host + ":" + port;
const socketURL = "wss://" + rootURL;
tcp.ws = new WebSocket(socketURL);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously, the big change here is that we're using WebSocket instead of cloudflare's "connect" utility.

}

async function startTls(host) {
throw new Error("Postgres SSL connections are not supported yet");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The connection goes over secure web sockets (wss://) so we don't need to double encrypt the content. As such, we don't support tls from postgres.

keep_alive && socket.setKeepAlive && socket.setKeepAlive(true, 1000 * keep_alive)
const s = StartupMessage()
write(s)
AuthenticationCleartextPassword()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This utilizes neon's support for "pipelining" these initial messages to the db. We send the cleartext password and ready for query events before the database actually requests those messages. This cuts down the number of round trips needed to the initialize the connection.

You read more about this here: https://neon.tech/blog/quicker-serverless-postgres

x === 49 ? ParseComplete : // 1
x === 116 ? ParameterDescription : // t
x === 84 ? RowDescription : // T
x === 82 ? noop : // R
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can safely ignore the "Authentication" request because we've already sent the cleartext password.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant