meta: Status update and v2.0.0 holy grail requests

[nhooyr]

I've been gone from this library for a while now but I'm back now. I'm going to be working on it full time till the end of the year.

I'm going to start by fixing all the minor outstanding issues and adding all currently requested features without breaking any backwards compatibility. I want v1 of this library to be rock solid and feature complete to the extent it makes sense.

But after that, I want to work on a v2 where I change the API a bit here and there to entirely perfect it. I don't anticipate major changes as I quite like the API. It has stood the test of time. My main irk with the API is that's a little too constrained. I don't think it should be necessary to sacrifice any performance whatsoever in using my library vs gorilla or gobwas.

I want the comparison to say there is absolutely no rational reason to use any other library except this one if you're writing WebSockets with Go except for stability (though v1 of my library is quite widely used now too though still about 10x less than Gorilla).

The aim is to strike an even better balance between performance and being idiomatic. I have a few ideas but nothing's finalized. I'm also a massively better engineer than I was in 2018 (5 years ago) so there's lots of code quality to improve too.

If anyone has suggestions, please share them here. Ideally those rooted in the long term use of my library. As I fix up v1 and my ideas become very concrete, I'll publish them here first to get feedback before I implement.

cc @tailscale @coder

[nhooyr]

v1 will be maintained alongside v2 for as long as it's feasible. For security issues I'd say at least 5 years if not more. After my push these next few months, I don't expect it'll need much maintenance work except for occasional minor fixes.

I know it sounds rich to promise such a long timeline given my very recent 2 year absence but my absence had good reasons that will not repeat.

[nhooyr]

Ooo, I'll maintain v1 indefinitely. I'll just reimplement it in terms of v2.

[s0kil]

@nhooyr Great!, We are using the lib in prod, thanks for your work.

[emersion]

Thanks a lot for your dedication!

[balog-avaya]

@nhooyr terrific news, thank you for this fantastic library

[maggie44]

Is there a branch of fork that the work will take place on, to follow progress?

Annoyingly GitHub chooses to abandon useful dates on some posts and chooses to write 'commented last month' on your initial post, so I have no idea if this thread is 15 days old or 45 days old or whether the question is a little premature. Thanks GitHub.

[nhooyr]

@maggie44 Yea sorry it's been exactly a month. I had to stay on a bit longer with Terrastruct than I expected. No other branch to follow progress, will be on dev here.

[maggie44]

Good to know, thanks. I am at the beginning of a new project, been going through my web socket vendor options. The comparison section on the README is very convincing as is the code, and this post is very convincing too.

I came here through a Reddit post that may be worth adding to if you want to highlight that this is being resurrected: https://www.reddit.com/r/golang/comments/zu80o0/gorillawebsocket_or_melody/

[nhooyr]

Haha thanks glad you're considering my library.

I don't have a reddit account anymore unfortunately. I deleted mine after the whole third party app debacle earlier this year. I would appreciate it if you could mention it.

[maggie44]

Haha thanks glad you're considering my library.

I don't have a reddit account anymore unfortunately. I deleted mine after the whole third party app debacle earlier this year. I would appreciate it if you could mention it.

Afraid I don't have one either. Hopefully someone will do the honours.

Thanks for your work on the library. Starting to implement today and will follow along with the progress 👍 (aware of potential breaking changes in v2, will cross that bridge when I get there).

[kenshaw]

@nhooyr really great to see you back working on this, as I've been using this package for some time for various WebSocket use cases. Thank you for all your hard work!

If you're going to go to v2, would it be possible to also reconsider the package path/name? Now that Go modules are used everywhere, it really is much nicer/cleaner to have github.com/... as the package path. A humble request as a avid fan of your dev work!

[nhooyr]

Thank you for your kind words @kenshaw

Can you clarify the benefit of using a github url for the import path vs a vanity import like I have now? With Go modules in particular.

[DavidS-ovm]

[edit: just found #311, don't mind me]

Please add this to your v2 list:

"github.com/golang/protobuf/proto" is deprecated: Use the "google.golang.org/protobuf/proto" package instead.deprecated(default)
Package proto provides functionality for handling protocol buffer messages. In particular, it provides marshaling and unmarshaling between a protobuf message and the binary wire format.

See https://developers.google.com/protocol-buffers/docs/gotutorial for more information.

Deprecated: Use the "google.golang.org/protobuf/proto" package instead.

[kenshaw]

The change would cause it to be more standardized/similar to most other Go packages that are built/developed by an individual -- that is, the majority of packages built by individuals are on GitHub, GitLab or some other 3rd-party hosted service. It ensures some bit of future continuity, and is useful to be able to browse directly to the URL. I do realize your URL currently redirects to the GitHub URL. However, there are times when personal websites go down, and one might not remember what the canonical repo location is.

Additionally, there are custom scripts that monitor updates (not using the Go modules toolchain and libraries), such as bash scripts in closed source repos that are used to build dependency lists and artifacts. It can be quite the problem (or impossible!) to modify/alter these scripts to recognize (via way of an allowlist, or other mechanism) to support these custom package URLs, especially when not the owner/maintainer.

Again, I'd like to emphasize how much I appreciate your work! I'm extensively using your package, and would have chosen to use it for chromedp had it been available when I first wrote it! Would love to look at using it again once you get to v2! Looking forward to it ... !

[rjeczalik]

However, there are times when personal websites go down, and one might not remember what the canonical repo location is.

This is easily fixable by deploying a simple static page using GitHub Pages. Since both the Pages and Repositories are hosted by the same infrastructure, it does not really matter from availability viewpoint whether you use vanity import paths in Go.

Additionally, there are custom scripts that monitor updates (not using the Go modules toolchain and libraries), such as bash scripts in closed source repos that are used to build dependency lists and artifacts. It can be quite the problem (or impossible!) to modify/alter these scripts to recognize (via way of an allowlist, or other mechanism) to support these custom package URLs, especially when not the owner/maintainer.

Would it be possible for you to give an example of such "custom scripts"? There are a lot of packages in Go ecosystem using vanity import paths, Kubernetes and all the tooling around it to name a few, and I am pretty curious about such request to not use vanity import paths, as it is the first time I hear it and would like to learn more.

[kenshaw]

@rjeczalik While I can't share any specific scripts, I can describe the situation. There are financial institutions that have closed infrastructure, where their networks have limited external accessibility.

In order to get a package / library, a request needs to be put into writing for it to be added to their internal downloads, or it needs to be vendored with their code. As part of these systems, there is much custom work that has been done to collect the dependencies to create audit trails, as well as for compliance reasons. Since much of these scripts work on "allow lists", one has to put into writing a separate request. Because so many Go packages are hosted on GitHub, there are exceptions to just allow, eg, github.com/ paths.

My request is simply that it would be simpler and easier for all involved if it had a github.com/ path.

[daenney]

My request is simply that it would be simpler and easier for all involved if it had a github.com/ path.

That seems a little presumptuous, as evidenced by everyone else not having this problem. Custom import domains aren't a rarity in Go. The consequence of compliance tooling not being able to understand that shouldn't get exported to the Go ecosystem and create an obligation on everyone else to rename an import path and only use a subset of available code hosting solutions because some institution wants to use a particular library.

Resolving a custom import path domain to the code hosting platform can be done in a single line of shell script:

curl https://nhooyr.io/websocket/ | xmllint --html --xpath "string(//meta[@name='go-import']/@content)" -

There are many other tools you could use, but given both cURL and libxml2 (which provides xmllint) ubiquitous nature I doubt that should cause any complications even in regulatory constrained environments. That command results in the following output:

nhooyr.io/websocket git https://github.com/nhooyr/websocket

With that, it shouldn't be too hard to integrate that into any automated script and scanners, and clear any import paths that resolve to a trusted code hosting platform automatically. You could do this check every time, or cache that information and refresh it once per whatever-interval-you-need. That should also alleviate any concern about private/personal infrastructure staying up.

[thara]

You don't need cURL/libxml2 or other hacks to avoid this problem.
Just use replace directive in go.mod and we can download this module from GitHub directly.

such as:

replace nhooyr.io/websocket v1.8.11 => github.com/nhooyr/websocket v1.8.11

[mitar]

Replace directives are useful only for local development and not really something you can use in published code.

[thara]

@mitar That doesn't means that replace directive must not be used in production.

e.g. kubernetes
https://github.com/kubernetes/kubernetes/blob/22f3ce5e5ebfb62a6b86ab95bf0ef05588b8d297/go.mod#L226C1-L229

[mitar]

This then limits for example being able to directly run Kubernetes main command: golang/go#44840

But probably that is not the use case they want to support anyway.

[thara]

I see.

However, we could also says replace directive still has been useful to executable binaries in closed infrastructures.

[nhooyr]

https://coder.com/blog/websocket

Sorry for the misleading promises everyone. I hope you understand. Coder will take much better care of this library than I can.

Sincerely,
nhooyr

[maggie44]

https://coder.com/blog/websocket

Sorry for the misleading promises everyone. I hope you understand. Coder will take much better care of this library than I can.

Sincerely, nhooyr

@nhooyr, a common pattern is for people to start things like this on GitHub, stick around long enough for lots of people to become dependant on it, then just walk away, abandon the project and stop replying to concerns or reviewing PRs. That’s the complete opposite of what you have done here. Not only did you create a great tool, you remained connected to it for those who used it even while other priorities took over, and ultimately then went and found a reputable owner to support the project going forward! Your work here is a testament to how open source projects should be done. Thanks for the commitment.