Decouple elasticsearch-js from the Node.js core APIs

[adamyeats]

Related to #1680.

🚀 Feature Proposal

I tried importing elasticsearch-js into a Deno project recently (imported via Skypack), but looks like it (or, more precisely, @elastic/transport) currently depends on Node.js' zlib library. I'm sure there are other Node.js APIs used across the codebase also. This makes usage of the library in Deno a no-go for the time being.

Motivation

As the library is named elasticsearch-js, IMHO it sort of implies that we support the broader JavaScript ecosystem with this library (client-side usage being the exception to the rule), rather than just Node.js. Decoupling elasticsearch-js from the core Node.js libraries would allow us to support not only Deno, but any other JavaScript runtimes that emerge.

It might be more pragmatic on the first pass to look into optionally shimming the Node.js core libaries somehow, this might save a lot of implementation work. However, I'm not sure if such a thing would be possible in the case of zlib we might need a better solution in that case.

[JoshMock]

Hey @adamyeats, thanks for the proposal! As was mentioned in #1680, Deno support is not at the top of our priority list right now, but it is encouraging to see adoption growing steadily. I'd like to explore this as time allows.

My biggest concern is how this would work with Undici, our default HTTP client. At first glance it appears to depend on at least assert, stream, async_hooks and buffer. I'd have to double-check how straightforward it would be to shim those before making any promises. 😎

[JoshMock]

Pinging @srikanthmanvi, in case you wanted to look at Deno adoption numbers. The numbers from the State of JavaScript survey in 2021 vs. 2022 are a very rough indicator of adoption rate.

[srikanthmanvi]

Thank you for the info @JoshMock .

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove the stale label, or leave a comment, or this will be closed in 14 days.

[JoshMock]

Relevant. Deno/Bun support is on our longer-term roadmap for this library, just not an immediate priority.

[karfau]

FYI: I'm able to use import * as elasticsearch from "npm:@elasticsearch/elasticsearch"; in Deno already without any issues.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove the stale label, or leave a comment, or this will be closed in 14 days.

[karfau]

Just to give a little update here, I finally managed to actually use the code that I claimed "works just fine on Deno": it doesn't work.

The client always throws a ConnectionError complaining about not having the right http protocol.

I just used the information from my debug output to connect to the host using the native fetch API to POST my documents, which works like a charm.

[JoshMock]

@karfau Can you provide some example code that did not work? Having some integration tests to run against Deno (and other runtimes) is going to be valuable when starting this work.

[karfau]

Is there something like a public demo elastic cloud that I can use for the setup? Or some free tier option?
I think I will be able to provide a minimal example.

[JoshMock]

If you have Docker, from the root of the repo you can run .buildkite/run-elasticsearch.sh, setting env vars for STACK_VERSION (e.g. 8.13.0) and DETACH=true if you want it to run in the background. This will pull down and start up a local single-node instance of Elasticsearch and print its URL. This is the setup we use to run our clients' integration tests against a real ES instance, so it's perfect for writing minimal examples around.

[JoshMock]

See #2341