Create js-kubo-rpc-client library #9125

lidel · 2022-07-19T13:41:52Z

ETA: 2022-12-16

Part of #8959 – based on discussions from IPFS Thing 2022
GO counterpart: #9124

Current state

Kubo shares RPC client with JS-IPFS:

https://www.npmjs.com/package/ipfs-http-client

After renaming GO-IPFS to Kubo, this state of things is arguably a bad user experience:

js-ipfs and Kubo are joined at the RPC hip – unable to improve their respective RPC interfaces, and pretending to be interoperable:
- js-ipfs provides gRPC endpoint while Kubo does not
- we don't test all RPC command because implementations diverged already
- making change to RPC in Kubo forces us to do the same change in js-ipfs, and makes it extremely hard to coordinate releases to minimize breaking changes to rpc client library users (example: fix: multibase in pubsub http rpc #8183 & fix(pubsub): multibase in pubsub http rpc js-ipfs#3922)

Desired state

Ideally, we should have one Kubo client at https://github.com/ipfs/js-kubo-rpc-client and kubo-rpc-client on NPM.

Keeping go- prefix is important because we want consistency across languages,
and Kubo team will maintain go-kubo-rpc-client too.

TODO

Below is a broad strokes plan, feel free to adjust if something else makes more sense:

Followups

Below are good improvements, but since we didn't have them with ipfs-http-client, we're not requiring them here.

CI test that fails when a new command is added to Kubo and is not supported by kubo-rpc-client yet
- trackled separately in Automation to detect new Kubo commands js-kubo-rpc-client#47

cc @2color @BigLep @achingbrain @SgtPooki

The text was updated successfully, but these errors were encountered:

SgtPooki · 2022-08-18T19:37:14Z

Created https://github.com/ipfs/js-kubo-rpc-client.
Used https://support.github.com/contact?tags=rr-forks&subject=Detach%20Fork&flow=detach_fork to detach the forked repo from ipfs/js-ipfs

SgtPooki · 2022-08-18T23:15:01Z

Trying to generate documentation, but I can't seem to do so and can't find instructions at https://github.com/ipfs/js-ipfs

It seems like aegir doesn't support generating docs anymore (aegir docs command no longer exists), and they used to use typedoc, so I'm going to use https://github.com/TypeStrong/typedoc directly

Action is running right now: https://github.com/ipfs/js-kubo-rpc-client/actions/runs/2886171949

Updated the action, ipfs/js-kubo-rpc-client@cc2823b, and the site is now visible at https://ipfs.github.io/js-kubo-rpc-client/

SgtPooki · 2022-08-19T19:17:13Z

Created PR to add js-kubo-rpc-client to ipfs/github-mgmt#43

SgtPooki · 2022-08-19T19:26:32Z

Using personal NPM_TOKEN for publish js-kubo-rpc-client to npm automatically with github action via aegir until I get access to npm service account to create more permanent NPM_TOKEN

SgtPooki · 2022-08-19T19:39:16Z

Unified CI should be enabled for js-kubo-rpc-client once protocol/.github#371 and ipfs/github-mgmt#43 are merged

SgtPooki · 2022-08-19T19:47:02Z

Updated NPM_TOKEN to use automated token from service account

SgtPooki · 2022-08-19T19:51:00Z

update https://github.com/ipfs/js-ipfsd-ctl to support both ipfsHttpModule and kuboRpcModule
(we will also rename binary package, but that is tracked separately in ipfs/npm-kubo#51)

Starting work on this now.

SgtPooki · 2022-08-19T20:10:20Z

published https://www.npmjs.com/package/js-kubo-rpc-client so I can bring the lib into https://github.com/ipfs/js-ipfsd-ctl

see ipfs/kubo#9125

SgtPooki · 2022-08-23T18:37:27Z

decide if js-kubo-rpc-client CI should reuse tests against Core APIs interface-ipfs-core

@lidel @BigLep @aschmahmann, who should I talk to about this? Which stakeholders make the final call for this item?

SgtPooki · 2022-08-23T18:49:30Z

Created milestone for tracking each task: https://github.com/ipfs/js-kubo-rpc-client/milestone/1

BigLep · 2022-08-23T18:58:23Z

decide if js-kubo-rpc-client CI should reuse tests against Core APIs interface-ipfs-core

@lidel @BigLep @aschmahmann, who should I talk to about this? Which stakeholders make the final call for this item?

@SgtPooki : We need to make sure that kubo (RPC server) and the js-kupo-rpc-client can interopoerate and that we don't break compatibility between them. We should take whatever tests we had in the past to bootstrap our efforts, but we no longer need to adhere to the "Core API" concept. Basically, I would write tests that invoke the js-kubo-rpc client directly, rather than through legacy "Core API" interfaces.

SgtPooki · 2022-08-23T23:51:26Z

So it sounds like Kubo RPC Client should have it's own tests, stripped out from interface-ipfs-core. And we should only bring over the tests that are applicable for kubo's supported RPC calls.

I can make sense of that I think. Thanks

achingbrain · 2022-09-23T14:40:50Z

In js-ipfs triage we have come up with a 4th option:

The neat trick of the Core API is that it's supported by js-ipfs in-process and both js-ipfs and Kubo over http. We have leveraged this to write the interop suite which is a) massively useful in proving the two implementations can talk to each other and b) about to cause significant type-related pain now that we want to allow the RPC APIs to diverge.

The 4th option is to just type the parts of the API that the interop suite uses, store those types in the interop suite and have it use them instead of the types from the Core API.

The theory is that both kubo-rpc-client and ipfs-http-client will have overlap with these types so tsc should be happy with them, and then both implementations (just Kubo really since js-ipfs is getting No New Features:tm:) are free to innovate in other areas of the API.

When Kubo breaks the API so badly that the interop suite types are no longer valid, hopefully we'll have the shiny new version of IPFS-on-js that will have it's own interop suite that maybe orchestrates kubo via some sort of API external to it a la go-libp2p-daemon's simple protobuf-based API and doesn't depend on the RPC client at all.

SgtPooki · 2022-09-23T17:07:49Z

type the parts of the API that the interop suite uses, store those types in the interop suite and have it use them instead of the types from the Core API.

Excellent. The interop work living in the interop package makes the most sense and should keep us all unblocked for the longest period. Type augmentations should help keep the diverging types manageable.

* feat: add js-kubo-rpc-client see ipfs/kubo#9125 * chore: update kubo-rpc-client package name * fix: use cross-env to set USE_KUBO_JS * fix: Remove USE_KUBO_JS env var * chore(deps): remove cross-env * docs(readme): update npm install cmds for using kubo-rpc-client * chore: log whether kubo-rpc-client or ipfs-http-client is in use * chore(tests): use kubo-rpc-client for commmunication with go controllers * chore(lint): fix lint failure * fix: addCorrectRpcModule returns the additionalOpts object * docs(readme): correct kubo-rpc-client name Co-authored-by: Alex Potsides <[email protected]> * fix: use correct logger namespace convention * fix: move kubo-rpc-client to devDeps * fix: support generics for Controller type * chore(deps): use [email protected] * feat!: convert to typescript Converts this module to typescript * chore: fix linting * chore: fix tests * chore: apply suggestions from code review * fix: resolve issues with js-kubo-rpc-client in typescript * fix: use kuboRpcModule for go in factory.spec.ts * fix: broken build from bad merge * chore: address PR comments Co-authored-by: Alex Potsides <[email protected]>

BigLep · 2022-11-19T00:08:47Z

FYI I made a "CI test that fails when a new command is added to Kubo and is not supported by kubo-rpc-client yet" as followup that isn' required for closing this issue since we didn't have that with the ipfs-http-client.

SgtPooki · 2022-12-05T18:46:47Z

Note that the latest work to close out the last action item in this issue (ipfs/js-kubo-rpc-client#5) is being handled via pull request ipfs/js-kubo-rpc-client#83.

I'll be meeting with @hacdias tomorrow (2022-12-06) to go over some of these tests and confirm next steps and handoff requirements.

Note that the original interface tests were testing against [email protected], and so the done criteria for closing this issue does not include updating interface tests to 0.17.

ninja-edit: There are technically two last items as seen on the milestone tracking the closing of this issue: https://github.com/ipfs/js-kubo-rpc-client/milestone/1, but the "revert patch-only release changes" is trivial so was not mentioned in the original comment.

SgtPooki · 2022-12-07T05:24:41Z

Latest actions required prior to handoff: ipfs/js-kubo-rpc-client#83 (comment)

SgtPooki · 2022-12-15T22:40:05Z

Note that ipfs/js-kubo-rpc-client#83 has been merged!

SgtPooki · 2022-12-16T00:37:50Z

We should be good to close this. will defer to @BigLep and @lidel

BigLep · 2022-12-17T07:04:06Z

Thanks @SgtPooki for bringing this to this point. The done criteria are satisfied, thus I think it's reasonable to close this out. I know there are additional tasks to do to further harden js-kubo-rpc-client and those are in ipfs/js-kubo-rpc-client#56

Closing - thanks again!

2color · 2023-01-13T12:26:33Z

With this issue closed, should we point all users of https://github.com/ipfs/js-ipfs/tree/master/packages/ipfs-http-client to this?

The NPM page could also use a notice: https://www.npmjs.com/package/ipfs-http-client

achingbrain · 2023-01-13T12:53:43Z

ipfs-http-client is still used to connect to js-ipfs, so not all users, but it could use a message saying to use the kubo rpc client with kubo.

lidel added the kind/maintenance Work required to avoid breaking changes or harm to project's status quo label Jul 19, 2022

lidel added this to IPFS Shipyard Team Jul 19, 2022

lidel moved this to 🥞 Todo in IPFS Shipyard Team Jul 19, 2022

This was referenced Jul 19, 2022

Point users at canonical kubo/client/rpc library #9124

Closed

Minimal rename of go-ipfs in 2022Q2 #8959

Closed

List other Kubo RPC API clients ipfs/ipfs-docs#1194

Closed

BigLep mentioned this issue Jul 24, 2022

[Tracking Issue] Cleanup Kubo RPC Client Story #9142

Open

3 tasks

SgtPooki self-assigned this Aug 18, 2022

SgtPooki added a commit to SgtPooki/js-ipfsd-ctl that referenced this issue Aug 19, 2022

feat: add js-kubo-rpc-client

20ea2f9

see ipfs/kubo#9125

SgtPooki mentioned this issue Aug 19, 2022

feat: add js-kubo-rpc-client ipfs/js-ipfsd-ctl#764

Merged

This was referenced Sep 2, 2022

Looking for a maintainer familiar with Python! 🙏 ipfs-shipyard/py-ipfs-http-client#316

Closed

Looking for a maintainer familiar with Java! 🙏 ipfs-shipyard/java-ipfs-http-client#194

Closed

This was referenced Oct 5, 2022

docs: ipfs-http-client -> kubo-rpc-client #9331

Merged

docs: mention kubo-rpc-client ipfs/ipfs-docs#1299

Merged

feat: use kubo-rpc-client ipfs-examples/js-ipfs-examples#517

Merged

tinytb added kind/enhancement A net-new feature or improvement to an existing feature effort/weeks Estimated to take multiple weeks labels Oct 21, 2022

SgtPooki mentioned this issue Nov 3, 2022

webui and ipfs-desktop do depend on js-ipfs/js-ipfs-http-api ipfs/js-ipfs#2542

Closed

SgtPooki mentioned this issue Nov 28, 2022

IPFS Desktop and Webui are decoupled from js-ipfs ipfs/ipfs-gui#107

Open

SgtPooki added the starmaps label Nov 28, 2022

BigLep assigned hacdias Dec 1, 2022

SgtPooki moved this from 🏃‍♀️ In Progress to 🔎 In Review in IPFS Shipyard Team Dec 16, 2022

This was referenced Dec 16, 2022

feat: kubo 0.18 with /quic-v1 and /webtransport ipfs/ipfs-desktop#2363

Merged

ipfs-http-client migration: Additional tasks ipfs/js-kubo-rpc-client#56

Open

BigLep closed this as completed Dec 17, 2022

Repository owner moved this from 🔎 In Review to 🎉 Done in IPFS Shipyard Team Dec 17, 2022

lidel mentioned this issue Dec 18, 2022

Breaks with Kubo 0.18 when ipfs.swarm.peers returns /quic-v1 ipfs/ipfs-companion#1118

Closed

SgtPooki mentioned this issue Jan 12, 2023

feat!: replace ipfs-http-client with kubo-rpc-client ipfs/ipfs-webui#2079

Closed

2color mentioned this issue Jan 13, 2023

Interplanetary Apps: Disco Chat - Blog Post ipfs/ipfs-blog#517

Merged

3 tasks

2color mentioned this issue Jan 13, 2023

docs: update-ipfs-http-client ipfs/js-ipfs#4293

Merged

rmlearney-digicatapult mentioned this issue Feb 25, 2024

README.md out of keeping with link to WIP milestone? ipfs/js-kubo-rpc-client#216

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create js-kubo-rpc-client library #9125

Create js-kubo-rpc-client library #9125

lidel commented Jul 19, 2022 •

edited by SgtPooki

Loading

SgtPooki commented Aug 18, 2022 •

edited

Loading

SgtPooki commented Aug 18, 2022 •

edited

Loading

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 19, 2022 •

edited

Loading

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 23, 2022

SgtPooki commented Aug 23, 2022

BigLep commented Aug 23, 2022

SgtPooki commented Aug 23, 2022

achingbrain commented Sep 23, 2022

SgtPooki commented Sep 23, 2022

BigLep commented Nov 19, 2022

SgtPooki commented Dec 5, 2022 •

edited

Loading

SgtPooki commented Dec 7, 2022

SgtPooki commented Dec 15, 2022

SgtPooki commented Dec 16, 2022

BigLep commented Dec 17, 2022

2color commented Jan 13, 2023

achingbrain commented Jan 13, 2023

Create js-kubo-rpc-client library #9125

Create js-kubo-rpc-client library #9125

Comments

lidel commented Jul 19, 2022 • edited by SgtPooki Loading

Current state

Desired state

TODO

Followups

SgtPooki commented Aug 18, 2022 • edited Loading

SgtPooki commented Aug 18, 2022 • edited Loading

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 19, 2022 • edited Loading

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 19, 2022

SgtPooki commented Aug 23, 2022

SgtPooki commented Aug 23, 2022

BigLep commented Aug 23, 2022

SgtPooki commented Aug 23, 2022

achingbrain commented Sep 23, 2022

SgtPooki commented Sep 23, 2022

BigLep commented Nov 19, 2022

SgtPooki commented Dec 5, 2022 • edited Loading

SgtPooki commented Dec 7, 2022

SgtPooki commented Dec 15, 2022

SgtPooki commented Dec 16, 2022

BigLep commented Dec 17, 2022

2color commented Jan 13, 2023

achingbrain commented Jan 13, 2023

lidel commented Jul 19, 2022 •

edited by SgtPooki

Loading

SgtPooki commented Aug 18, 2022 •

edited

Loading

SgtPooki commented Aug 18, 2022 •

edited

Loading

SgtPooki commented Aug 19, 2022 •

edited

Loading

SgtPooki commented Dec 5, 2022 •

edited

Loading