Asks for libp2p team 2019 Roadmap

[momack2]

We need to have concrete actionable asks for the libp2p team to be surfaced at their meetup next week and tentatively incorporated into their 2019 roadmap planning.

Rough ideas from various 2019 planning discussions:

• DHT crawler and debugging tool customization -- one command tell us what is going on
• Fast (<5 sec) mutable name resolution for any IPNS record
  • ipns-pubsub stable and enabled by default (package manager need from Feedback from package managers from the Reproducible Builds Summit notes#366)
• "There is a set of runnable benchmarks which can measure real world data transfer speed of the go-IPFS system as a whole against traditional file exchange tools" (a shared item with go-ipfs - OKRs - 2019 Q1 Go Core Dev team-mgmt#794)
• "Total wall-clock time for finding via the DHT and fetching data doesn’t exceed 3s (on average) for first byte across various node configurations (ex geographical distance)."
• p2p transport (aka bluetooth or equivalent)
  • support for a variety of device types (desktop/mobile/IoT)
  • support for nearby node discovery and fully p2p (offline) discovery
• Ability to add a 1m sharded index without disabling content routing

@ipfs/wg-captains @ipfs/go-team @ipfs/javascript-team - can you think of additional requests we should be surfacing to the libp2p team?

[hsanjuan]

Very concrete task that I would not mind if it gets done for me: libp2p/go-libp2p#328

[daviddias]

From top of my mind:

• Good API and system to manage connections
  • Track reputation (how much we benefited from a connection)
  • How much data was exchanged so far and in which protocols
  • Ability to take samples (i.e. track for the next 30 seconds and give me a snapshot)
• Stable implementation of the Rendezvous Protocol. Sunset for once the *-star transports (Removing the Central Rendezvous Points for the *-star transports libp2p/js-libp2p#134)
• Network Visualizer Network Visualization Tool libp2p/js-libp2p#215
• Stable implementation of WebRTC in Go and in Node.js
• Complete libp2p-utp transport for JS and Go
• Complete migration to Async Await/Generators Awesome Endeavour: Async Iterators libp2p/js-libp2p#266
• GossipSubJS
• A great RPC library that the ecosystem can follow simple-rpc libp2p/libp2p#23

[hsanjuan]

I would also like authorization mechanisms (whitelists/ blacklists) per registered protocol.

[lidel]

I believe most of this is wip or accounted for, but for the record, would be great to go over some asks related to improving experience in web browsers:

• js-libp2p-websocket-star
  • Plan to replace ws-star with ws-stardust? Consider replacing with stardust? (I've rewritten ws-star, and this time it's good) libp2p/js-libp2p-websocket-star#70
    • While at it, we need to land some means to stop suiciding js-ipfs startup if ws-star is down - inconsistent node startup behavior depending on attempted ws-star swarm binding js-ipfs#1619 & Experiment with websocket-star in js-ipfs fallback ipfs-shipyard/ipfs-share-files#63 (comment)
• js-libp2p-webrtc-star
  • Plan to get it into shape and enable by default?
    • The transport is disabled by default now because it still breaks browsers when run for a longer periods of time – Stable transports in the browser js-ipfs#1088 (comment)
  • Remove the need for centralized socket.io server as an exchange mechanism? - feat: Utilize interface-data-exchange to build a completly new webrtc-star libp2p/js-libp2p-webrtc-star#148 (comment)
    • For some use cases this could be alternative for "p2p"/Bluetooth transport mentioned by @momack2
• Plan/ETA for having Rendezvous Protocol with relay discovery enabled by default?
  (to avoid/remove ws-star)
• Prioritize improving performance of js-libp2p-mplex in browser context
  • Initial feedback (Avoid multiple conversions between node streams and pull streams libp2p/js-libp2p-mplex#89) from NearForm (ipfs/benchmarks), suggests it is a low hanging fruit for improving libp2p performance in browser

[daviddias]

js-libp2p-websocket-star
js-libp2p-webrtc-star

The plan is to sunset these two and avoid any "short term solutions that turn into multiple year solutions".

[momack2]

Sounds like we should definitely have a discussion re the various -star transports and what the plan is. Can folks also annotate their proposals with a rough priority? AFAIK these three are all super critical:

• Fast (<5 sec) mutable name resolution for any IPNS record - ipns-pubsub stable and enabled by default (package manager need from Feedback from package managers from the Reproducible Builds Summit notes#366)
• Total wall-clock time for finding via the DHT and fetching data doesn’t exceed 3s (on average) for first byte across various node configurations (ex geographical distance).
• Ability to add a 1m sharded index without disabling content routing

Any others that are P0?

• Rendezvous Protocol?
• Async Iterators?
• Peer Reputation?

What about P1?

[terichadbourne]

I believe this one is already on @mgoelzer's radar, but something to keep in mind during your planning is workshop content for the LibP2P Days / IPFS Conf week in late June. If there's content you're developing for that event that could be built to use ProtoSchool's platform as its delivery mechanism, that would be great for our curriculum!

[kevina]

Also, as part of the work towards the switch to base32 CidV0 it might be a good idea to switch the providers (and hence the DHT) to use raw multihashes instead of full CIDs. Full issue: https://github.com/libp2p/go-libp2p-routing/issues/32.

Since we (well at least go-ipfs) have a hack in place to allow lookup of blocks regardless of the CID version (basically we try both CIDv0 and CIDv1 when applicable) I am no longer sure if this should be a high priority, especially since it will prevent newer nodes from discovering CIDv1 context on older nodes.

[raulk]

p2p transport (aka bluetooth or equivalent)

^^ local peer exchange is important here.

[anacrolix]

If IPNS lookups require recursive resolution, <5s may not be achievable.

DHT can aim to resolve lookups within a period, but time to first byte includes IPFS behaviour out of its control. So the goal there must be only the DHT time component.

What is a sharded index?

[ghost]

I just dropped "Implement IPNS-over-PubSub-and-DNS" into the IPFS OKRs for Q1, which I'm committing to do. It'll be a new IPNS mechanism in addition to the DHT (already in prototype stage), and it aims to add realtime-ish capabilities. That'll take pressure off the DHT regarding IPNS performance.

[anorth]

Filecoin team would appreciate self-dial functionality: libp2p/go-libp2p#328

Alternatively, if it's decided to be definitely rejected, let us know so that we (and any other project) can be confident we need to work around it at a higher level.

[raulk]

@anorth I’ll dig into this soon. I think we’ll need some kind of virtual interface that short circuits such dials and loops them back into the host without touching the network. This feature should be explicitly opt-in on the host level, and protocols should declare they’re capable of handling reflexive streams, e.g. some protocols like DHT or pubsub cannot handle this as an invariant (and it makes no sense).

cc/ @Stebalien

[Stebalien]

This was an ask from cluster too. IMO, services that need to self-dial should be rewritten to not self-dial for performance however, I agree we should still support it.

(we have to consider how this interacts with the connection manager and/or make the connection manager less murder-happy)

[raulk]

re: peer self-dialling; high level design here: libp2p/go-libp2p#328 (comment)