Use MDNS to find DHT peers instead of discovery keys

[RangerMauve]

The use of MDNS in discovery-swarm gets kind of noisy as you get more and more archives that you're replicating.

I propose to reduce network traffic by using MDNS to discover other hyperswarm nodes to form a DHT that's used for the actual peer discovery.

The flow would look like:

• Peer loads up
• Listens for MDNS messages for hyperswarm nodes
• Sends out MDNS packets saying it's a hyperswarm node
• Sees peers over MDNS, adds them to a "local DHT"
• When doing an announce, send it to both the local and main DHTs
• When doing a lookup, search the local DHT first then the main one (Or parallel?)

Some cool properties:

• When a new peer connects, if any of the other nodes sees them, they'll be added to the DHT and see all other peers
• Don't need to broadcast as often since new peers will be boostrapped into the DHT and old ones will see the new peers
• Peer lookup is more unified

[mafintosh]

Another way to describe your flow is to simply resolve bootstrap.hyperswarm.local to yourself and use that as the bootstrap address.

I think your idea has merit, def worth trying out.

[pvh]

This sounds almost exactly like what we were hoping for, only we called it "peer discovery". In our use-case most documents have a small number of peers associated with them and there's a high degree of likelihood that the feeds you want will be found in a peer you're already talking to. (We're doing a lot of small-group collaboration rather than large-audience publication.)

[RangerMauve]

pvh: That sounds pretty relevant to the stuff @tinchoz49 is doing with @geut

[RangerMauve]

Is there any way to extract that logic into a reusable module?

[pvh]

We don't have an implementation. We've retreated to our discovery-cloud implementation which is just a websocket echo server at the moment and look forward to returning...

[RangerMauve]

@davidmarkclements @mafintosh thoughts on this?

[RangerMauve]

It'd be cool if we adhered to zeroconf. It'd make my life a bit easier for react-native apps via react-native-zeroconf

[RangerMauve]

Here's a conversation we had about this in the Dat working group on IRC:

• rangermauve: is anyone interested in talking about the MDNS changes I proposed? Use MDNS to find DHT peers instead of discovery keys #7
• pfrazee: I like the idea. minimize multicast as much as possible
• rangermauve: I was thinking of tackling it sometime after integrating hyperswarm into the SDK
• pfrazee: I dont see any problem there, mafintosh ?
• rangermauve: Maybe I should write up a DEP for it first? :P
• mafintosh: there might be some hidden complexities
• rangermauve: What sort of stuff should I watch out for?
• mafintosh: since it’s not service disc but peer disc rhe protocol might fight you. ie everyone is a server/client but you don’t wanna find yourself. that’s pretty easy in the current dns way
• rangermauve: Well, I pictured it as service disc for the "hyperswarm DHT" and then going from there. Like, a local-only hyperswarm DHT
• mafintosh: Ah, its the dht one sorry
• rangermauve: Yeah yeah. Basically, we reduce MDNS traffic to bootstrapping into a local DHT and then do all lookups against that
• mafintosh: it’s gonna be a lot more traffic for smaller networks and less for huge ones
• rangermauve: More traffic for MDNS advertising for every key?
• mafintosh: yea a lot more
• rangermauve: I would have thought that querying the DHT for a key would be less overall traffic. Where would it be coming from?
• mafintosh: announces and dht pings since the topology is small (For a dht scale)
• rangermauve: Hmm. What would be a good path forward for handling lots of keys on a local network like Ink and Switch did?
• mafintosh: it’s prob much more efficient to just group dns records and do a randomised delay to see if another peer responded for you. but if your local network is massive the dht is better. massive meaning 1000s of computers
• rangermauve: Yeah, I think that'll be something we'd run into in the solarpunk future with pirate mesh networks
• mafintosh: we actually support storing local addrs in the dht also. hat’s prob the way to scale that case assuming you have internet access
• rangermauve: K I'll update the issue with that and keep thinking, then. :P