Gateway: ensure consistent caching of DNS records

[hsanjuan]

The gateway code uses https://github.com/multiformats/go-multiaddr-dns which by default would use net.DefaultResolver and not cache anything. It also registers two default resolvers for .eth and .crypto. In this case using DoH urls for which it uses https://github.com/libp2p/go-doh-resolver, which does include caching of results.

This means some dns results are going to be cached while others depend on what DNS resolver Go is actually using (which depends on CGO iirc). When not using CGO, Go would use its own resolver and not cache anything. When using CGO, go would use the system's resolver which may or not cache stuff depending what is on the system.

Is my understanding correct @lidel ?

Ideally we should be caching all results. This has been used before in the ecosystem: https://github.com/rs/dnscache

[lidel]

Yes, we seem to lack cache for the default DNS resolver from OS. Having universal cache in boxo/gateway makes sense.

cc @hacdias - you have been looking at this in context of TTL, thoughts where would be best place to wire it up?

[hacdias]

I'm working on reworking the namesys package here: #459. This will allow us to bubble up TTLs to the gateway and actually use them in requests. I don't mind adding support for DNS TTLs - but perhaps in a separate PR.

I also want to mention that DNS caching has been talked here before: multiformats/go-multiaddr-dns#28

[hsanjuan]

TTL bubbling is a bit different from caching. Just saying there should be a layer (perhaps above Resolver) that does the caching consistently for everything that is resolved (based on TTL, why not).

[hacdias]

Yes, #459 also includes caching, but just for IPNS. If the DNSResolver were to return a TTL, namesys (which uses both IPNSResolver and DNSResolver) would cache resolved dns lookups.