Bug: DoH client is missing in-memory cache

[lidel]

Found the problem can be seen when opening http://localhost:3000/ipns/docs.ipfs.tech without subdomain isolation (e.g. commit 1ef0094).

Bunch of times /ipfs/assets/fooo is being resolved (separate problem, but will go away when I finish #30):

This edge case surfaced a weakness in our DoH implementation: no local cache.
And each time we trigger the same DNS lookup for _dnslink.assets:

This works fast only because we added cache in https://github.com/ipshipyard/waterworks-infra/pull/30, but is wasteful and eats into the pool of HTTP request our SW can make.

We need to have in-memory cache which ideally respects TTL from DNS, or at least caches results for 1 minute before asking upstream for an update.

[SgtPooki]

this should be resolved with the latest helia-verified-fetch and new dns updates. will confirm and then resolve

[2color]

I just tried with inbrowser.dev and it seems that even with ipfs/helia-verified-fetch#19 merged and included in the latest release, caching is not working properly. For example see in the second screenshot the logs to the console, most times it isn't resolved from cache.

I'm gonna look into this.

[2color]

ipfs/helia-verified-fetch#34 should resolve this.

I was under the false impression that we cache the DNSLink record in the datastore inside @helia/ipns, but that doesn't seem to be the case.

[2color]

Strange that even though we had a bug (resolved by ipfs/helia-verified-fetch#34) with the cache in verified-fetch, it seems that @multiformats/dns used by @helia/ipns wasn't returning from its own cache, which results in many DoH requests.

I'll investigate this

[SgtPooki]

we should be able to close this once the latest @helia/ipns is released

[SgtPooki]

helia ipns is released. we need to update verified-fetch and then update here.

https://github.com/ipfs/helia/releases/tag/ipns-v7.2.0

[SgtPooki]

This issue is resolved on main.

[2color]

Also confirming that from my tests this is resolved!

[SgtPooki]

@aschmahmann reported issues with multiple dns-queries still, but we're having trouble reproducting.

Adin, can you provide the commit hash (found in HTML of root domain) of the site you're using, and a screenshot of what you're seeing for "multiple dns queries" ?

[aschmahmann]

@SgtPooki not able to see within the same browser request the HTML of the root domain to get the commit hash, but I've got this as the loaded js and there's a tag on the JS file name that IIUC should help with versioning.

and here are the multiple dns requests

[SgtPooki]

@aschmahmann you should be able to go to <url>/#/ipfs-sw-config to get the config page, which still uses the same index.html, which will have the commit hash. That will make tracking things down a lot easier.

[SgtPooki]

@aschmahmann do you still get this error on inbrowser.dev?

[SgtPooki]

confirmed this is happening intermittently for Adin on latest. might be a race condition? might be CORS pre-flight being blocked causing issues? we need to dive in deeper, but it's a hard to repro problem that is inconsistent

[SgtPooki]

I believe this was fixed by #435 & #436

@aschmahmann (or anyone) can you confirm this is still happening on inbrowser.dev ?

[github-actions]

Oops, seems like we needed more information for this issue, please comment with more details or this issue will be closed in 7 days.