chore: remove sharness tests ported to conformance testing

[laurentsenta]

Sister PR: ipfs/gateway-conformance#92

To complete the first iteration of conformance testing, I am going through all the tests, double-checking what was ported, and removing duplicate tests.

Tasks

• update conformance with the changes in 3da4e5b

[laurentsenta]

@lidel @hacdias I created this PR to review what was ported to conformance testing (tests related to Kubo's API or configurations won't be ported for example) and eventually remove duplicate code.

There are around 300 test cases over 13 files involved.

Are we willing to go through with this and eventually merge? We'll need to implement changes like 3da4e5b in conformance instead.

[galargh]

🥳 🎈 🎉

[lidel]

Thank you @laurentsenta for this herculean effort ❤️

I agree we want to remove duplicates, so we only maintain conformance testing, but there seems to be a set of "duplicates" which I feel we should still keep, because even tho it is a duplicate of gateway conformance test, it is also testing Kubo-specific things like configuration, or some Kubo-specific HTML responses, which we want to remove from conformance at some point.

See comments inline – some examples there, lmk if my thinking is correct, or if we have tests for these elsewhere.

[lidel]

⚠️ tests of Kubo-specific HTML behavior we have here should not be removed, as we have no other tests for these (conformance will not check for "Index of" or inspect links in breadcrumbs).

I know we have tests for this in boxo too, but that is not always enough.
Things like dnslink gw: hash column should be a CID link to cid.ipfs.tech could pass in boxo but be broken in Kubo due to invalid Gateway.PublicGateways config wiring.

MAybe rename this (and other tests like this) to t0115-gateway-kubo-specific-html-dir-listing.sh to make it very clear the tests we kept are specific to Kubo.

[laurentsenta]

We do have them in conformance, for example:
https://github.com/ipfs/gateway-conformance/blob/d921d6259689aca6a75774e31978ea8f88d3a667/tests/path_gateway_unixfs_test.go#L50-L65
(living in ipfs/gateway-conformance#92)

Later we'll implement HTML expectations to remove the Kubo-strings checks (ipfs/gateway-conformance#17).

[laurentsenta]

@lidel you thumb's up'ped, but I would like to be 100% sure, is that enough? Would you prefer I restore the file in this PR?

[laurentsenta]

@lidel last 6 commits are restores for the file you flagged, thanks again for the deep review, let me know if that's all good!

[laurentsenta]

Marking this PR ready for review, the failing sharness tests are the same in master.

[lidel]

Ack, sharness was failing due to unrelated DNS issue – fixed it in https://github.com/protocol/infra/pull/1211.
and is now green.

Run out of time today, but will make final look next week (just want to make another pass).

[hacdias]

ping @lidel as you said you wanted to take one last pass

[lidel]

Thank you for addressing the feedbackn and once again, for driving this cleanup @laurentsenta ❤️

Merging this sgtm, we keep some bare minimum for most important surface (subdomains, used in Brave and ipfs-companion) + kubo-specific behavior,
but the coverage efforts should happen in https://github.com/ipfs/gateway-conformance.

(I am still a bit anxious we've missed something due to the size of changes, and lose coverage around niche edge case, but merging this is a lesser evil, and a forcing function for all of us to invest time into specs and gateway-conformance 👍 )