Container registry supporting referrers API?

[iain-macdonald]

Hi soci-snapshotter maintainers,

I’ve been playing around with the snapshotter for a little bit and I’m interested in trying to get the snapshotter working with a local container registry implementing the referrers API so that I don’t have to pass in the index digest for rpull commands.

I followed the instructions in GETTING_STARTED.md and they all succeeded. Then I poked around on my locally running container registry to see if I could find the referrers link between my container image and the soci index of that container. It looks like the registry mentioned in the docs (ghcr.io/oci-playground/registry:v3.0.0-alpha.1) supports the (a?) referrers API:

> curl localhost:5000/v2/rabbitmq/_oci/ext/discover | jq
...
{
  "extensions": [
    {
      "name": "distribution",
      "url": "insert link",
      "description": "distribution extension adds tag history and manifest list functionality",
      "endpoints": [
        "_distribution/registry/manifests",
        "_distribution/registry/taghistory"
      ]
    },
    {
      "name": "oci",
      "url": "https://github.com/opencontainers/distribution-spec/blob/main/extensions/_oci.md",
      "description": "oci extension enables listing of supported registry and repository extensions",
      "endpoints": [
        "_oci/ext/discover",
        "_oci/artifacts/referrers"
      ]
    }
  ]
}

And I have the image and soci index digest available locally:

> sudo ctr i ls
...
docker.io/library/rabbitmq:latest --> sha256:bb689f7356e9bbf4906e9c6781cf74202c3d18cadbf5ca9d887c63504a2594f4 100.5 localhost:5000/rabbitmq:latest → sha256:810eec5b9a1b89ef4c5b7e03e540300c98b4e69b8f6b7c41a5261cb0923cb998      

> sudo ./out/soci index list
...
sha256:478d3217d4e5f64c373b55633599eaf7c62516447b8ab542e1bf810b4f972f9c    1454    docker.io/library/rabbitmq:latest

But when I query the referrers API, I get back a 404 error:

> curl localhost:5000/v2/rabbitmq/_oci/artifacts/referrers?digest=sha256:810eec5b9a1b89ef4c5b7e03e540300c98b4e69b8f6b7c41a5261cb0923cb998      
404 page not found

I tried a few other things as well, including:

• Passing the digest of the manifest or a layer
• Querying .../_oras/artifacts/referrers instead of .../_oci/artifacts/referrers
• Using the ghcr.io/oci-playground/registry:latest and registry container registries
  Unfortunately, none of those worked for me.

Given that the soci-snapshotter code supports fetching the digest of the soci index using the referrers API, I assume that you’ve found and used a container registry that supports the referrers API. Could you let me know where I can find one?

Thanks!

[Kern--]

Hi, thanks for your interest in SOCI!

I've answered your actual question below, but I thought it would be worth highlighting that we recently added support for existing registries.

If you create your SOCI index with the following, it should work with most registries.

soci create --legacy-registry $IMAGE

We have an open issue to make this the default behavior so you won't need the --legacy-registry flag: #434

In case you're curious, this doesn't actually use the referrers API. The technical details are here: https://github.com/opencontainers/distribution-spec/blob/main/spec.md#unavailable-referrers-api

---

To answer your original question, the ORAS project that originally introduced the referrers API is deprecated and was replaced by an OCI working group. That group adopted the ORAS referrers API with some modification which will be released in the oci distribution spec v1.1.0. There's a release candidate that you can read at https://github.com/opencontainers/distribution-spec/blob/v1.1.0-rc1/spec.md

Of particular interest to your question, the referrers API was moved to /v2/<name>/referrers/<digest>. This endpoint does work with ghcr.io/oci-playground/registry:v3.0.0-alpha.1

$ curl localhost:5000/v2/rabbitmq/referrers/sha256:810eec5b9a1b89ef4c5b7e03e540300c98b4e69b8f6b7c41a5261cb0923cb998
{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[{"mediaType":"application/vnd.oci.artifact.manifest.v1+json","digest":"sha256:c40b7a7779363365328db1281214d78a6ae110407bd99d3be21e69cac2355cb9","size":1454,"annotations":{"com.amazon.soci.build-tool-identifier":"AWS SOCI CLI v0.1"},"artifactType":"application/vnd.amazon.soci.index.v1+json"}]}

Since the spec is still a release candidate, support across registry implementations is mixed so for now we'd recommend using the solution I mentioned above.

[iain-macdonald]

That did it for me. Looks like you've updated the documentation on the Getting Started page as well so hopefully no one else runs into this issue. Thanks, Kern (and team)!