Apko.lock contains duplicated repositories when combined from multiple files

[sfc-gh-ptabor]

In https://github.com/chainguard-dev/rules_apko/pull/64/files/78c551af671b24f2b5c80a404005761b90147bbe#diff-9912b6950a30993bc94f913745c42a5d3b00be7b99eb96fe6f4b6342886a5420 we observed lock files to be generated with duplicates like:

    {
        "name": "dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64",
        "url": "https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz",
        "architecture": "x86_64"
      },
      {
        "name": "dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64",
        "url": "https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz",
        "architecture": "x86_64"
      },
      {
        "name": "dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64",
        "url": "https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz",
        "architecture": "x86_64"
      },

It would be better if only unique entries were placed (especially if all (name, url, arch) matches:

The code to fix is here:

apko/pkg/build/types/image_configuration.go

Line 88 in 9541220

repos := append([]string{}, baseIc.Contents.Repositories...)

In theory dups can be also created here (but are less likely to happen, but just for sale of consistency we can call unique here as well:

• https://github.com/chainguard-dev/apko/blob/95412200ebe139d324ef0de8ee7d56f615ba90d2/internal/cli/lock.go#L188C45-L188C57

• apko/pkg/build/apk.go

  Line 34 in 9541220

  repos := sets.List(sets.New(bc.ic.Contents.Repositories...).Insert(bc.o.ExtraRepos...))