Sign/Verify with Non-PGP Key Material

[tri-adam]

Description of the Pull Request (PR):

Add --key option to singularity sign and singularity verify, which expect PEM-encoded key material that will be used to sign/verify the image. Add end-to-end tests to cover --key usage.

Extend internal/app/singularity package to support signing with a signature.Signer and verifying with a signature.Verifier.

Centralize test keys and images that are used by multiple packages to test/.

This fixes or addresses the following GitHub issues:

• Related to Support sign/verify with X.509 certificates #1095

[dtrudg]

This is looking good to me. I have a couple of questions...

1. Should the '--key' flag have an associated env var, so that it's straightforward to set a default key for all verification operations etc.?

2. If the top level test directory is only going to contain test data, and not code for tests, would it be reasonable to use testdata instead? As I understand it, the go toolchain would then ignore any .go files in there.... which would make it a good place to 'hide' deprecated plugin examples that we may still need to test compile for a time, but don't want to keepin in examples, have appearing in go doc etc.

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

👍 on centralising the test data. I think there might be some more that is essentially duplicated between unit and e2e tests, that should be found and dug out at some point when we have some time.

[dtrudg]

Do we want to associate this with an env var also?

Potentially a user could set an env var to have the key used always, without needing to specify the flag, in an environment that requires that?

[dtrudg]

Ditto the signing - do we want to expose with an env var also?

[tri-adam]

1. Should the '--key' flag have an associated env var, so that it's straightforward to set a default key for all verification operations etc.?

Good idea, although this may introduce some subtlety for signing, since the type of signature being generated would then depend on the environment. Perhaps it would be worth adding some visual feedback?

Environment variable not set:

$ singularity sign image.sif 
Signing image with PGP key material
Signature created and applied to image.sif

Environment variable set:

$ export SIGN_KEY=private.pem
$ singularity sign image.sif 
Signing image with key material from 'private.pem'
Signature created and applied to image.sif

2. If the top level test directory is only going to contain test data, and not code for tests, would it be reasonable to use testdata instead? As I understand it, the go toolchain would then ignore any .go files in there.... which would make it a good place to 'hide' deprecated plugin examples that we may still need to test compile for a time, but don't want to keepin in examples, have appearing in go doc etc.

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

I wonder if it might be worth following the guidelines here, as I think the current structure was based on those recommendations to some extent? That wouldn't preclude us from having a test/testdata directory for the reasons you've articulated, while also supporting test code (e2e->test/e2e, for example).

[tri-adam]

Also going to raise a situation that just occurred to me... in the event that an image is signed both with PGP and non-PGP key material:

$ singularity sign image.sif
$ singularity sign --key private.pem image.sif

This can't be successfully verified, since using --key disables PGP key material as a source:

$ singularity verify image.sif 
Verifying image: image.sif
FATAL:   Failed to verify container: integrity: key material not provided for DSSE envelope signature

$ singularity verify --key public.pem image.sif 
Verifying image: image.sif
FATAL:   Failed to verify container: integrity: key material not provided for PGP clear-sign signature

Perhaps verify should instead consider --key to be additive to PGP key material sources?

[dtrudg]

I wonder if it might be worth following the guidelines here, as I think the current structure was based on those recommendations to some extent? That wouldn't preclude us from having a test/testdata directory for the reasons you've articulated, while also supporting test code (e2e->test/e2e, for example).

Okay, agreed. I don't really think that pattern is one we should necessarily be wedded to, for the reasons articulated in golang-standards/project-layout#117 - but we are sorta following it in places :-)

Perhaps verify should instead consider --key to be additive to PGP key material sources?

That would make sense to me... but, tbh, I'd prefer to throw it out to at least some minimal discussion if we can. The reason being that there is also a bit of a question on and / or behavior on PGP signatures alone... before we possibly combine them with DSSE.

This on the apptainer/sif fork also comes to mind once we think about signature combinations - apptainer/apptainer#462

R.E. the env var stuff... yup, the visual feedback makes perfect sense.

These latter 2 points seem like they can be captured as separate issues, and addressed. So putting a ✅ on here now.

[tri-adam]

These latter 2 points seem like they can be captured as separate issues, and addressed. So putting a ✅ on here now.

Thanks, I'll merge this. I've opened #1148 and #1150 to open up discussion on the relevant points.