Improve Image Compaction Logic

[tri-adam]

When OptDeleteCompact is passed to (*FileImage).DeleteObject, an error is thrown if the object being deleted is not last in the SIF image:

sif/pkg/sif/create.go

Line 502 in 28ad31b

var errCompactNotImplemented = errors.New("compact not implemented for non-last object")

This makes compaction difficult to use, since the caller needs to be aware of this limitation, and work around it.

We should extend compaction to work regardless of the position of the object being deleted.

[tri-adam]

sylabs/oci-tools#62 (comment) is a useful case study. Here, the caller cannot easily determine whether an object is "last", and so this option isn't used (even though it may be desirable to compact the image.)

The documentation for OptDeleteCompact states the image will "be compacted following object deletion" without mention of the current requirement of having the specified object be last in the image. Given that, I wonder if it might be worth considering having OptDeleteCompact return a nil error when called with an object that isn't last? The semantics would then be along the lines of "compact if possible", which probably is what a caller expects.

The only reason I can think of to not do this is that perhaps a user might expect that OptDeleteCompact will de-fragment the image, getting rid of unused space between objects. I think we could address this through documentation, making it clear that this option will remove unused space at the end of the image only? And in the future if we wanted to support de-fragmentation, a separate option could be added for that.

@dtrudg any thoughts here?