Skip to content

Commit

Permalink
Modify the ExtractSingleSquash, adding policy.
Browse files Browse the repository at this point in the history
Overall, there are 2 "good things" done by this change:
1. Fix a bug where the current code tries each option every time.
The problem with doing that is really that the kernel mount option
didn't work very well.  It would fail with "already mounted", and
then squashfuse would end up getting used to mount over the top.

2. Fix race conditions in the current code.

overlay.Unpack starts a thread pool and tries to unpack all layers
at once.  That is fine if there are no duplicate layers.  But
if there are duplicate layers used by a stacker.yaml file, then
there were races on extraction.  The end result really was that things
would get mounted more than once.

Example stacker that shows this:

    l1:
      from:
        type: docker
        url: docker://busybox:latest
      run: |
        echo build layer 1

    l2:
      from:
        type: docker
        url: docker://busybox:latest
      run: |
        echo build layer 1

There, the busybox layer would get extracted multiple times.

3. Allow the user to control the list of extractors.
If they knew that they could not use kernel mounts (or could, but didn't
want to) or wanted to use unsquashfs only they can now do that.

   STACKER_SQUASHFS_EXTRACT_POLICY=kmount stacker build ..

or

   STACKER_SQUASHFS_EXTRACT_POLICY=squashfuse kmount stacker build ...

This adds a SquashExtractor interface, with 3 implementers
(KernelExtractor, SquashFuseExtractor, UnsquashfsExtractor).

A ExtractPolicy is basically a list of Extractors to try.
The first time ExtractPolicy is used it will try each of the Extractors
in order.  It then stores the result in .Extractor and uses that
subsequently.
  • Loading branch information
smoser committed Sep 28, 2023
1 parent f5d5b62 commit 0fc8954
Show file tree
Hide file tree
Showing 2 changed files with 313 additions and 132 deletions.
2 changes: 0 additions & 2 deletions pkg/overlay/pack.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,7 @@ func (o *overlay) Unpack(tag, name string) error {

pool := NewThreadPool(runtime.NumCPU())

log.Debugf("Hey, lets mount %d layers for %s:%s", len(manifest.Layers), tag, name)
for _, curLayer := range manifest.Layers {
log.Debugf("Layer: %s", curLayer.Digest)
pool.Add(func(ctx context.Context) error {
return unpackLayer(curLayer, cacheDir,
overlayPath(o.config.RootFSDir, curLayer.Digest, "overlay"))
Expand Down
Loading

0 comments on commit 0fc8954

Please sign in to comment.