--compressed-caching=false doesn't disable compression

[bombela]

The doc says:

--compressed-caching Compress the cached layers. Decreases build time, but increases memory usage. (default true)

Which leads one to believe this is disabling the compression of layers.
This is not the case as far as my understanding of the code, followed by a quick test.

What this flag really does, is disable the in RAM caching of compressed data in the library google/go-containerregistry/pkg/v1/tarball. To save on CPU time by avoiding re(de?)-compressing the same data multiple time. In exchange, this appears to consume a tremendous about of RAM as reported by many issues. Which is consistent with with my understanding of the code, that it is holding the full layer in memory. A common word for it is memoization.

References:

• https://github.com/GoogleContainerTools/kaniko/blob/main/pkg/executor/push.go#L354
• https://github.com/google/go-containerregistry/blob/main/pkg/v1/tarball/layer.go#L136

A quick test to confirm this flag doesn't control the compression of layers:

cat > Dockerfile <<EOF
FROM alpine:3.16
COPY . /a
COPY . /b
COPY . /c
EOF

docker run -v ./:/v -ti bitnami/kaniko:1.23.2 --dockerfile=/v/Dockerfile \
	--no-push --tar-path=/v/cctrue.tar --compressed-caching=true

docker run -v ./:/v -ti bitnami/kaniko:1.23.2 --dockerfile=/v/Dockerfile \
	--no-push --tar-path=/v/ccfalse.tar --compressed-caching=false

echo "#### cctrue"
tar -tf cctrue.tar
echo "#### ccfalse"
tar -tf ccfalse.tar

Output:

#### cctrue
sha256:12d2b3fe86e5722bda3648ccd8402a67fb9ee18c142d71b7ac60f92fb9e393cc
a88dc8b54e91eb6b19695ef7e04865926d4df23004f414a3ee86978617492d4d.tar.gz
303915328988a22dad3559aa6af7ad29b55076da9f6bdd58be25d1cc71069f24.tar.gz
d7d77856ba99ebafdff7aec06daf2b2a0b3f131c5ca7deb4c15b9d28ed80cc36.tar.gz
7cba05a07aa183f21b9acee8cebe9837caae7febf899d1a17d75892bfce96e0f.tar.gz
manifest.json
#### ccfalse
sha256:560a028b0b74de335292d0a6e086f3cebaa16413e601645af6f7464e0f6fb305
a88dc8b54e91eb6b19695ef7e04865926d4df23004f414a3ee86978617492d4d.tar.gz
fc9d0f195949895d1d45dec868b167f4c80ebd20bc116ac676925b44534cd8dd.tar.gz
b966dfeb8c95e028a939a2c4cc25e0e3937b5587b50fc6cebcdddb73345a59c3.tar.gz
c30066a7bba373044b32935a421152006be6ada4d169b557430f4ea8c7c35230.tar.gz
manifest.json

Assuming my findings are correct, we could change the doc to:

--compressed-caching Hold compressed data in memory to save on compressing multiple time the same data. Consumes as much memory as the compressed sizes of layers. True by default.