Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from moby:master #179

Open
wants to merge 4,436 commits into
base: master
Choose a base branch
from
Open

[pull] master from moby:master #179

wants to merge 4,436 commits into from

Conversation

pull[bot]
Copy link

@pull pull bot commented Apr 27, 2021

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

jarqvi and others added 28 commits October 2, 2024 09:12
Signed-off-by: MohammadHasan Akbari <[email protected]>
history: remove records without attached blobs at startup
…tiple constraints

This fixes a problem with the new protobuf marshaling with the standard
library. LLB digests are now forced into deterministic marshaling to
ensure they produce the same digest when marshaled multiple times.

In addition, the marshal cache has also been fixed to work in
multi-threaded frontends with multiple different constraints.
Previously, if an LLB vertex was used in multiple goroutines and
marshaled concurrently, the cache would be broken. This could cause
certain problems when a specific node was used multiple times in the
same LLB tree.

Signed-off-by: Jonathan A. Sternberg <[email protected]>
llb: deterministic marshaling for protobuf and store results from multiple constraints
This test has been flaky, see #5384

Skip on Windows for now as investigations go on. This
is to allow the rest of the pipeline to remain
unpoisoned.

Signed-off-by: Anthony Nandaa <[email protected]>
llbsolver: add input validation to policy recompute
ARGs are inherited by downstream build stages, if declared in a base
stage. The documentation ambiguously stated that ARGs are out of scope
in other stages, where it really meant other stages that are not
child/downstream stages.

Signed-off-by: David Karlsson <[email protected]>
vendor: go.etcd.io/bbolt v1.3.11
tests: skip TestContextChangeDirToFile on Windows
docs: remove `from` limitation for onbuild
docs: fix incorrect information about arg scoping
client: allow non-octal chmod config for fileop.copy
The relative paths option for protoc generators doesn't work well when
it comes to dependencies. This simplifies the code generation to avoid
using `go generate` and to use one global command for protoc generation.

This is similar to docker/buildx#2713 since the
same problems with code generation occur here too.

Signed-off-by: Jonathan A. Sternberg <[email protected]>
Fix progress lines like `[6/4] step`

Signed-off-by: Tonis Tiigi <[email protected]>
Set error location for ONBUILD errors to the command
that pulled in the image with ONBUILD definitions.

Currently location was incorrectly set to the first
line as ONBUILD was parsed as a separate source code.

Location is handled both at parse time (for invalid
ONBUILD definition) and run time (when command set
via ONBUILD errors).

Signed-off-by: Tonis Tiigi <[email protected]>
Return error when AppArmor is unsupported and profile specified
This workaround was added in 414edfa, pending
changes in containerd. Those changes were included in containerd 1.7.12, which
was vendored in bce2f79, so the workaround
should no longer be needed.

I also reformatted the code, as there's no longer a need to wrap this function
since the list of arguments has been reduced.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
protobuf: normalize how protobuf files are generated
vtproto is an extra protobuf compiler that generates special methods
suffixed with `VT` that create typed and unrolled marshal and unmarshal
functions similar to gogo that can be used for performance sensitive
code. These extensions are optional for code to use but buildkit uses
them.

A codec is also included to utilize vtproto for grpc code. If the
package `github.com/moby/buildkit/util/grpcutil/encoding/proto` is
imported then vtproto will be used if it exists and otherwise it will
use the standard marshaling and unmarshaling methods.

This codec has an important difference from the default codec. The
default codec will always reset messages before unmarshaling. In most
cases, this is unnecessary and is only relevant for `RecvMsg` on
streams. In most cases, if we are passing in an existing message to this
method, we want to reuse the buffers. This codec will always merge the
message when unmarshaling instead of resetting the input message.

Signed-off-by: Jonathan A. Sternberg <[email protected]>
Multiple fixes for ONBUILD behavior
dockerfile: add support for non-octal COPY --chmod in labs
There is a possibility to get a digest mismatch error
if the metadata for previous download does not point to
a valid reference anymore.

To mitigate this, check that ref that etag points to
is still valid before using it.

Additionally `.cacheKey` property was not previously
set in the cases where old reference was reused. This
caused a case where even if the download needed to be
performed again, it always failed validation, even if
the digest had not actually changed since previous download.

There is still a small possibility that gc/prune request
will delete the downloaded record in between cachemap and
exec call and that the contents changes in the server
at that exact time. To fix that case we would need to
modify cachemap so that it can keep hold of references
until build is complete.

Signed-off-by: Tonis Tiigi <[email protected]>
dvdksn and others added 30 commits November 26, 2024 07:48
docs: add --checksum only supports sha256
tests: frontend/dockerfile: update integration tests for windows/wcow
ci: use edge releases of buildx
When recomputing digests happens, the metadata is lost. This previously
happened only with sources that were affected by source policies so it
wasn't noticed, but now any LLB Op that is rewritten is affected since
the gogo protobuf switch will cause all digests to be rewritten if the
frontend and buildkit are using different versions.

Signed-off-by: Jonathan A. Sternberg <[email protected]>
llbsolver: tie op metadata to the op before recomputing digests
chore: fix some function name in comment
Add maximum timeout to each test and log all buildkitd
goroutines when a test gets stuck. Currently tests can time out
and will print daemon logs but these would already be after
shutdown signal is sent to the daemon.

Signed-off-by: Tonis Tiigi <[email protected]>
This test can hit rate-limiting from Github and run long.

Signed-off-by: Tonis Tiigi <[email protected]>
Follow up on the #5554 PR series. Enabling a number
of previously skipped tests for Windows platform.

- [x] `testInvalidExporter`
- [x] `testParallelLocalBuilds`
- [x] `testCallInfo`
- [x] `testFrontendVerifyPlatforms`
- [x] `testClientCustomGRPCOpts`
- [x] `testRunValidExitCodes`

Count: 5

> NOTE: this is a top-to-bottom runthrough of all the tests
> in `client/client_test.go` that were straight-forward.
> The rest will require code changes, especially most needing
> `llb.AddMount` support on Windows. These will be addressed
> in separate follow-up PRs.

Signed-off-by: Anthony Nandaa <[email protected]>
…aces

integration: log goroutine trace on test timeout
Calling marshal changes the internal state of the op, for example
addCap() helper adds capability constraints. These can race with
same map being read by another Marshal call. Locking the Marshal
function itself also makes sure that the cache is not recomputed
in this case.

Signed-off-by: Tonis Tiigi <[email protected]>
no changes in vendored files

full diff: moby/moby@v27.4.0-rc.2...v27.4.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
no changes in vendored files

full diff: docker/cli@v27.4.0-rc.2...v27.4.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Tonis Tiigi <[email protected]>
Signed-off-by: Tonis Tiigi <[email protected]>
tests: client: enable a batch of integration tests for wcow (pt.2)
llb: avoid concurrent map write on parallel marshal
vendor: github.com/docker/docker, github.com/docker/cli v27.4.0
dockerfile: add test for step names stability
This is the third patch release of the 1.2.z release branch of runc. It
primarily fixes some minor regressions introduced in 1.2.0.

- Fixed a regression in use of securejoin.MkdirAll, where multiple
  runc processes racing to create the same mountpoint in a shared rootfs
  would result in spurious EEXIST errors. In particular, this regression
  caused issues with BuildKit.
- Fixed a regression in eBPF support for pre-5.6 kernels after upgrading
  Cilium's eBPF library version to 0.16 in runc.

full diff: opencontainers/runc@v1.2.2...v1.2.3
release notes: https://github.com/opencontainers/runc/releases/tag/v1.2.3

Signed-off-by: Sebastiaan van Stijn <[email protected]>
Dockerfile: update runc binary to v1.2.3
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2.1.0 to 2.2.0.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](softprops/action-gh-release@01570a1...7b4da11)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
…s/action-gh-release-2.2.0

build(deps): bump softprops/action-gh-release from 2.1.0 to 2.2.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.