Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Windows] Fix nerdctl build tar parsing error #3604

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

TinaMor
Copy link
Contributor

@TinaMor TinaMor commented Oct 28, 2024

PR Description

Linked Issue: containerd/nerdctl#3629

This PR separates output streams for the buildctl build command to avoid parsing errors specifically on Windows terminals.

The buildctl build command sends both TTY logs and tarball (binary) data to stdout, but Windows terminals cannot differentiate between these, causing errors like "archive/tar: invalid tar header." To resolve this, the tarball output is directed to a file, isolating it from TTY output. If a destination tar file is not specified, a default file is created, which is then loaded into the containerd image store.

@TinaMor TinaMor force-pushed the tinamor/fix-build-with-tty branch 3 times, most recently from 94cff78 to 2d18366 Compare October 29, 2024 07:35
    - Write buildctl output to terminal and tarball to file

Signed-off-by: Christine Murimi <[email protected]>
@TinaMor TinaMor changed the title Separate TTY and tar output in nerdctl build with tty Fix Windows tar parsing error by separating tty and tar outputs in buildctl Oct 29, 2024
@TinaMor TinaMor changed the title Fix Windows tar parsing error by separating tty and tar outputs in buildctl [Windows] Fix nerdctl build tar parsing error Oct 29, 2024
@TBBle
Copy link
Contributor

TBBle commented Nov 1, 2024

I need to work through this later, but I'm unclear how the current code works on non-Windows. How do those platforms distinguish a tar file from log output on stdout?

They shouldn't be passing through a termInal on the way, the terminal is where nerdctl's stdio is connected, so I don't understand what's different on Windows.

@apostasie
Copy link
Contributor

apostasie commented Nov 1, 2024

Hey @TBBle
I’ll have a look later today.
Curious too.

@AkihiroSuda AkihiroSuda added the area/ci e.g., CI failure label Nov 1, 2024
@TinaMor
Copy link
Contributor Author

TinaMor commented Nov 5, 2024

@TBBle, @apostasie

Still investigating the issue. This is a temp fix and the PR is WIP (may and may not be the final solution).
Currrently trying to understand the following:

  1. What is the performance impact of this fix if using a larger image eg mcr.microsoft.com/windows/servercore:ltsc2022 vs mcr.microsoft.com/windows/nanoserver:ltsc2022
  2. Is it the Windows terminal?
  3. Why are the build output and tarball all going to the same location on Windows alone?
  4. How does it work for progress=plain?

@apostasie
Copy link
Contributor

Deeply confused about this.

Why are the build output and tarball all going to the same location on Windows alone?

This does not make any sense.
I am squinting at nerdctl source and cannot figure it out.

Now suspecting buildctl might be the problem.

@apostasie
Copy link
Contributor

buildctl "output" goes to stderr.

Is it the case on windows?

foo="$(buildctl --addr=unix:///run/user/501/buildkit/buildkitd.sock build --progress=auto --frontend=dockerfile.v0 --local=context=/tmp/ --output=type=docker --local=dockerfile=/tmp --opt=filename=Dockerfile)"

or

buildctl --addr=unix:///run/user/501/buildkit/buildkitd.sock build --progress=auto --frontend=dockerfile.v0 --local=context=/tmp/ --output=type=docker --local=dockerfile=/tmp --opt=filename=Dockerfile >/dev/null

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ci e.g., CI failure
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants