runhcs returns unknown shim failure, exit code 2

[mtelvers]

I am trying to use runhcs to create a container. I have diligently followed @christian-korneck's blog post and the example in #435 but I am unable to get it to work. I have tried this on both Windows Server Core 2019 and 2022 using both Server Core and normal images.

To get started, I have installed the Go and the runhcs and wclayer tools.

Start-BitsTransfer https://go.dev/dl/go1.22.4.windows-amd64.msi
msiexec /qb /i go1.22.4.windows-amd64.msi
go install github.com/Microsoft/hcsshim/cmd/wclayer@latest
go install github.com/Microsoft/hcsshim/cmd/runhcs@latest

There is some discrepancyy around whether the scratch layer should just be an empty folder or whether it should created using wclayer but either way it doesn't work.

I am going to follow the example code in e2e_matrix_test.go.

Create some temporary working directories L175-L187

mkdir c:\temp
mkdir c:\temp\scratch
mkdir c:\temp\bundle

Create an empty spec L190-L199

$config = Invoke-RestMethod -uri https://raw.githubusercontent.com/Microsoft/hcsshim/d0b3bfc2ea9303a6a506da319f67fe827530b91e/test/functional/assets/defaultwindowsspec.json -Method Get
$config.process.args = @("cmd", "/c", "echo hello")
$config.windows.layerFolders = New-Object System.Collections.ArrayList

Build up a list of layers defined in the base image we are using L201-L213

$image = "mcr.microsoft.com/windows/servercore:ltsc2019"
docker image pull $image
$imageInspect = docker image inspect $image | ConvertFrom-JSON
$layer = $imageInspect.GraphDriver.Data.dir
while ($layer) {
  $config.windows.layerFolders.Add($layer)
  $layer = Get-Content "$layer\layerchain.json" | ConvertFrom-Json
}

Add the empty scratch layer L214. I've tried using wclayer to create an empty scratch layer stacking the layers from above as the parents but it doesn't make a difference.

$config.windows.layerFolders.Add("c:\temp\scratch")

Create config.json L216-L227

$config | ConvertTo-Json | Set-Content c:\temp\bundle\config.json

Then invoke runhcs aka L250. I've tried it with and without the PID file and logs.

runhcs create --pid-file c:\temp\pid --shim-log c:\temp\log.txt --vm-log c:\temp\vm.txt --bundle c:\temp\bundle mycontainer

runhcs list shows that this has worked and the pid file is populated.

ID            PID         STATUS      BUNDLE           CREATED                        OWNER
mycontainer   700         created     c:\temp\bundle   2024-06-06T07:48:32.4423724Z

Let's start the container L272

PS> runhcs start mycontainer
unknown shim failure, exit code 2

The process is now dead

PS> runhcs list
ID            PID         STATUS      BUNDLE           CREATED                        OWNER
mycontainer   0           stopped     c:\temp\bundle   2024-06-06T07:48:32.4423724Z

The log file shows an invalid memory address error

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x1168c53]

goroutine 1 [running]:
github.com/sirupsen/logrus.(*Entry).Log(0xc000390000?, 0x15b3df8?, {0xc0002def70?, 0x140bc20?, 0xc00007c000?})
        C:/Users/Administrator/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:303 +0x13
github.com/sirupsen/logrus.(*Entry).Info(...)
        C:/Users/Administrator/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:321
github.com/Microsoft/hcsshim/internal/cmd.(*Cmd).Start(0xc000078000)
        C:/Users/Administrator/go/pkg/mod/github.com/!microsoft/[email protected]/internal/cmd/cmd.go:215 +0x55e
main.init.func12(0xc00008a840)
        C:/Users/Administrator/go/pkg/mod/github.com/!microsoft/[email protected]/cmd/runhcs/shim.go:170 +0xa87
github.com/urfave/cli.HandleAction({0x1426ae0?, 0x1531688?}, 0x4?)
        C:/Users/Administrator/go/pkg/mod/github.com/urfave/[email protected]/app.go:524 +0x50
github.com/urfave/cli.Command.Run({{0x14dd8f7, 0x4}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x1504e6b, 0x45}, {0x0, ...}, ...}, ...)
        C:/Users/Administrator/go/pkg/mod/github.com/urfave/[email protected]/command.go:175 +0x685
github.com/urfave/cli.(*App).Run(0xc000085500, {0xc0000c4000, 0xb, 0x10})
        C:/Users/Administrator/go/pkg/mod/github.com/urfave/[email protected]/app.go:277 +0xb3b
main.main()
        C:/Users/Administrator/go/pkg/mod/github.com/!microsoft/[email protected]/cmd/runhcs/main.go:171 +0xf5e

I have put all of the above PowerShell code in a Gist, including the steps to tidy up (such as using wclayer to unmount and remove the image).

I assume that I am just doing something wrong or that runhcs has been replaced with something else. However, I noticed that @katiewasnothere updated the code just a couple of weeks ago so I think it probably should be a working tool. Any thoughts would be greatly appreciated.

This is what config.json looked like

{
  "ociVersion":  "1.0.1",
  "process":  {
          "args":  [
                 "cmd",
                 "/c",
                 "echo hello"
               ],
          "env":  [
              ],
          "cwd":  "c:\\"
        },
  "windows":  {
          "layerFolders":  [
                     "C:\\ProgramData\\docker\\windowsfilter\\c10daac0e9206e70c3c0836d7ee76486878153c386012ea89987c7276518f03f",
                     "C:\\ProgramData\\docker\\windowsfilter\\7b798f80f190350c03eebac95f039abf22225dec2d28954b2b92fd7488492000",
                     "c:\\temp\\scratch"
                   ]
        }
}