Windows 2022 - packer-windows-update-elevated.ps1 repeatedly fails

[bronandrews]

Hi,
I am using this plugin v0.15.0 to install updates during a fresh Windows 2022 build.
I'm including Important updates only, with the following syntax.

provisioner "windows-update" {
      search_criteria = "AutoSelectOnWebSites=1 and IsInstalled=0"
      filters = [
         "include:$true"
      ]
}

It finds new Updates and seems to carry out the install, but consistently fails when trying to exit Windows Modules Installer...
sample output log looks like...

2024/05/09 14:33:10 ui: hyperv-iso.appbase: Waiting for the Windows Modules Installer to exit...
2024/05/09 14:33:56 ui: hyperv-iso.appbase: Waiting for operation to complete (system performance: 4% cpu; 13% memory)...
memory)...
2024/05/09 14:41:05 ui: hyperv-iso.appbase: Waiting for operation to complete (system performance: 1% cpu; 13% memory)...
2024/05/09 14:41:55 packer-plugin-hyperv_v1.1.3_x5.0_windows_amd64.exe plugin: 2024/05/09 14:41:55 [ERROR] Remote command exited with '1': PowerShell -ExecutionPolicy Bypass -OutputFormat Text -File C:/Windows/Temp/packer-windows-update-elevated.ps1
2024/05/09 14:41:55 packer-plugin-hyperv_v1.1.3_x5.0_windows_amd64.exe plugin: 2024/05/09 14:41:55 [INFO] RPC endpoint: Communicator ended with: 1
2024/05/09 14:41:55 [INFO] 3047 bytes written for 'stdout'
2024/05/09 14:41:55 [INFO] 0 bytes written for 'stderr'
2024/05/09 14:41:55 [INFO] RPC client: Communicator ended with: 1
2024/05/09 14:41:55 [INFO] RPC endpoint: Communicator ended with: 1
2024/05/09 14:41:55 packer-plugin-windows-update_v0.15.0_x5.0_windows_amd64.exe plugin: 2024/05/09 14:41:55 [INFO] 0 bytes written for 'stderr'
2024/05/09 14:41:55 packer-plugin-windows-update_v0.15.0_x5.0_windows_amd64.exe plugin: 2024/05/09 14:41:55 [INFO] 3047 bytes written for 'stdout'
2024/05/09 14:41:55 packer-plugin-windows-update_v0.15.0_x5.0_windows_amd64.exe plugin: 2024/05/09 14:41:55 [INFO] RPC client: Communicator ended with: 1
2024/05/09 14:41:55 packer-plugin-windows-update_v0.15.0_x5.0_windows_amd64.exe plugin: 2024/05/09 14:41:55 Retryable error: Windows update script exited with non-zero exit status: 1

As its "retryable" - it repeats this 4 more times, until :
retry count exhausted. Last err: Windows update script exited with non-zero exit status
then the packer process exits and server build is removed.

I've tried various things with repeating calls to the provisioner, and the max_retries setting.
When I manually restart the server while this is happening, it seems to break out of the deadlock and the packer process continues.
But that is not realistic when building a stack of new servers.

Any suggestions about how to get over this issue?
Looking back over Open issues, feels like this error has been a problem for a while.....will it be fixed?

Many thanks!

[ghost]

Been having the same issue. Looking for solutions.

[rvishalturaga]

Hi, I am facing a similar issue as well but for a Windows Server 2019 Datacenter VM. We are using Azure Image Builder and running the WindowsUpdate customizer. The packer-windows-update-elevated.ps1 script runs initially and then gives a non zero error. I observed a non zero exit code (101) after which the VM restarts. After the VM is back the same script is run after which I get the following error

PACKER 2024/07/18 13:43:29 ui error: ==> azure-arm: The argument 'C:/Windows/Temp/packer-windows-update-elevated.ps1' to the -File parameter does not exist. Provide the path to an existing '.ps1' file as an argument to the -File parameter.
[83dbfb93-12a1-413f-a601-e186228f

[Raeykall]

Hi all, just to report I am seeing the same issue when attempting to use the Packer for Win11 Multi-Session

[5c307259-2810-456f-9b21-df1c667950ba] PACKER 2024/07/29 09:55:38 ui: azure-arm: ERROR: at , C:\Windows\Temp\packer-windows-update.ps1: line 190
[5c307259-2810-456f-9b21-df1c667950ba] PACKER 2024/07/29 09:55:38 ui: azure-arm: ERROR: at , : line 1
[5c307259-2810-456f-9b21-df1c667950ba] PACKER 2024/07/29 09:55:38 ui: azure-arm: ERROR EXCEPTION: System.Management.Automation.RuntimeException: You cannot call a method on a null-valued expression.

[rvishalturaga]

Hi @Raeykall, Can you please share in detail on what the error code is? Maybe paste a bigger section of the customizations log? Maybe we can work together to understand the issue and resolve it.

My expertise is based on Azure Image Builder only so I'm not sure to what level I can help but I want to contribute to how much ever I can. :)

[jonildoasantos]

Hi, I have a similar issue with Windows 11 Enterprise Multi-Session using Image Builder...

[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:02 ui: ==> azure-arm: Running Windows update...
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:02 packer-plugin-azure plugin: 2024/07/31 02:41:02 [INFO] starting remote command: PowerShell -ExecutionPolicy Bypass -OutputFormat Text -File C:/Windows/Temp/packer-windows-update-elevated.ps1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:09 ui: azure-arm: Searching for Windows updates...
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 packer-plugin-azure plugin: 2024/07/31 02:41:34 [INFO] command 'PowerShell -ExecutionPolicy Bypass -OutputFormat Text -File C:/Windows/Temp/packer-windows-update-elevated.ps1' exited with code: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 packer-plugin-azure plugin: 2024/07/31 02:41:34 [INFO] RPC endpoint: Communicator ended with: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 [INFO] 0 bytes written for 'stderr'
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 [INFO] 1100 bytes written for 'stdout'
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 [INFO] RPC client: Communicator ended with: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 [INFO] RPC endpoint: Communicator ended with: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR: You cannot call a method on a null-valued expression.
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 packer-plugin-windows-update plugin: 2024/07/31 02:41:34 [INFO] 1100 bytes written for 'stdout'
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 packer-plugin-windows-update plugin: 2024/07/31 02:41:34 [INFO] 0 bytes written for 'stderr'
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 packer-plugin-windows-update plugin: 2024/07/31 02:41:34 [INFO] RPC client: Communicator ended with: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR: at , C:\Windows\Temp\packer-windows-update.ps1: line 190
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR: at , : line 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR EXCEPTION: System.Management.Automation.RuntimeException: You cannot call a method on a null-valued expression.
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR EXCEPTION: at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR EXCEPTION: at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR EXCEPTION: at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR EXCEPTION: at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR EXCEPTION: at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 ui: azure-arm: ERROR EXCEPTION: at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:34 packer-plugin-windows-update plugin: 2024/07/31 02:41:34 Retryable error: Windows update script exited with non-zero exit status: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:39 packer-plugin-azure plugin: 2024/07/31 02:41:39 [INFO] starting remote command: PowerShell -ExecutionPolicy Bypass -OutputFormat Text -File C:/Windows/Temp/packer-windows-update-elevated.ps1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:45 ui: azure-arm: Searching for Windows updates...
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 [INFO] 0 bytes written for 'stderr'
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 packer-plugin-azure plugin: 2024/07/31 02:41:50 [INFO] command 'PowerShell -ExecutionPolicy Bypass -OutputFormat Text -File C:/Windows/Temp/packer-windows-update-elevated.ps1' exited with code: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 packer-plugin-azure plugin: 2024/07/31 02:41:50 [INFO] RPC endpoint: Communicator ended with: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 [INFO] 1100 bytes written for 'stdout'
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 [INFO] RPC client: Communicator ended with: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 ui: azure-arm: ERROR: You cannot call a method on a null-valued expression.
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 [INFO] RPC endpoint: Communicator ended with: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 packer-plugin-windows-update plugin: 2024/07/31 02:41:50 [INFO] 1100 bytes written for 'stdout'
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 packer-plugin-windows-update plugin: 2024/07/31 02:41:50 [INFO] 0 bytes written for 'stderr'
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 packer-plugin-windows-update plugin: 2024/07/31 02:41:50 [INFO] RPC client: Communicator ended with: 1
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 ui: azure-arm: ERROR: at , C:\Windows\Temp\packer-windows-update.ps1: line 190
[8a6c2380-c4f9-480c-a69a-9d0bd11f2f17] PACKER 2024/07/31 02:41:50 ui: azure-arm: ERROR: at , : line 1

[pvess]

Any update on this issue from someone?

[ricardofalc]

Any update on this issue from someone?

Have you tried the options given here:

1. Exception thrown while "Searching for Windows updates..." #144
2. https://learn.microsoft.com/en-us/windows/release-health/status-windows-11-23H2#3351msgdesc

[saily]

Hi @bronandrews, it took me a while to get this fixed, but it seems the provisioner does not work properly if you override the default shell for openssh which I did.

I've set the default shell to powershell using a registry entry as shown below. This caused failures without a clear reason.

# FIXME: This causes failure with windows-update provisioner, so disabled it!
New-ItemProperty -Path "HKLM:\SOFTWARE\OpenSSH" -Name DefaultShell -Value "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -PropertyType String -Force

Hope this helps and @rgl requesting your input if this could be valid reason.

[bronandrews]

Hi @saily
Many thanks for your update - I've just been testing this again...

As before, I'm using packer to provision a Hyper-V VM from a Windows2022 ISO image
WindowsServer2022English64-bitX22-74290.iso

This time I'm using the most recent windows-update plugin version (0.16.7)
and I can confirm I was already setting the default shell the powershell, as per your fix - this gets done via a script that gets called on first login to the machine...

As before, It does find the Windows updates I've specified (Important only) and looks to successfully install them... (takes about 30 mins)

2024/09/03 16:05:14 ui: hyperv-iso.appbase: Searching for Windows updates...
2024/09/03 16:05:19 ui: hyperv-iso.appbase: Found Windows update (2024-07-09; 71.86 MB): 2024-07 Cumulative Update for .NET Framework 3.5, 4.8 and 4.8.1 for Microsoft server operating system version 21H2 for x64 (KB5041016)
2024/09/03 16:05:19 ui: hyperv-iso.appbase: Found Windows update (2024-08-13; 71.27 MB): Windows Malicious Software Removal Tool x64 - v5.127 (KB890830)
2024/09/03 16:05:19 ui: hyperv-iso.appbase: Found Windows update (2024-09-03; 158.33 MB): Security Intelligence Update for Microsoft Defender Antivirus - KB2267602 (Version 1.417.468.0) - Current Channel (Broad)
2024/09/03 16:05:19 ui: hyperv-iso.appbase: Found Windows update (2024-08-13; 25404.94 MB): 2024-08 Cumulative Update for Microsoft server operating system version 21H2 for x64-based Systems (KB5041160)
2024/09/03 16:05:19 ui: hyperv-iso.appbase: Downloading Windows updates (4 updates; 25706.4 MB)...
2024/09/03 16:06:04 ui: hyperv-iso.appbase: Installing Windows updates...
...
...
...

But still getting the same error, still consistently fails when exiting the installer

2024/09/03 16:35:38 ui: hyperv-iso.appbase: Waiting for the Windows Modules Installer to exit...
2024/09/03 16:35:44 ui: hyperv-iso.appbase: Waiting for operation to complete (system performance: 16% cpu; 15% memory)...
2024/09/03 16:36:24 packer-plugin-hyperv_v1.1.3_x5.0_windows_amd64.exe plugin: 2024/09/03 16:36:24 [ERROR] Remote command exited with '1': PowerShell -ExecutionPolicy Bypass -OutputFormat Text -File C:/Windows/Temp/packer-windows-update-elevated.ps1
2024/09/03 16:36:24 packer-plugin-hyperv_v1.1.3_x5.0_windows_amd64.exe plugin: 2024/09/03 16:36:24 [INFO] RPC endpoint: Communicator ended with: 1
2024/09/03 16:36:24 [INFO] 0 bytes written for 'stderr'
2024/09/03 16:36:24 [INFO] 3174 bytes written for 'stdout'
2024/09/03 16:36:24 [INFO] RPC client: Communicator ended with: 1
2024/09/03 16:36:24 [INFO] RPC endpoint: Communicator ended with: 1
2024/09/03 16:36:24 packer-plugin-windows-update_v0.16.7_x5.0_windows_amd64.exe plugin: 2024/09/03 16:36:24 [INFO] 3174 bytes written for 'stdout'
2024/09/03 16:36:24 packer-plugin-windows-update_v0.16.7_x5.0_windows_amd64.exe plugin: 2024/09/03 16:36:24 [INFO] 0 bytes written for 'stderr'
2024/09/03 16:36:24 packer-plugin-windows-update_v0.16.7_x5.0_windows_amd64.exe plugin: 2024/09/03 16:36:24 [INFO] RPC client: Communicator ended with: 1
2024/09/03 16:36:24 packer-plugin-windows-update_v0.16.7_x5.0_windows_amd64.exe plugin: 2024/09/03 16:36:24 Retryable error: Windows update script exited with non-zero exit status: 1
2024/09/03 16:36:29 packer-plugin-hyperv_v1.1.3_x5.0_windows_amd64.exe plugin: 2024/09/03 16:36:29 [DEBUG] Opening new ssh session
2024/09/03 16:36:29 packer-plugin-hyperv_v1.1.3_x5.0_windows_amd64.exe plugin: 2024/09/03 16:36:29 [DEBUG] starting remote command: PowerShell -ExecutionPolicy Bypass -OutputFormat Text -File C:/Windows/Temp/packer-windows-update-elevated.ps1

and it retries 4 times until....
retry count exhausted. Last err: Windows update script exited with non-zero exit status

Any other ideas?

[bronandrews]

Hi again @saily
Apologies - on reflection, I think I misunderstood your fix,
you're suggesting NOT to change the default shell to powershell? As latest until the Windows updates are complete?

I will test changing the default later in the process & revert