Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): macos-latest + windows-latest #17

Merged
merged 16 commits into from
Mar 19, 2024
Merged

chore(ci): macos-latest + windows-latest #17

merged 16 commits into from
Mar 19, 2024

Conversation

Tieske
Copy link
Member

@Tieske Tieske commented Jan 30, 2024

testing ci additions

hishamhm added a commit to luarocks/luarocks that referenced this pull request Feb 15, 2024
@hishamhm
Copy link
Member

@Tieske no idea what's causing this error, unfortunately, but I've added a mitigation to the crash that should hopefully provide more diagnostics if it happens again: luarocks/luarocks@8d0df02

It's probably about time I push a new LuaRocks release, too...

@hishamhm
Copy link
Member

The Windows dependency actually built this time, but the Windows bindir wasn't added to the PATH... 🤔

@Tieske
Copy link
Member Author

Tieske commented Mar 1, 2024

which component is responsible for adding the BinDir? I saw the luarocks action does set the path, but somehow doesn't work.

The error is a PowerShell one, so seems the commands are executed in PowerShell. Does the action update the correct paths?

@Tieske
Copy link
Member Author

Tieske commented Mar 1, 2024

@hishamhm
Copy link
Member

hishamhm commented Mar 4, 2024

this looks correct; https://github.com/hishamhm/gh-actions-luarocks/blob/master/main.js#L121

Yeah, this was supposed to be the one to do it. I have no idea why this didn't work

@hishamhm
Copy link
Member

hishamhm commented Mar 4, 2024

Can you work around it by explicitly setting the PATH in your CI script? It would be interesting to know if that at least works... (If that doesn't work, then it's a sign of something more deeply broken in the luarocks installation...)

@Tieske
Copy link
Member Author

Tieske commented Mar 5, 2024

So I added some debugging, but it (the busted executable) seems to be gone....

Here's the log: https://github.com/lunarmodules/luasystem/actions/runs/8161889298/job/22311635078?pr=17#step:6:13

Here's an edited version with commands and results:

luarocks config deploy_bin_dir

C:\Users\runneradmin\AppData\Roaming\luarocks\bin

$env:PATH

C:\Program Files\PowerShell\7;
C:\Users\runneradmin\AppData\Roaming\luarocks\bin;
D:\a\luasystem\luasystem\.luarocks\bin;
D:/a/luasystem/luasystem/.lua/bin;
C:\Program Files\MongoDB\Server\5.0\bin;
C:\aliyun-cli;
C:\vcpkg;
C:\Program Files (x86)\NSIS\;
C:\tools\zstd;
C:\Program Files\Mercurial\;
C:\hostedtoolcache\windows\stack\2.15.1\x64;
C:\cabal\bin;
C:\\ghcup\bin;
C:\mingw64\bin;
C:\Program Files\dotnet;
C:\Program Files\MySQL\MySQL Server 8.0\bin;
C:\Program Files\R\R-4.3.2\bin\x64;
C:\SeleniumWebDrivers\GeckoDriver;
C:\SeleniumWebDrivers\EdgeDriver\;
C:\SeleniumWebDrivers\ChromeDriver;
C:\Program Files (x86)\sbt\bin;
C:\Program Files (x86)\GitHub CLI;
C:\Program Files\Git\bin;
C:\Program Files (x86)\pipx_bin;
C:\npm\prefix;
C:\hostedtoolcache\windows\go\1.21.7\x64\bin;
C:\hostedtoolcache\windows\Python\3.9.13\x64\Scripts;
C:\hostedtoolcache\windows\Python\3.9.13\x64;
C:\hostedtoolcache\windows\Ruby\3.0.6\x64\bin;
C:\Program Files\OpenSSL\bin;
C:\tools\kotlinc\bin;
C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.402-6\x64\bin;
C:\Program Files\ImageMagick-7.1.1-Q16-HDRI;
C:\Program Files\Microsoft SDKs\Azure\CLI2\wbin;
C:\ProgramData\kind;
C:\ProgramData\Chocolatey\bin;
C:\Windows\system32;
C:\Windows;
C:\Windows\System32\Wbem;
C:\Windows\System32\WindowsPowerShell\v1.0\;
C:\Windows\System32\OpenSSH\;
C:\Program Files\dotnet\;
C:\Program Files\PowerShell\7\;
C:\Program Files\Microsoft\Web Platform Installer\;
C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;
C:\Program Files\Microsoft SQL Server\150\Tools\Binn\;
C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;
C:\Program Files\Microsoft SQL Server\130\DTS\Binn\;
C:\Program Files\Microsoft SQL Server\140\DTS\Binn\;
C:\Program Files\Microsoft SQL Server\150\DTS\Binn\;
C:\Program Files\Microsoft SQL Server\160\DTS\Binn\;
C:\Strawberry\c\bin;
C:\Strawberry\perl\site\bin;
C:\Strawberry\perl\bin;
C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;
C:\Program Files\TortoiseSVN\bin;
C:\Program Files\CMake\bin;
C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.7\bin;
C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;
C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;
C:\Program Files\nodejs\;
C:\Program Files\Git\cmd;
C:\Program Files\Git\mingw64\bin;
C:\Program Files\Git\usr\bin;
C:\Program Files\GitHub CLI\;
c:\tools\php;
C:\Program Files (x86)\sbt\bin;
C:\Program Files\Amazon\AWSCLIV2\;
C:\Program Files\Amazon\SessionManagerPlugin\bin\;
C:\Program Files\Amazon\AWSSAMCLI\bin\;
C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;
C:\Program Files\LLVM\bin;
C:\Users\runneradmin\.dotnet\tools;
C:\Users\runneradmin\.cargo\bin;
C:\Users\runneradmin\AppData\Local\Microsoft\WindowsApps

dir D:\a\luasystem\luasystem\.luarocks\bin

    Directory: D:\a\luasystem\luasystem\.luarocks\bin

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           2/27/2024  3:19 PM        4165805 luarocks-admin.exe
-a---           2/27/2024  3:19 PM        4165784 luarocks.exe

dir C:\Users\runneradmin\AppData\Roaming\luarocks\bin

Get-ChildItem: D:\a\_temp\8321d916-67e9-4167-864d-9ed40ff651c7.ps1:5
Line |
   5 |  dir C:\Users\runneradmin\AppData\Roaming\luarocks\bin
     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot find path 'C:\Users\runneradmin\AppData\Roaming\luarocks\bin' because it does not exist.
Error: Process completed with exit code 1.

@Tieske
Copy link
Member Author

Tieske commented Mar 5, 2024

hmmm... this entry D:/a/luasystem/luasystem/.lua/bin; is not properly normalized for Windows (slash instead of back-slash). So maybe that's the culprit.

@Tieske
Copy link
Member Author

Tieske commented Mar 5, 2024

Also not the case;

dir D:\a\luasystem\luasystem\.lua\bin

     Directory: D:\a\luasystem\luasystem\.lua\bin
Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
la---           2/29/2024  9:24 AM              0 lua.exe -> luajit-2.1.0-beta3.exe
-a---           2/29/2024  9:24 AM         480666 lua51.dll
-a---           2/29/2024  9:24 AM          29875 luajit-2.1.0-beta3.exe

@Tieske
Copy link
Member Author

Tieske commented Mar 5, 2024

so there are 3 path entries for Lua, 1 doesn't exist in the system, the other 2 do not have the busted executable.

Got any follow ideas to look into @hishamhm ??

@Tieske
Copy link
Member Author

Tieske commented Mar 5, 2024

so here's more:

luarocks show busted

busted 2.2.0-1 - Elegant Lua unit testing
An elegant, extensible, testing framework. Ships with a large amount of useful
asserts, plus the ability to write your own. Output in pretty or plain terminal
format, JSON, or TAP for CI integration. Great for TDD and unit, integration,
and functional tests.
License:      	MIT <http://opensource.org/licenses/MIT>
Homepage:     	https://lunarmodules.github.io/busted/
Installed in: 	C:\Users\runneradmin\AppData\Roaming\luarocks
Depends on:
	lua >= 5.1 (using 5.1-1)
	lua_cliargs 3.0 (using 3.0-2)
	luasystem >= 0.2.0 (using scm-0)
	dkjson >= 2.1.0 (using 2.7-1)
	say >= 1.4-1 (using 1.4.1-3)
	luassert >= 1.9.0-1 (using 1.9.0-1)
	lua-term >= 0.1 (using 0.8-1)
	penlight >= 1.3.2 (using 1.13.1-1)
	mediator_lua >= 1.1.1 (using 1.1.2-0)
Indirectly pulling:
	dkjson (using 2.7-1)
	lua-term (using 0.8-1)
	lua_cliargs (using 3.0-2)
	luafilesystem (using 1.8.0-1)
	luassert (using 1.9.0-1)
	luasystem (using scm-0)
	mediator_lua (using 1.1.2-0)
	penlight (using 1.13.1-1)
	say (using 1.4.1-3)

This is weird. It says it is installed, but doesn't list a single installed file. And hence also not the executable.

Here's what I get locally:

busted 2.2.0-1 - Elegant Lua unit testing

An elegant, extensible, testing framework. Ships with a large amount of useful
asserts, plus the ability to write your own. Output in pretty or plain terminal
format, JSON, or TAP for CI integration. Great for TDD and unit, integration,
and functional tests.

License:        MIT <http://opensource.org/licenses/MIT>
Homepage:       https://lunarmodules.github.io/busted/
Installed in:   /usr/local/Cellar/[email protected]/1.21.4.1/luarocks

Commands:
        /usr/local/Cellar/[email protected]/1.21.4.1/luarocks/bin/busted (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/bin/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/bin/busted)

Modules:
        busted.block (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/block.lua)
        busted.compatibility (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/compatibility.lua)
        busted.context (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/context.lua)
        busted.core (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/core.lua)
        busted.done (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/done.lua)
        busted.environment (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/environment.lua)
        busted.execute (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/execute.lua)
        busted.fixtures (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/fixtures.lua)
        busted.init (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/init.lua)
        busted.languages.ar (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/ar.lua)
        busted.languages.de (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/de.lua)
        busted.languages.en (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/en.lua)
        busted.languages.es (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/es.lua)
        busted.languages.fr (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/fr.lua)
        busted.languages.is (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/is.lua)
        busted.languages.it (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/it.lua)
        busted.languages.ja (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/ja.lua)
        busted.languages.ko (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/ko.lua)
        busted.languages.nl (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/nl.lua)
        busted.languages.pt-BR (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/pt-BR.lua)
        busted.languages.ro (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/ro.lua)
        busted.languages.ru (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/ru.lua)
        busted.languages.th (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/th.lua)
        busted.languages.ua (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/ua.lua)
        busted.languages.zh (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/languages/zh.lua)
        busted.luajit (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/luajit.lua)
        busted.modules.cli (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/cli.lua)
        busted.modules.configuration_loader (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/configuration_loader.lua)
        busted.modules.files.lua (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/files/lua.lua)
        busted.modules.files.moonscript (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/files/moonscript.lua)
        busted.modules.files.terra (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/files/terra.lua)
        busted.modules.filter_loader (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/filter_loader.lua)
        busted.modules.helper_loader (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/helper_loader.lua)
        busted.modules.luacov (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/luacov.lua)
        busted.modules.output_handler_loader (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/output_handler_loader.lua)
        busted.modules.standalone_loader (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/standalone_loader.lua)
        busted.modules.test_file_loader (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/modules/test_file_loader.lua)
        busted.options (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/options.lua)
        busted.outputHandlers.TAP (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/outputHandlers/TAP.lua)
        busted.outputHandlers.base (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/outputHandlers/base.lua)
        busted.outputHandlers.gtest (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/outputHandlers/gtest.lua)
        busted.outputHandlers.json (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/outputHandlers/json.lua)
        busted.outputHandlers.junit (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/outputHandlers/junit.lua)
        busted.outputHandlers.plainTerminal (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/outputHandlers/plainTerminal.lua)
        busted.outputHandlers.sound (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/outputHandlers/sound.lua)
        busted.outputHandlers.utfTerminal (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/outputHandlers/utfTerminal.lua)
        busted.runner (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/runner.lua)
        busted.status (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/status.lua)
        busted.utils (/usr/local/Cellar/[email protected]/1.21.4.1/luarocks/share/lua/5.1/busted/utils.lua)

Depends on:
        lua >= 5.1 (using 5.1-1)
        lua_cliargs 3.0 (using 3.0-2)
        luasystem >= 0.2.0 (using 0.3.0-1)
        dkjson >= 2.1.0 (using 2.6-1)
        say >= 1.4-1 (using 1.4.1-3)
        luassert >= 1.9.0-1 (using 1.9.0-1)
        lua-term >= 0.1 (using 0.8-1)
        penlight >= 1.3.2 (using 1.13.1-1)
        mediator_lua >= 1.1.1 (using 1.1.2-0)

Indirectly pulling:
        dkjson (using 2.6-1)
        lua-term (using 0.8-1)
        lua_cliargs (using 3.0-2)
        luafilesystem (using 1.8.0-1)
        luassert (using 1.9.0-1)
        luasystem (using 0.3.0-1)
        mediator_lua (using 1.1.2-0)
        penlight (using 1.13.1-1)
        say (using 1.4.1-3)

@Tieske
Copy link
Member Author

Tieske commented Mar 8, 2024

managed to add Dependencies utility to check the exports, but they seem fine:

{
  "Exports": [
    {
      "Ordinal": 1,
      "Name": "luaopen_lfs",
      "ExportByOrdinal": false,
      "VirtualAddress": 8880,
      "ForwardedName": ""
    }
  ]
}

I thought maybe the weird error was because it couldn't find the function to open the lib.

@hishamhm
Copy link
Member

hishamhm commented Mar 8, 2024

what's this weird error message?
D:/a/luasystem/luasystem/.lua/bin\lua.exe: error loading module 'lfs' from file 'C:\Users\runneradmin\AppData\Roaming\luarocks\lib\lua\5.4\lfs.dll': The specified module could not be found.

Yeah, that's the mystery...

Thanks for making the local build for comparing with the logs... the cl.exe and link.exe command line look effectively identical, as far as I can tell.

@hishamhm
Copy link
Member

hishamhm commented Mar 8, 2024

Oh, looks like this may be the cause: https://github.com/leafo/gh-actions-lua/pull/38/files

including various pending PRs from leafo/gh-actions-lua
@hishamhm
Copy link
Member

hishamhm commented Mar 8, 2024

@Tieske ...and we got a green build at last!!

I pulled these four PRs into my fork of gh-actions-lua:

(I didn't pull leafo/gh-actions-lua#44 because that seems like a deeper change, which may also need fixing conflicts with the other ones, and I'm not sure if that would work out of the box anyway — I didn't test it, just wanted the issue we have at hand here to get over with.)

I had to delete the cache for the Lua 5.4 windows build manually. And it looks like the cache key in gh-actions-lua doesn't check for gcc vs msvc. It probably needs a super-specific cache key with all of the msvc variants stuff... LuaRocks is also bad at this regard, we just call everything "win32" and only differ on CPU arch... so heads up on that.

Another thing to note is that I think the combo luajit+msvc isn't supported by these actions yet, because we're using the Makefile to build luajit. (And so does leafo/gh-actions-lua#44, so that PR wouldn't help on this front either).

But I think we're in a pretty good shape to get this PR cleaned up and your luasystem CI set up. And I think I'm looking good for releasing LuaRocks 3.11 as well!

@Tieske
Copy link
Member Author

Tieske commented Mar 8, 2024

ha, finally! thx for the hard work @hishamhm 🙏

So where to go from here; clean up and merge using the forks, or wait for @leafo to merge them upstream? (or maybe even move the upstream projects into LunarModules and set up a larger support team if @leafo is ok with that?)

@hishamhm
Copy link
Member

hishamhm commented Mar 9, 2024

I'd say just clean up and merge using the forks as-is for the time being. If anything else happens with the actions, you can always get back here and have them point elsewhere.

@hishamhm
Copy link
Member

@Tieske do you have anything else you need to change here? I think even keeping the Dependencies Analyzer stuff around should be fine (I've gated it for the msvc run only). So I'd say how about we take this PR, squash it into a single commit and merge? You can always add more CI jobs later if you feel like it.

@Tieske
Copy link
Member Author

Tieske commented Mar 11, 2024

over the weekend I tried adding also a MinGW job, but that failed. Ideally we'd have jobs for msvc and MinGW. Haven't looked into the cache key issue you mentioned before yet, maybe that's the culprit.

Also; wondering if we should keep this as 1 workflow, or have separate ones for posix-like and Windows.

@hishamhm
Copy link
Member

The existing one for "windows-latest, luajit" is mingw already, isn't it? it's not MSVC, and uses gcc on Windows.

@hishamhm
Copy link
Member

using different Lua versions for the mingw and msvc builds avoids the cache issue for now. I can see the appeal of testing every NxM combination but the current list already provides quite decent coverage.

to keep the caches not mixed (cache-key doesn't include toolchain)
@Tieske
Copy link
Member Author

Tieske commented Mar 15, 2024

ok, tried that, but now it keeps looking for the msvc toolchain; https://github.com/lunarmodules/luasystem/actions/runs/8297960620/job/22710219967?pr=17

Added a mingw build for a different Lua version. In the logs you can see that the 2 msvc specific steps are skipped (as they should). But it still looks for the cl command;

Error: Failed to install Lua: Error: Unable to locate executable file: cl. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also verify the file has a valid extension for an executable file.

@Tieske Tieske marked this pull request as ready for review March 19, 2024 09:57
@Tieske
Copy link
Member Author

Tieske commented Mar 19, 2024

ok, so turns out msvc is required to build Puc Rio Lua versions by the action, So no MinGW for those versions.

So the LuaJIT builds will test the MinGW toolchain, since they are build using that one. And the PuC Rio ones will use MSVC. I think that should suffice.

@Tieske Tieske merged commit 7ef8c46 into master Mar 19, 2024
23 checks passed
@Tieske Tieske deleted the ci-win branch March 19, 2024 10:05
Tieske added a commit that referenced this pull request Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants