Setup-lisp doesn't notice various failures, and reports success without running proper tests.

[3b]

With setup-lisp@v2, if the installation of a lisp fails (seems to fail on at least allegro,clisp,ecl on linux currently), it silently ignores it and runs tests on sbcl instead (or whatever lisp was installed in cached setup if it matches some other run), and the CI run passes.

With setup-lisp@v3, ecl,clisp still fails to install and runs tests on another lisp.
Allegro and ccl-bin/1.12.2 install but then 'passes' the tests with errors like

> Error: Invalid pathname component :HOME
> While executing: CCL::%PATH-STD-QUOTES, in process listener(1).

or

Warning: An error occurred
         (attempt to take the length of a non-sequence: :HOME) during
         the reading or evaluation of -e "(ros:run '((:eval\"(ros:asdf)\")(:eval\"(ros:quicklisp)\")(:load\"/home/runner/.roswell/init.lisp\")(:script \"/home/runner/work/_actions/40ants/run-tests/v2/run-tests.ros\"\"3b-bmfont\")(:quit ())))"
International Allegro CL Free Express Edition
10.1 [32-bit Linux (x86)] (Jan 23, 2023 8:47)
...
CL-USER(1): CL-USER(1): EOF
Really exit lisp [n]? 
; Initial Lisp Listener will be killing processes: 
;; processes killed
; Exiting

[3b]

the :home errors are possibly contagious through caches or something, switched back to v2 and still got same thing on ccl until I deleted all the CI caches.

[svetlyak40wt]

Oh, seems there are a few issues. I'm trying to resolve them now.

[svetlyak40wt]

These issues are fixed in #14

I'll tag a new revision with v3 tag after a check on few my systems. For a while you can use v3.1.0 tag.