Support loading the test suite from the file system

[mjambon]

For testing our application, we have essentially a test folder named foo that contains input files. Each input file gives its name to a test case. If foo/ doesn't exist, it's an error and we can't run the tests. However, we want to be able to run --help to inspect command-line options manually from any location in the file system, without foo/ being present.

In order to build the test suite, we must inspect folder foo, resulting in the test suite that we pass to Alcotest.run. It goes like this:

let main () =
  Alcotest.run "semgrep-core" (tests ())

let () = main ()

If we run the test program /path/to/test.exe --help, tests () will be evaluated. It will look for foo/ and will fail instead of showing the help page.

Our workaround is something close to this:

(*
   This allows running the test program with '--help' from any folder
   without getting an error due to not being able to load test data.
*)
let tests_with_delayed_error () =
  try tests ()
  with e ->
     ["Error", ["cannot load test data - not a real test", `Quick, (fun () -> raise e)]]

let main () =
  Alcotest.run "semgrep-core" (tests_with_delayed_error ())

let () = main ()

While this solution works, it still tries to load the test data when it doesn't have to. It's also not an obvious workaround. If Alcotest.run allowed delaying the evaluation of the test suite, it would be easier.

I propose the following alternate run function:

(** Same as [run] but delay the construction of the test suite until it's really needed. *)
val run_dynamic_suite : 
  ( ?argv:string array -> string -> (unit -> unit test list) -> return ) with_options

The change is the test list argument which is now (unit -> unit test list). A similar run_with_args alternative would have to be provided too.