Los Angeles Meetup Thoughts: Moving from clojure.test to lazytest

[NoahTheDuke]

From the slack channel:

The Los Angeles meetup did a walkthrough of LazyTest tonight -- all very interesting stuff -- and the question came up:
If you have an existing codebase with a lot of existing tests, written with clojure.test, what is the story for a migration to LazyTest?
You can't do it strictly incrementally, because whatever clojure.test-compatible test runner you're using won't see the lazy tests, and the LazyTest runner won't see the existing tests.
Even a global find'n'replace on clojure.test to lazytest.experimental.interfaces.clojure-test wouldn't be sufficient (use-fixtures, for example).

[seancorfield]

My "solution" to this is to modify my test runners to look for both types of tests and run both clojure.test/run-tests and lazytest.repl/run-tests and merge the results -- once LazyTest stops using :test metadata (see #4).

My Polylith runner current looks for tests like this: https://github.com/seancorfield/polylith-external-test-runner/blob/main/bases/external-test-runner-cli/src/org/corfield/external_test_runner_cli/main.clj#L64-L70

Then it runs both and merges the result: https://github.com/seancorfield/polylith-external-test-runner/blob/main/bases/external-test-runner-cli/src/org/corfield/external_test_runner_cli/main.clj#L102-L106

And my Calva REPL snippets have been updated to run both on a per-namespace basis: https://github.com/seancorfield/vscode-calva-setup/blob/develop/calva/config.edn#L114-L143

[NoahTheDuke]

For results displaying/checking:

I did something similar when I moved Splint from clojure.test to Lazytest, as seen in this now-deleted file: https://github.com/NoahTheDuke/splint/blob/f851ef7933a74f54187d466a29482a1e4b4b577b/test/noahtheduke/splint/utils/test_runner.clj

A simple (merge-with (fnil + 0 0) ctr-result lt-result) works, but you can be more clever with it as you desire.

For actually transitioning the tests themselves, here's a couple different ideas:

1. Just move everything in one go. Here's an example from seesaw.
2. Move slowly, one namespace or test at a time, running both clojure.test and lazytest (either in a single call like my example and Sean's above, or in two separate clojure -M:... calls).
3. If you don't rely on use-fixtures or custom reporting functions, you can use the experimental clojure.test interface to switch a namespace by changing (:require [clojure.test :refer [deftest testing is are]]) to (:require [lazytest.experimental.interfaces.clojure-test :refer [deftest testing is are]]).

Fundamentally tho, the switch is pretty one-to-one: deftest becomes defdescribe, is becomes it + expect, and testing becomes describe. The only hitch is that expect must be wrapped in an it, so you have to either use expect-it to stay consistent, or bundle multiple assertions into a single test case.

This does highlight one small gap in Lazytest that I haven't found a good solution for. In clojure.test, you can write a simple deftest with a handful of assertions, but in Lazytest, you need to nest the assertions in an outer test case:

(deftest example-test
  (is (= 1 (one))
  (is (= 2 (two))
  (is (= 3 (three))))

; vs

(defdescribe example
  (it "works"
    (expect (= 1 (one))
    (expect (= 2 (two))
    (expect (= 3 (three)))))

It's a small point of friction, but I'm not pleased with it at the moment.

[seancorfield]

I would consider the necessary addition of something more descriptive as a "feature" not a "bug"! I like testing for that reason.

[NoahTheDuke]

I tend to agree. I think i've felt this pain mostly because I wrote a lot of test files for splint that were named for the splint rule, then the only test was named for the rule, and then the single test case doc string was... what? the rule name again?

for example: https://github.com/NoahTheDuke/splint/blob/main/test/noahtheduke/splint/rules/lint/let_if_test.clj

I think that's a particularly weird scenario tho, and most usage will be better for the requirement.

[seancorfield]

Maybe some of this could go into a "Migration" section in the README?

And at some point, the README could link to runners that run both, once we have releases of them...

[seancorfield]

As a follow-up note here, both my Polylith external test runner and my fork of Cognitect's test-runner have been updated to:

• use lazytest.find/find-var-test-value to determine whether a given Var is a LazyTest or not
• use lazytest.repl/* functions to run tests for one or more namespaces
• pass modified options into those functions to support include/exclude/var/output options

Both of those test runners can be used to deal with incremental migration from a pure clojure.test/Expectations codebase to a LazyTest codebase.