refactor: extend the integration testing framework to support multiple languages #2362
Conversation
alecthomas
added
the
run-all
alecthomas
force-pushed
the
aat/multi-language-integration-tests
branch
from
f9f8385
to
7153aa7
Compare
| ic.AssertWithRetry(t, func(t testing.TB, ic TestContext) { | ||
| _, err := ic.Controller.Status(ic, connect.NewRequest(&ftlv1.StatusRequest{})) | ||
| assert.NoError(t, err) | ||
| for _, language := range opts.languages { |
There was a problem hiding this comment.
So my test deploys both of them and has them talk to each other. The go module is used as a kind of source of truth, and the Java module is basically proxying the go module to make sure that both the client invocations and responses are handled correctly.
This change looks like it would not really support testing multiple languages in the same cluster?
There was a problem hiding this comment.
It will actually support multiple languages in the same cluster, the language name just points to a directory - the directory can contain modules from multiple languages if necessary.
Though that said, there are commonly two types of integration tests we've encountered:
- Testing conformance with FTL subsystems.
- Testing language interop.
We've found the latter much less common because conformance tests largely encompass language interop too, because if a language complies with the conformance tests it works with other languages by definition.
There was a problem hiding this comment.
For example, take TestHttpIngress. This tests that a module correctly conforms to the HTTP ingress "spec". Just dropping a Java module into the testdata/java directory will test this on Java for "free".
There was a problem hiding this comment.
Similar with the PubSub tests, and so on. There are some tests somewhere that deal with encoding types too I think.
alecthomas
force-pushed
the
aat/multi-language-integration-tests
branch
from
7153aa7
to
0bebc0f
Compare
…e languages Previously we had full test coverage in each language, so we would just run them all, but as we're building up support for other languages for now, this change allows multiple languages to be supported per-test.
alecthomas
force-pushed
the
aat/multi-language-integration-tests
branch
from
0bebc0f
to
f54cb41
Compare
| in.WithFTLConfig("ftl-project-dr.toml"), | ||
| in.WithoutController(), |
There was a problem hiding this comment.
oh this is nice! Seems much more intuitive.
Previously we had full test coverage in each language, so we would just run them all, but as we're building up support for other languages for now, this change allows multiple languages to be supported per-test.
Also cleans up the test harness so there's just a single Run() function whose behaviour can be configured.