You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At present one of the problems of the current e2e tests is the fact that we are always testing provider interactions while checking for side effects that happen on consumer. Since this requires test scenarios to always start a provider chain (4 node instances), consumer chain (4 node instances) and a relayer between them the setup is quite long and difficult to debug while not contributing much to developer confidence.
A better strategy (we think) would be to separate the provider concerns from consumer concerns by creating e2e test suites that would address a subset of the issues.
You can use any testing setup you like, but using interchaintest is recommended.
The following flows should be tested with provider only setup:
consumer lifecycle (happy path, edgecases, failure cases; test all messages and queries)
correct handling of validators outside the provider chain max_validators for opt-In and top-N chains (requires multiple nodes)
consumer proposals (add/remove + upgrade opt-in to top-N via governance prop)
generating consumer chain genesis
provider genesis export (test that output matches expectation; might require fixes in the code)
The minimum number of nodes is 1 with a maximum of 3 nodes. Since there are no consensus concerns that we should immediately address, we can focus on using 1 provider node for most of the tests above to save compute power in CI.
Any tests that require slashing and reward distribution should not be part of this test suite.
The most important event that happens on the provided for the consumer chain is creating the consumer genesis file. You should make sure that the operations performed always yield the expected results as per the created consumer genesis.
Consumer chain interactions and relaying should be added later.
The text was updated successfully, but these errors were encountered:
MSalopek
changed the title
Create EPIC with test cases that require only the provider (e.g., consumer lifecycle)
Refactor e2e testing to run provider actions only on provider
Sep 30, 2024
At present one of the problems of the current e2e tests is the fact that we are always testing provider interactions while checking for side effects that happen on consumer. Since this requires test scenarios to always start a provider chain (4 node instances), consumer chain (4 node instances) and a relayer between them the setup is quite long and difficult to debug while not contributing much to developer confidence.
A better strategy (we think) would be to separate the provider concerns from consumer concerns by creating e2e test suites that would address a subset of the issues.
You can use any testing setup you like, but using interchaintest is recommended.
The following flows should be tested with provider only setup:
max_validators
for opt-In and top-N chains (requires multiple nodes)The minimum number of nodes is 1 with a maximum of 3 nodes. Since there are no consensus concerns that we should immediately address, we can focus on using 1 provider node for most of the tests above to save compute power in CI.
Any tests that require slashing and reward distribution should not be part of this test suite.
The most important event that happens on the provided for the consumer chain is creating the consumer genesis file. You should make sure that the operations performed always yield the expected results as per the created consumer genesis.
Consumer chain interactions and relaying should be added later.
The text was updated successfully, but these errors were encountered: