Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor e2e testing to run provider actions only on provider #2315

Open
7 tasks
Tracked by #2283
MSalopek opened this issue Sep 30, 2024 · 0 comments
Open
7 tasks
Tracked by #2283

Refactor e2e testing to run provider actions only on provider #2315

MSalopek opened this issue Sep 30, 2024 · 0 comments
Assignees
Labels
scope: testing Code review, testing, making sure the code is following the specification.

Comments

@MSalopek
Copy link
Contributor

MSalopek commented Sep 30, 2024

At present one of the problems of the current e2e tests is the fact that we are always testing provider interactions while checking for side effects that happen on consumer. Since this requires test scenarios to always start a provider chain (4 node instances), consumer chain (4 node instances) and a relayer between them the setup is quite long and difficult to debug while not contributing much to developer confidence.

A better strategy (we think) would be to separate the provider concerns from consumer concerns by creating e2e test suites that would address a subset of the issues.

You can use any testing setup you like, but using interchaintest is recommended.

The following flows should be tested with provider only setup:

  • consumer lifecycle (happy path, edgecases, failure cases; test all messages and queries)
  • key assignment operations (happy path, edgecases, failure cases)
  • opt-IN/OUT operations (happy path, edgecases, failure cases)
  • correct handling of validators outside the provider chain max_validators for opt-In and top-N chains (requires multiple nodes)
  • consumer proposals (add/remove + upgrade opt-in to top-N via governance prop)
  • generating consumer chain genesis
  • provider genesis export (test that output matches expectation; might require fixes in the code)

The minimum number of nodes is 1 with a maximum of 3 nodes. Since there are no consensus concerns that we should immediately address, we can focus on using 1 provider node for most of the tests above to save compute power in CI.

Any tests that require slashing and reward distribution should not be part of this test suite.

The most important event that happens on the provided for the consumer chain is creating the consumer genesis file. You should make sure that the operations performed always yield the expected results as per the created consumer genesis.

Consumer chain interactions and relaying should be added later.

@github-project-automation github-project-automation bot moved this to 🩹 F1: Triage in Cosmos Hub Sep 30, 2024
@MSalopek MSalopek changed the title Create EPIC with test cases that require only the provider (e.g., consumer lifecycle) Refactor e2e testing to run provider actions only on provider Sep 30, 2024
@MSalopek MSalopek moved this from 🩹 F1: Triage to 📥 F2: Todo in Cosmos Hub Sep 30, 2024
@MSalopek MSalopek added scope: testing Code review, testing, making sure the code is following the specification. and removed needs-triage labels Sep 30, 2024
@mpoke mpoke moved this from 📥 F2: Todo to 🏗 F3: InProgress in Cosmos Hub Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope: testing Code review, testing, making sure the code is following the specification.
Projects
Status: 🏗 F3: InProgress
Development

No branches or pull requests

2 participants