docs: Add initial doc for system tests #22002
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
📝 WalkthroughWalkthroughThe changes include the introduction of a new document titled Changes
Possibly related PRs
Suggested labels
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
alpe
changed the title
|
Unrelated to the content, but you could make your editor follow the formatting rules: https://github.com/cosmos/cosmos-sdk/blob/main/.markdownlint.json |
* main: docs: amend docs for 52 changes (#21992) test: migrate e2e/authz to system tests (#21819) refactor(runtime/v2): use StoreBuilder (#21989) feat(schema): add API descriptors, struct, oneof & list types, and wire encoding spec (#21482) docs: add instructions to change DefaultGenesis (#21680) feat(x/staking)!: Add metadata field to validator info (#21315) chore(x/authz)!: Remove account keeper dependency (#21632)
julienrbrt
reviewed
Oct 1, 2024
tac0turtle
approved these changes
Oct 1, 2024
julienrbrt
approved these changes
Oct 2, 2024
julienrbrt
added
the
backport/v0.52.x
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (7)
docs/build/building-apps/06-system-tests.md (4)
11-25: LGTM with a minor suggestion: Well-explained implementation detailsThe section effectively explains the implementation of system tests using Go, introducing key components and providing useful examples and resources.
Consider removing the line break between "genesis or" and "transactions" on lines 24-25 to improve readability:
contains a step-by-step guide to building and running your first system test. It covers setting chain state via genesis -or -transactions and validation via transaction response or queries. +or transactions and validation via transaction response or queries.
27-42: LGTM with a formatting suggestion: Comprehensive design principles and guidelinesThe section provides valuable guidance for writing effective system tests. The principles are well-explained and cover important aspects of test design.
To adhere to the project's markdown linting configuration, please update the unordered list style from dashes to asterisks:
- - **Perspective:** Tests should mimic a human interacting with the chain from the outside. Initial states can be set via + * **Perspective:** Tests should mimic a human interacting with the chain from the outside. Initial states can be set via genesis or transactions to support a test scenario. - - **Roles:** The user can have multiple roles such as validator, delegator, granter, or group admin. + * **Roles:** The user can have multiple roles such as validator, delegator, granter, or group admin. - - **Focus:** Tests should concentrate on happy paths or business-critical workflows. Unit and integration tests are + * **Focus:** Tests should concentrate on happy paths or business-critical workflows. Unit and integration tests are better suited for more fine-grained testing. - - **Workflows:** Test workflows and scenarios, not individual units. Given the high setup costs, it is reasonable to + * **Workflows:** Test workflows and scenarios, not individual units. Given the high setup costs, it is reasonable to combine multiple steps and assertions in a single test method. - - **Genesis Mods:** Genesis modifications can incur additional time costs for resetting dirty states. Reuse existing + * **Genesis Mods:** Genesis modifications can incur additional time costs for resetting dirty states. Reuse existing accounts (node0..n) whenever possible. - - **Framework:** Continuously improve the framework for better readability and reusability. + * **Framework:** Continuously improve the framework for better readability and reusability.🧰 Tools
🪛 LanguageTool
[typographical] ~40-~40: Two consecutive dots
Context: ...states. Reuse existing accounts (node0..n) whenever possible. - Framework: C...(DOUBLE_PUNCTUATION)
🪛 Markdownlint
32-32: Expected: asterisk; Actual: dash
Unordered list style(MD004, ul-style)
34-34: Expected: asterisk; Actual: dash
Unordered list style(MD004, ul-style)
35-35: Expected: asterisk; Actual: dash
Unordered list style(MD004, ul-style)
37-37: Expected: asterisk; Actual: dash
Unordered list style(MD004, ul-style)
39-39: Expected: asterisk; Actual: dash
Unordered list style(MD004, ul-style)
41-41: Expected: asterisk; Actual: dash
Unordered list style(MD004, ul-style)
43-48: LGTM with a minor suggestion: Helpful debugging informationThis section provides valuable information on logging and debugging system tests.
To improve readability, consider using curly braces for the range in "node0..n":
- accounts (node0..n) whenever possible. + accounts (node{0..n}) whenever possible.This change makes the notation consistent with the
node{0..n}.outformat used earlier in the document.🧰 Tools
🪛 LanguageTool
[typographical] ~45-~45: Two consecutive dots
Context: ... and Debugging All output is logged tosystemtests/testnet/node{0..n}.out. Usually,node0.outis very no...(DOUBLE_PUNCTUATION)
49-58: LGTM with a formatting suggestion: Clear debugging instructionsThe step-by-step debugging instructions are clear and helpful for developers.
To adhere to the project's markdown linting configuration, please update the unordered list style from dashes to asterisks:
- - Start the test with one node only and verbose output: + * Start the test with one node only and verbose output: ```sh go test -v -tags=system_test ./ --run TestAccountCreation --verbose --nodes-count=1
- Copy the CLI command for the transaction and modify the test to stop before the command
- Start the node with
--home=<project-home>/tests/systemtests/testnet/node0/<binary-name>/in debug mode
- Execute CLI command from shell and enter breakpoints
- Copy the CLI command for the transaction and modify the test to stop before the command
- Start the node with
--home=<project-home>/tests/systemtests/testnet/node0/<binary-name>/in debug mode
- Execute CLI command from shell and enter breakpoints
<details> <summary>🧰 Tools</summary> <details> <summary>🪛 Markdownlint</summary><blockquote> 50-50: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 56-56: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 57-57: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 58-58: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) </blockquote></details> </details> </blockquote></details> <details> <summary>docs/build/building-modules/16-testing.md (3)</summary><blockquote> `89-100`: **LGTM! Consider adding a brief explanation of message factories.** The updates to the Simulations section provide a clearer and more accurate description of the current state of simulations in the SDK. The change from `x/gov/` to `x/bank/` for the example is also appropriate. Consider adding a brief explanation of what message factories are and how they relate to simulations. This would help readers better understand the concept, especially those who might be new to the SDK. --- `101-107`: **LGTM! Consider clarifying the use of `simapp` vs. custom binaries.** The updates to the System Tests section (formerly End-to-end Tests) provide a more accurate and comprehensive description of these tests. The change in terminology and test location is consistent with the current SDK structure. The sentence about using the `simapp` binary might be confusing. Consider clarifying that while the SDK uses `simapp` for its own testing, users should use their own application binary when testing their specific blockchain application. This would help prevent any misunderstanding about which binary to use in different contexts. --- Line range hint `1-116`: **LGTM! Consider adding a brief introduction to the changes.** The document maintains a consistent structure and now provides a more comprehensive overview of testing methodologies in the Cosmos SDK. The updates to the Simulations and System Tests sections integrate well with the existing content. Consider adding a brief introduction at the beginning of the document highlighting the recent changes to the testing methodologies. This would help readers quickly identify what's new or updated in the document. </blockquote></details> </blockquote></details> <details> <summary>📜 Review details</summary> **Configuration used: .coderabbit.yml** **Review profile: CHILL** <details> <summary>📥 Commits</summary> Files that changed from the base of the PR and between 52d8b2eb43df267d80a6bbbd2b7067879466377d and 1026a68c2b18e59689dfbccd5cc9fc9b390d8c9d. </details> <details> <summary>📒 Files selected for processing (2)</summary> * docs/build/building-apps/06-system-tests.md (1 hunks) * docs/build/building-modules/16-testing.md (1 hunks) </details> <details> <summary>🧰 Additional context used</summary> <details> <summary>📓 Path-based instructions (2)</summary><blockquote> <details> <summary>docs/build/building-apps/06-system-tests.md (1)</summary> Pattern `**/*.md`: "Assess the documentation for misspellings, grammatical errors, missing documentation and correctness" </details> <details> <summary>docs/build/building-modules/16-testing.md (1)</summary> Pattern `**/*.md`: "Assess the documentation for misspellings, grammatical errors, missing documentation and correctness" </details> </blockquote></details> <details> <summary>🪛 LanguageTool</summary><blockquote> <details> <summary>docs/build/building-apps/06-system-tests.md</summary><blockquote> [typographical] ~40-~40: Two consecutive dots Context: ...states. Reuse existing accounts (node0..n) whenever possible. - **Framework:** C... (DOUBLE_PUNCTUATION) --- [typographical] ~45-~45: Two consecutive dots Context: ... and Debugging All output is logged to `systemtests/testnet/node{0..n}.out`. Usually, `node0.out` is very no... (DOUBLE_PUNCTUATION) </blockquote></details> </blockquote></details> <details> <summary>🪛 Markdownlint</summary><blockquote> <details> <summary>docs/build/building-apps/06-system-tests.md</summary><blockquote> 32-32: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 34-34: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 35-35: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 37-37: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 39-39: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 41-41: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 50-50: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 56-56: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 57-57: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 58-58: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) </blockquote></details> </blockquote></details> </details> <details> <summary>🔇 Additional comments (3)</summary><blockquote> <details> <summary>docs/build/building-apps/06-system-tests.md (3)</summary><blockquote> `1-6`: **LGTM: Frontmatter and title are well-formatted** The frontmatter is correctly structured, and the title "System Tests" is clear and concise. --- `7-10`: **LGTM: Clear and informative introduction** The introduction effectively explains the purpose and benefits of system tests, providing a good context for readers. --- `1-58`: **Excellent introduction to system tests** This document provides a comprehensive and well-structured introduction to system tests for blockchain applications. It covers key aspects such as the purpose of system tests, implementation details, design principles, and debugging techniques. The content is informative and valuable for developers working on blockchain applications. With the minor formatting suggestions implemented, this document will serve as an excellent resource for the project. <details> <summary>🧰 Tools</summary> <details> <summary>🪛 LanguageTool</summary><blockquote> [typographical] ~40-~40: Two consecutive dots Context: ...states. Reuse existing accounts (node0..n) whenever possible. - **Framework:** C... (DOUBLE_PUNCTUATION) --- [typographical] ~45-~45: Two consecutive dots Context: ... and Debugging All output is logged to `systemtests/testnet/node{0..n}.out`. Usually, `node0.out` is very no... (DOUBLE_PUNCTUATION) </blockquote></details> <details> <summary>🪛 Markdownlint</summary><blockquote> 32-32: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 34-34: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 35-35: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 37-37: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 39-39: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 41-41: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 50-50: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 56-56: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 57-57: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) --- 58-58: Expected: asterisk; Actual: dash Unordered list style (MD004, ul-style) </blockquote></details> </details> </blockquote></details> </blockquote></details> </details> <!-- This is an auto-generated comment by CodeRabbit for review status -->
|
I assume the dead link in https://github.com/cosmos/cosmos-sdk/actions/runs/11143215342/job/30967800126?pr=22002#step:5:472 is fixed when this is published |
mergify bot
pushed a commit
that referenced
this pull request
Oct 2, 2024
(cherry picked from commit 5c4f4ac)
12 tasks
julienrbrt
pushed a commit
that referenced
this pull request
Oct 2, 2024
Co-authored-by: Alexander Peters <[email protected]>
This was referenced Oct 2, 2024
Merged
Merged
This was referenced Oct 3, 2024
This was referenced Oct 10, 2024
12 tasks
This was referenced Nov 4, 2024
Closed
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
See: #21429
Start doc about system tests
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
!in the type prefix if API or client breaking changeCHANGELOG.mdReviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.
I have...
Summary by CodeRabbit
New Features
Documentation