Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to run and view output of a single test case #20

Merged
merged 2 commits into from
Apr 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,26 @@ solana-test-suite run-tests --input-dir <input_dir> --solana-target <solana_targ
**Note:** Each `.so` target file name should be unique.


### Single instruction

You can pick out a single test case and run it to view the instruction effects via output with the following command:

```sh
solana-test-suite debug-instruction --input-dir <input_dir> --target <shared_lib>
```

| Argument | Description |
|-----------------|-----------------------------------------------------------------------------------------------------|
| `--input` | Input file |
| `--target` | Shared object (.so) target file path to debug |


### Debugging

For failing test cases, it may be useful to analyze what could have differed between Solana and Firedancer. You can execute a Protobuf message (human-readable or binary) through the desired client as such:

```sh
solana-test-suite debug-instruction --input-dir <input_dir> --executable-path <executable_path> --debugger <gdb,rust-gdb,etc>
solana-test-suite debug-instruction --input-dir <input_dir> --target <shared_lib> --debugger <gdb,rust-gdb,etc>
```

| Argument | Description |
Expand Down
36 changes: 35 additions & 1 deletion src/test_suite/test_suite.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,13 @@
from google.protobuf import text_format
from test_suite.constants import LOG_FILE_SEPARATOR_LENGTH
import test_suite.invoke_pb2 as pb
from test_suite.codec_utils import encode_input
from test_suite.codec_utils import encode_input, encode_output
from test_suite.multiprocessing_utils import (
check_consistency_in_results,
generate_test_case,
initialize_process_output_buffers,
merge_results_over_iterations,
process_instruction,
process_single_test_case,
build_test_results,
)
Expand All @@ -26,6 +27,39 @@
)


@app.command()
def execute_single_instruction(
file: Path = typer.Option(None, "--input", "-i", help="Input file"),
shared_library: Path = typer.Option(
Path("impl/firedancer/build/native/clang/lib/libfd_exec_sol_compat.so"),
"--target",
"-t",
help="Shared object (.so) target file path to execute",
),
randomize_output_buffer: bool = typer.Option(
False,
"--randomize-output-buffer",
"-r",
help="Randomizes bytes in output buffer before shared library execution",
),
):
_, instruction_context = generate_test_case(file)
assert instruction_context is not None, f"Unable to read {file.name}"

# Initialize output buffers and shared library
initialize_process_output_buffers(randomize_output_buffer=randomize_output_buffer)
lib = ctypes.CDLL(shared_library)
lib.sol_compat_init()

# Execute and cleanup
instruction_effects = process_instruction(lib, instruction_context)
lib.sol_compat_fini()

# Print human-readable output
encode_output(instruction_effects)
print(instruction_effects)


@app.command()
def debug_instruction(
file: Path = typer.Option(None, "--input", "-i", help="Input file"),
Expand Down
Loading