changes

.gitignore

@@ -167,4 +167,4 @@ cython_debug/
 scratch/
 
 # protosol
-protosol/
+protosol/

commands.md

@@ -17,20 +17,19 @@ $ solana-test-suite [OPTIONS] COMMAND [ARGS]...
 **Commands**:
 
 * `create-fixtures`: Create test fixtures from a directory of...
-* `debug-instr`
 * `debug-mismatches`: Run tests on a set of targets with a list...
 * `debug-non-repros`: Run tests on a set of targets with a list...
-* `decode-protobuf`: Convert messages to human-readable format.
-* `exec-instr`: Execute message(s) and print the effects.
-* `instr-from-fixtures`: Extract messages from fixtures.
+* `decode-protobufs`: Convert Context and/or Fixture messages to...
+* `execute`: Execute Context or Fixture message(s) and...
+* `fix-to-ctx`: Extract Context messages from Fixtures.
 * `list-harness-types`: List harness types available for use.
 * `regenerate-all-fixtures`: Regenerate all fixtures in provided...
-* `regenerate-fixtures`: Regenerate messages by checking FeatureSet...
+* `regenerate-fixtures`: Regenerate Fixture messages by checking...
 * `run-tests`: Run tests on a set of targets with a...
 
 ## `solana-test-suite create-fixtures`
 
-Create test fixtures from a directory of messages.
+Create test fixtures from a directory of Context and/or Fixture messages.
 Effects are generated by the target passed in with --solana-target or -s. 
 You can also pass in additional targets with --target or -t 
 and use --keep-passing or -k to only generate effects for test cases that match.
@@ -43,33 +42,17 @@ $ solana-test-suite create-fixtures [OPTIONS]
 
 **Options**:
 
-* `-i, --input-dir PATH`: Either a file or directory containing messages  [default: corpus8]
-* `-h, --default-harness-type TEXT`: Harness type to use for context protobufs  [default: InstrHarness]
+* `-i, --input PATH`: Input protobuf file or directory of protobuf files  [default: corpus8]
+* `-h, --default-harness-type TEXT`: Harness type to use for Context protobufs  [default: InstrHarness]
 * `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path  [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
-* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have defined
+* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have required function entrypoints defined
 * `-o, --output-dir PATH`: Output directory for fixtures  [default: test_fixtures]
 * `-p, --num-processes INTEGER`: Number of processes to use  [default: 4]
 * `-r, --readable`: Output fixtures in human-readable format
 * `-k, --keep-passing`: Only keep passing test cases
 * `-g, --group-by-program`: Group fixture output by program type
 * `--help`: Show this message and exit.
 
-## `solana-test-suite debug-instr`
-
-**Usage**:
-
-```console
-$ solana-test-suite debug-instr [OPTIONS]
-```
-
-**Options**:
-
-* `-i, --input PATH`: Input file
-* `-t, --target PATH`: Shared object (.so) target file path to debug  [default: impl/lib/libsolfuzz_firedancer.so]
-* `-d, --debugger TEXT`: Debugger to use (gdb, rust-gdb)  [default: gdb]
-* `-h, --default-harness-type TEXT`: Harness type to use for context protobufs  [default: InstrHarness]
-* `--help`: Show this message and exit.
-
 ## `solana-test-suite debug-mismatches`
 
 Run tests on a set of targets with a list of FuzzCorp mismatch links.
@@ -85,7 +68,7 @@ $ solana-test-suite debug-mismatches [OPTIONS]
 **Options**:
 
 * `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path  [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
-* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have defined  [default: /home/kbhargava/repos/firedancer/build/native/gcc/lib/libfd_exec_sol_compat.so]
+* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have required function entrypoints defined  [default: /home/kbhargava/repos/firedancer/build/native/gcc/lib/libfd_exec_sol_compat.so]
 * `-o, --output-dir PATH`: Output directory for messages  [default: debug_mismatch]
 * `-u, --repro-urls TEXT`: Comma-delimited list of FuzzCorp mismatch links
 * `-s, --section-names TEXT`: Comma-delimited list of FuzzCorp section names
@@ -107,61 +90,62 @@ $ solana-test-suite debug-non-repros [OPTIONS]
 **Options**:
 
 * `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path  [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
-* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have defined  [default: /home/kbhargava/repos/firedancer/build/native/gcc/lib/libfd_exec_sol_compat.so]
+* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have required function entrypoints defined  [default: /home/kbhargava/repos/firedancer/build/native/gcc/lib/libfd_exec_sol_compat.so]
 * `-o, --output-dir PATH`: Output directory for messages  [default: debug_mismatch]
 * `-u, --repro-urls TEXT`: Comma-delimited list of FuzzCorp mismatch links
 * `-s, --section-names TEXT`: Comma-delimited list of FuzzCorp section names
 * `-f, --fuzzcorp-url TEXT`: Comma-delimited list of FuzzCorp section names  [default: https://api.dev.fuzzcorp.asymmetric.re/uglyweb/firedancer-io/solfuzz/bugs/]
 * `--help`: Show this message and exit.
 
-## `solana-test-suite decode-protobuf`
+## `solana-test-suite decode-protobufs`
 
-Convert messages to human-readable format.
+Convert Context and/or Fixture messages to human-readable format.
 
 **Usage**:
 
 ```console
-$ solana-test-suite decode-protobuf [OPTIONS]
+$ solana-test-suite decode-protobufs [OPTIONS]
 ```
 
 **Options**:
 
-* `-i, --input PATH`: Either a message or directory of messages  [default: raw_context]
-* `-o, --output-dir PATH`: Output directory for base58-encoded, human-readable messages  [default: readable_context]
+* `-i, --input PATH`: Input protobuf file or directory of protobuf files  [default: raw_context]
+* `-o, --output-dir PATH`: Output directory for base58-encoded, Context and/or Fixture human-readable messages  [default: readable_context]
 * `-p, --num-processes INTEGER`: Number of processes to use  [default: 4]
-* `-h, --default-harness-type TEXT`: Harness type to use for context protobufs  [default: InstrHarness]
+* `-h, --default-harness-type TEXT`: Harness type to use for Context protobufs  [default: InstrHarness]
 * `--help`: Show this message and exit.
 
-## `solana-test-suite exec-instr`
+## `solana-test-suite execute`
 
-Execute message(s) and print the effects.
+Execute Context or Fixture message(s) and print the Effects.
 
 **Usage**:
 
 ```console
-$ solana-test-suite exec-instr [OPTIONS]
+$ solana-test-suite execute [OPTIONS]
 ```
 
 **Options**:
 
-* `-i, --input PATH`: Input file or directory of files
+* `-i, --input PATH`: Input protobuf file or directory of protobuf files
+* `-h, --default-harness-type TEXT`: Harness type to use for Context protobufs  [default: InstrHarness]
 * `-t, --target PATH`: Shared object (.so) target file path to execute  [default: impl/firedancer/build/native/clang/lib/libfd_exec_sol_compat.so]
 * `-r, --randomize-output-buffer`: Randomizes bytes in output buffer before shared library execution
 * `--help`: Show this message and exit.
 
-## `solana-test-suite instr-from-fixtures`
+## `solana-test-suite fix-to-ctx`
 
-Extract messages from fixtures.
+Extract Context messages from Fixtures.
 
 **Usage**:
 
 ```console
-$ solana-test-suite instr-from-fixtures [OPTIONS]
+$ solana-test-suite fix-to-ctx [OPTIONS]
 ```
 
 **Options**:
 
-* `-i, --input-dir PATH`: Input directory containing messages  [default: fixtures]
+* `-i, --input PATH`: Input Fixture file or directory of Fixture files  [default: fixtures]
 * `-o, --output-dir PATH`: Output directory for messages  [default: instr]
 * `-p, --num-processes INTEGER`: Number of processes to use  [default: 4]
 * `--help`: Show this message and exit.
@@ -192,16 +176,15 @@ $ solana-test-suite regenerate-all-fixtures [OPTIONS]
 
 **Options**:
 
-* `-i, --input-dir PATH`: Input test-vectors directory  [default: corpus8]
+* `-i, --input PATH`: Input test-vectors directory  [default: corpus8]
 * `-o, --output-dir PATH`: Output directory for regenerated fixtures  [default: /tmp/regenerated_fixtures]
 * `-t, --target PATH`: Shared object (.so) target file path to execute  [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
 * `-s, --stubbed-target PATH`: Stubbed shared object (.so) target file path to execute  [default: /home/kbhargava/repos/solfuzz-agave/target/x86_64-unknown-linux-gnu/release/libsolfuzz_agave_stubbed.so]
 * `--help`: Show this message and exit.
 
 ## `solana-test-suite regenerate-fixtures`
 
-Regenerate messages by
-checking FeatureSet compatibility with the target shared library. 
+Regenerate Fixture messages by checking FeatureSet compatibility with the target shared library. 
 
 **Usage**:
 
@@ -211,7 +194,7 @@ $ solana-test-suite regenerate-fixtures [OPTIONS]
 
 **Options**:
 
-* `-i, --input-dir PATH`: Either a file or directory containing messages  [default: corpus8]
+* `-i, --input PATH`: Either a file or directory containing messages  [default: corpus8]
 * `-t, --target PATH`: Shared object (.so) target file path to execute  [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
 * `-o, --output-dir PATH`: Output directory for regenerated fixtures  [default: regenerated_fixtures]
 * `-d, --dry-run`: Only print the fixtures that would be regenerated
@@ -220,8 +203,7 @@ $ solana-test-suite regenerate-fixtures [OPTIONS]
 
 ## `solana-test-suite run-tests`
 
-Run tests on a set of targets with a directory of 
-or messages.
+Run tests on a set of targets with a directory of Context and/or Fixture messages.
 
 Note: each `.so` target filename must be unique.
 
@@ -233,7 +215,8 @@ $ solana-test-suite run-tests [OPTIONS]
 
 **Options**:
 
-* `-i, --input PATH`: Single input file or input directory containing or messages  [default: corpus8]
+* `-i, --input PATH`: Input protobuf file or directory of protobuf files  [default: corpus8]
+* `-h, --default-harness-type TEXT`: Harness type to use for Context protobufs  [default: InstrHarness]
 * `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path  [default: /home/kbhargava/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
 * `-t, --target PATH`: Shared object (.so) target file paths  [default: /home/kbhargava/repos/firedancer/build/native/gcc/lib/libfd_exec_sol_compat.so]
 * `-o, --output-dir PATH`: Output directory for test results  [default: test_results]

src/test_suite/multiprocessing_utils.py

@@ -61,7 +61,7 @@ def process_target(
 
 def read_context_serialized(harness_ctx: HarnessCtx, test_file: Path) -> str | None:
     """
-    Reads in test files and generates an InstrContext Protobuf object for a test case.
+    Reads in test files and generates an Context Protobuf object for a test case.
 
     Args:
         - test_file (Path): Path to the instruction context message.
@@ -197,7 +197,7 @@ def read_fixture(fixture_file: Path) -> message.Message | None:
     return fixture
 
 
-def decode_single_test_case(harness_ctx: HarnessCtx, test_file: Path) -> int:
+def decode_single_test_case(test_file: Path) -> int:
     """
     Decode a single test case into a human-readable message
 
@@ -207,21 +207,34 @@ def decode_single_test_case(harness_ctx: HarnessCtx, test_file: Path) -> int:
     Returns:
         - int: 1 if successfully decoded and written, 0 if skipped.
     """
-    serialized_instruction_context = read_context_serialized(harness_ctx, test_file)
+    if test_file.suffix == ".fix":
+        fn_entrypoint = extract_metadata(test_file).fn_entrypoint
+        harness_ctx = HARNESS_ENTRYPOINT_MAP[fn_entrypoint]
+        serialized_protobuf = read_fixture_serialized(test_file)
+    else:
+        harness_ctx = globals.default_harness_ctx
+        serialized_protobuf = read_context_serialized(harness_ctx, test_file)
 
     # Skip if input is invalid
-    if serialized_instruction_context is None:
+    if serialized_protobuf is None:
         return 0
 
     # Encode the input fields to be human readable
-    instruction_context = harness_ctx.context_type()
-    instruction_context.ParseFromString(serialized_instruction_context)
-    harness_ctx.context_human_encode_fn(instruction_context)
+    if test_file.suffix == ".fix":
+        output = harness_ctx.fixture_type()
+    else:
+        output = harness_ctx.context_type()
+
+    output.ParseFromString(serialized_protobuf)
+
+    if test_file.suffix == ".fix":
+        harness_ctx.context_human_encode_fn(output.input)
+        harness_ctx.effects_human_encode_fn(output.output)
+    else:
+        harness_ctx.context_human_encode_fn(output)
 
     with open(globals.output_dir / (test_file.stem + ".txt"), "w") as f:
-        f.write(
-            text_format.MessageToString(instruction_context, print_unknown_fields=False)
-        )
+        f.write(text_format.MessageToString(output, print_unknown_fields=False))
     return 1
 
 
@@ -246,7 +259,6 @@ def process_single_test_case(
     # Execute test case on each target library
     results = {}
     for target in globals.target_libraries:
-        print(f"Processing {target}...")
         instruction_effects = process_target(
             harness_ctx,
             globals.target_libraries[target],
@@ -392,8 +404,11 @@ def run_test(test_file: Path) -> tuple[str, int, dict | None]:
             - Dictionary of target library names and file-dumpable serialized instruction effects
     """
     # Process fixtures through this entrypoint as well
-    fn_entrypoint = extract_metadata(test_file).fn_entrypoint
-    harness_ctx = HARNESS_ENTRYPOINT_MAP[fn_entrypoint]
+    if test_file.suffix == ".fix":
+        fn_entrypoint = extract_metadata(test_file).fn_entrypoint
+        harness_ctx = HARNESS_ENTRYPOINT_MAP[fn_entrypoint]
+    else:
+        harness_ctx = globals.default_harness_ctx
     context = serialize_context(harness_ctx, test_file)
     results = process_single_test_case(harness_ctx, context)
     pruned_results = harness_ctx.prune_effects_fn(harness_ctx, context, results)