Simplify customization of protocol updates and align better with node

[dhedey]

Summary

The aim is to simplify configuring custom protocol versions and (importantly) align this with the node.

• Adds ProtocolBuilder / ProtocolExecutor
• Simplifies TransactionScenarioExecutor
• Adds intelligent state update dumping when formatting a transaction receipt
• Fix scenario dumping to function like metering dumping:
  • We dump to files (manifests, costings, receipts, raw transactions) and a summary.
  • Like with metering, we sub-divide by the version of the protocol which the scenario is run against. This can serve as providing extra confidence that we haven't changed historic protocol versions by mistake.

NOTE: To avoid polluting this PR, I've moved the regeneration to See #1789, which should be merged into this one before merging this into develop.

Natural / Easy Extensions

The following are all rather quick and easy, but I might not have time to do them now, and I'm splitting them out to avoid this PR getting too big:

• Add callbacks for transaction execution on Protocol Builder to get callbacks on protocol transactions
• Add tests that protocol transactions can be typed - using this callback (in e.g. typed_substate_layout).
• For backwards compatibility divergence, output the transaction receipts, hashes for genesis and protocol updates (just like we do with a scenario - except without the manifest for flash transactions)
• Merge Genesis / Bootstrapping into Protocol Builder
  • Possibly with the inclusion of ProtocolUpdateBatchGroups sitting between Updates and Batches, so that Genesis can be modelled as 3 Batch Groups
• Add "safe_to_run_on_used_ledger" Metadata flag and a test which validates that:
  • If !safe_to_run_on_used_ledger then it's not set to run on testnets past genesis.
  • If safe_to_run_on_used_ledger then it's not referencing any pre-allocated account addresses in well-known

Future extensions:

• Have some static map of HashOfProtocolSettings => PostBootstrapClonableInMemoryDb in LedgerSimulator, to allow tests to run faster by skipping protocol initialization.

Testing

See the generated scenarios.

Update Recommendations

For dApp Developers

Transaction receipts now include state updates.

For Internal Integrators - Node

We can now use the following and map that to the BatchGenerator on the node side:

AnemoneSettings::all_enabled_as_default_for_network(&network_definition).create_batch_generator();
BottlenoseSettings::all_enabled_as_default_for_network(&network_definition).create_batch_generator();

(It was designed to align with Anemone/Bottlenose as-is on node's release/bottlenose branch)

[github-actions]

Docker tags
docker.io/radixdlt/private-scrypto-builder:5fc839cc9e

[github-actions]

Benchmark for 5fc839c

Click to view benchmark

Test	Base	PR	%
costing::bench_prepare_wasm	65.9±0.21ms	65.5±0.14ms	-0.61%
costing::decode_sbor	10.9±0.07µs	10.8±0.01µs	-0.92%
costing::decode_sbor_bytes	30.0±0.27µs	30.2±0.23µs	+0.67%
costing::deserialize_wasm	1287.3±3.95µs	1280.0±3.63µs	-0.57%
costing::instantiate_flash_loan	3.9±0.55ms	4.1±0.73ms	+5.13%
costing::instantiate_radiswap	5.7±0.06ms	5.6±0.07ms	-1.75%
costing::spin_loop	21.5±0.05ms	22.0±0.12ms	+2.33%
costing::validate_sbor_payload	34.3±0.08µs	34.7±0.23µs	+1.17%
costing::validate_sbor_payload_bytes	290.2±0.60ns	286.0±0.52ns	-1.45%
costing::validate_secp256k1	76.2±0.05µs	76.3±0.06µs	+0.13%
costing::validate_wasm	36.7±0.04ms	36.5±0.04ms	-0.54%
decimal::add/0	8.4±0.00ns	8.4±0.10ns	0.00%
decimal::add/rust-native	9.8±0.00ns	9.9±0.01ns	+1.02%
decimal::add/wasmer	111.7±0.15ns	112.8±0.02ns	+0.98%
decimal::add/wasmer-call-native	451.8±0.50ns	450.6±0.43ns	-0.27%
decimal::add/wasmi	624.8±0.56ns	664.9±1.26ns	+6.42%
decimal::add/wasmi-call-native	5.2±0.01µs	5.4±0.04µs	+3.85%
decimal::div/0	190.8±0.27ns	191.7±0.67ns	+0.47%
decimal::from_string/0	150.8±0.17ns	151.0±0.50ns	+0.13%
decimal::mul/0	143.0±0.04ns	142.8±0.18ns	-0.14%
decimal::mul/rust-native	138.9±0.11ns	138.9±0.14ns	0.00%
decimal::mul/wasmer	1495.3±0.35ns	1497.6±0.80ns	+0.15%
decimal::mul/wasmer-call-native	585.1±0.85ns	580.5±0.44ns	-0.79%
decimal::mul/wasmi	40.7±0.10µs	41.1±0.11µs	+0.98%
decimal::mul/wasmi-call-native	5.3±0.01µs	5.6±0.01µs	+5.66%
decimal::pow/0	651.9±0.32ns	653.9±0.56ns	+0.31%
decimal::pow/rust-native	629.3±0.40ns	628.8±0.23ns	-0.08%
decimal::pow/wasmer	6.5±0.00µs	6.6±0.01µs	+1.54%
decimal::pow/wasmer-call-native	1035.3±1.36ns	1026.1±1.08ns	-0.89%
decimal::pow/wasmi	194.0±0.48µs	194.4±0.21µs	+0.21%
decimal::pow/wasmi-call-native	5.1±0.01µs	5.3±0.01µs	+3.92%
decimal::root/0	7.7±0.02µs	7.9±0.01µs	+2.60%
decimal::sub/0	8.5±0.01ns	8.5±0.00ns	0.00%
decimal::to_string/0	437.6±0.38ns	439.9±0.24ns	+0.53%
precise_decimal::add/0	9.4±0.05ns	9.5±0.59ns	+1.06%
precise_decimal::add/rust-native	11.4±0.00ns	11.4±0.00ns	0.00%
precise_decimal::add/wasmer	126.3±0.07ns	127.1±0.14ns	+0.63%
precise_decimal::add/wasmer-call-native	492.7±0.52ns	493.9±0.48ns	+0.24%
precise_decimal::add/wasmi	786.1±0.67ns	832.1±1.94ns	+5.85%
precise_decimal::add/wasmi-call-native	6.5±0.01µs	7.2±0.02µs	+10.77%
precise_decimal::div/0	300.9±0.38ns	302.8±0.20ns	+0.63%
precise_decimal::from_string/0	195.4±0.31ns	195.1±0.15ns	-0.15%
precise_decimal::mul/0	361.1±4.27ns	345.3±1.08ns	-4.38%
precise_decimal::mul/rust-native	301.9±1.24ns	302.4±1.86ns	+0.17%
precise_decimal::mul/wasmer	3.4±0.00µs	3.4±0.00µs	0.00%
precise_decimal::mul/wasmer-call-native	862.1±0.92ns	820.1±2.08ns	-4.87%
precise_decimal::mul/wasmi	105.6±0.09µs	105.9±0.27µs	+0.28%
precise_decimal::mul/wasmi-call-native	7.1±0.02µs	7.3±0.07µs	+2.82%
precise_decimal::pow/0	1855.2±6.31ns	1850.1±2.17ns	-0.27%
precise_decimal::pow/rust-native	1469.6±4.96ns	1468.2±3.53ns	-0.10%
precise_decimal::pow/wasmer	16.1±0.01µs	16.1±0.01µs	0.00%
precise_decimal::pow/wasmer-call-native	2.1±0.00µs	2.1±0.00µs	0.00%
precise_decimal::pow/wasmi	509.9±1.43µs	512.2±0.61µs	+0.45%
precise_decimal::pow/wasmi-call-native	12.9±0.05µs	13.2±0.04µs	+2.33%
precise_decimal::root/0	55.4±0.15µs	56.2±0.02µs	+1.44%
precise_decimal::sub/0	9.5±0.00ns	9.5±0.00ns	0.00%
precise_decimal::to_string/0	725.4±0.98ns	730.4±1.47ns	+0.69%
schema::validate_payload	343.2±0.46µs	355.6±0.97µs	+3.61%
transaction::radiswap	5.4±0.02ms	5.3±0.02ms	-1.85%
transaction::transfer	1765.6±5.50µs	1734.8±8.83µs	-1.74%
transaction_processing::prepare	2.2±0.00ms	2.2±0.00ms	0.00%
transaction_processing::prepare_and_decompile	6.0±0.01ms	6.1±0.02ms	+1.67%
transaction_processing::prepare_and_decompile_and_recompile	24.0±0.14ms	25.0±1.53ms	+4.17%
transaction_validation::validate_manifest	42.1±0.02µs	42.2±0.03µs	+0.24%
transaction_validation::verify_bls_2KB	1026.3±32.88µs	1013.6±21.50µs	-1.24%
transaction_validation::verify_bls_32B	1024.8±39.29µs	1011.6±21.65µs	-1.29%
transaction_validation::verify_ecdsa	74.3±0.07µs	74.2±0.07µs	-0.13%
transaction_validation::verify_ed25519	54.9±0.17µs	55.0±0.04µs	+0.18%

[0xOmarA]

Did an initial review and pushed some comments.