Modify tig-worker executable to be a performance tester.

README.md

@@ -5,8 +5,7 @@ This repository contains the implementation of The Innovation Game (TIG).
 ## Important Links
 
 * [TIG's tech explainer](docs/1_basics.md)
-* [Getting started with Innovating](tig-algorithms/README.md)
-* [Getting started with Benchmarking](tig-benchmarker/README.md)
+* [Getting Started with Innovating](docs/1_basics.md)
 * [Challenge descriptions](tig-challenges/docs/knapsack.md)
 
 ## Repo Contents
@@ -64,6 +63,166 @@ A Rust crate for verifying and computing solutions.
 
 Solutions are computed by executing an algorithm in a WASM virtual machine ([TIG's fork of wasmi](https://github.com/tig-foundation/wasmi)).
 
+## Getting Started with Innovating
+
+### Setting up Private Fork
+
+Innovators will want to create a private fork so that they can test that their algorithm can be successfully compiled into WASM by the CI.
+
+1. Create private repository on GitHub
+2. Create empty git repository on your local machine
+    ```
+    mkdir tig-monorepo
+    cd tig-monorepo
+    git init
+    ```
+3. Setup remotes with origin pointed to your private repository
+    ```
+    git remote add origin <your private repo>
+    git remote add public https://github.com/tig-foundation/tig-monorepo.git
+    ```
+    
+4. Pulling `blank_slate` from TIG public repository (branch with no algorithms)
+    ```
+    git fetch public
+    git checkout -b blank_slate
+    git pull public blank_slate
+    ```
+    
+5. Push to your private repository
+    ```
+    git push origin blank_slate
+    ```
+
+### Checking out Existing Algorithms
+
+Every algorithm has its own `<branch>` with name `<challenge_name>/<algorithm_name>`.
+
+Only algorithms that are successfully compiled into WASM have their branch pushed to this public repository.
+
+Each algorithm branch will have 2 files:
+1. Rust code @ `tig-algorithms/src/<branch>.rs`
+2. Wasm blob @ `tig-algorithms/wasm/<branch>.wasm`
+
+To pull an existing algorithm from TIG public repository, run the following command:
+```
+git fetch public
+git pull public <branch>
+```
+
+### Developing Your Algorithm
+
+1. Pick a challenge (`<challenge_name>`) to develop an algorithm for
+2. Make a copy of an existing algorithm's rust code or `tig-algorithms/<challenge_name>/template.rs`
+3. Rename the file with your own `<algorithm_name>`
+4. Edit `tig-algorithms/<challenge_name>/mod.rs` to export your algorithm and test it:
+    ```
+    pub mod <algorithm_name>;
+
+    #[cfg(test)]
+    mod tests {
+        use super::*;
+        use tig_challenges::{<challenge_name>::*, *};
+
+        #[test]
+        fn test_<algorithm_name>() {
+            let difficulty = Difficulty {
+                // Uncomment the relevant fields.
+
+                // -- satisfiability --
+                // num_variables: 50,
+                // clauses_to_variables_percent: 300,
+                
+                // -- vehicle_routing --
+                // num_nodes: 40,
+                // better_than_baseline: 250,
+                
+                // -- knapsack --
+                // num_items: 50,
+                // better_than_baseline: 10,
+            };
+            let seed = 0;
+            let challenge = Challenge::generate_instance(seed, &difficulty).unwrap();    
+            <algorithm_name>::solve_challenge(&challenge).unwrap();
+        }
+    }
+    ```
+5. Check that your algorithm compiles & runs:
+    ```
+    cargo test -p tig-algorithms
+    ```
+
+Notes:
+* Do not include tests in your algorithm file. TIG will reject your algorithm submission.
+* Only your algorithm's rust code gets submitted. You should not be adding dependencies to `tig-algorithms` as they will not be available when TIG compiles your algorithm
+
+### Locally Compiling Your Algorithm into WASM 
+
+These steps replicate what TIG's CI does (`.github/workflows/build_algorithm.yml`):
+
+1. Set environment variables to match the algorithm you are compiling:
+    ```
+    export CHALLENGE=<challenge_name>
+    export ALGORITHM=<algorithm_name>
+    ```
+2. Compile your algorithm
+    ```
+    cargo build -p tig-wasm --target wasm32-wasi --release --features entry-point
+    ```
+3. Optimise the WASM and save it into `tig-algorithms/wasm`:
+    ```
+    mkdir -p tig-algorithms/wasm/${CHALLENGE}
+    wasm-opt target/wasm32-wasi/release/tig_wasm.wasm -o tig-algorithms/wasm/${CHALLENGE}/${ALGORITHM}.wasm -O2 --remove-imports
+    ```
+
+### Testing Performance of Algorithms
+
+Performance testing is done by `tig-worker` in a sandboxed WASM Virtual Machine.
+
+**IMPORTANT**: You can compile / test existing algorithms as binary executables, but be sure to throughly vet the code beforehand for malicious routines!
+
+1. Pull an existing algorithm or compile your own algorithm to WASM
+2. Set environment variables to match the algorithm you are testing:
+    ```
+    export CHALLENGE=<challenge_name>
+    export ALGORITHM=<algorithm_name>
+    ```
+3. Pick a difficulty & create `settings.json`:
+    ```
+    {
+        "block_id": "",
+        "algorithm_id": "",
+        "challenge_id": "",
+        "player_id": "",
+        "difficulty": [50, 300]
+    }
+    ```
+4. Test the algorithm:
+    ```
+    cargo run -p tig-worker --release -- settings.json tig-algorithms/wasm/${CHALLENGE}/${ALGORITHM}.wasm
+    ```
+
+Notes:
+* You can query the latest difficulty ranges via TIG's API:
+    ```
+    query https://api.tig.foundation/play/get-block for <block_id>
+    query https://api.tig.foundation/play/get-challenges?block_id=<block_id> for qualifier_difficulties
+    ```
+
+### Checking CI Successfully Compiles Your Algorithm
+
+TIG pushes all algorithms to their own branch which triggers the CI (`.github/workflows/build_algorithm.yml`).
+
+To trigger the CI on your private repo, your branch just needs to have a particular name:
+```
+git checkout -b <challenge_name>/<algorithm_name>
+git push origin <challenge_name>/<algorithm_name>
+```
+
+### Making Your Submission
+
+You will need to burn 0.001 ETH to make a submission. Visit https://play.tig.foundation/innovator and follow the instructions.
+
 ## License
 
 Placeholder

tig-benchmarker/src/benchmarker.rs

@@ -11,7 +11,7 @@ use std::collections::{HashMap, HashSet};
 use tig_api::*;
 use tig_structs::{config::WasmVMConfig, core::*};
 use tig_utils::*;
-use tig_worker::compute_solution;
+use tig_worker::{compute_solution, ComputeResult};
 
 type Result<T> = std::result::Result<T, String>;
 
@@ -560,15 +560,13 @@ async fn do_benchmark() -> Result<()> {
             blob = download_wasm_blob(&job.settings.algorithm_id).await?;
             last_algorithm_id = job.settings.algorithm_id.clone();
         }
-        if let Ok(solution_data) = compute_solution(
+        if let Ok(ComputeResult::ValidSolution(solution_data)) = compute_solution(
             &job.settings,
             nonce,
             blob.as_slice(),
             job.wasm_vm_config.max_memory,
             job.wasm_vm_config.max_fuel,
-        )
-        .map_err(|e| e.to_string())?
-        {
+        ) {
             if solution_data.calc_solution_signature() <= job.solution_signature_threshold {
                 let mut state = mutex().lock().await;
                 if let Some(Some(solutions_meta_data)) = (*state)

tig-challenges/src/lib.rs

@@ -27,8 +27,10 @@ where
     }
 
     fn verify_solution(&self, solution: &T) -> Result<()>;
-    fn verify_solution_from_json(&self, solution: &str) -> Result<Result<()>> {
-        Ok(self.verify_solution(&serde_json::from_str(solution)?))
+    fn verify_solution_from_json(&self, solution: &str) -> Result<()> {
+        let solution = serde_json::from_str(solution)
+            .map_err(|e| anyhow!("Failed to parse solution: {}", e))?;
+        self.verify_solution(&solution)
     }
 }
 

tig-protocol/src/context.rs

@@ -1,6 +1,7 @@
 pub use anyhow::{Error as ContextError, Result as ContextResult};
 use tig_structs::{config::*, core::*};
 
+#[derive(Debug, Clone, PartialEq)]
 pub enum SubmissionType {
     Algorithm,
     Benchmark,

tig-wasm/src/entry_point_template.rs

@@ -7,20 +7,25 @@ use tig_utils::compress_obj;
 
 #[no_mangle]
 pub fn entry_point(seed: u32, difficulty: Difficulty, ptr: *mut u8, max_length: usize) {
-    let challenge = Challenge::generate_instance(seed, &difficulty).expect("Failed to generate challenge");    
-    if let Ok(Some(solution)) = {ALGORITHM}::solve_challenge(&challenge) {
-        if challenge.verify_solution(&solution).is_ok() {
-            let mut buffer = Vec::<u8>::new();
-            let compressed = compress_obj(solution);
-            buffer.extend((compressed.len() as u32).to_be_bytes());
-            buffer.extend(compressed);                
-            if buffer.len() > max_length {
-                panic!("Encoded solution exceeds maximum length");
-            }
-
-            for (i, &byte) in buffer.iter().enumerate() {
-                unsafe { *ptr.add(i) = byte };
-            }
+    let challenge =
+        Challenge::generate_instance(seed, &difficulty).expect("Failed to generate challenge");
+    let (is_solution, compressed) =
+        if let Ok(Some(solution)) = {ALGORITHM}::solve_challenge(&challenge) {
+            (
+                challenge.verify_solution(&solution).is_ok(),
+                compress_obj(solution),
+            )
+        } else {
+            (false, Vec::<u8>::new())
+        };
+    let mut buffer = Vec::<u8>::new();
+    buffer.push(is_solution as u8);
+    buffer.extend((compressed.len() as u32).to_be_bytes());
+    buffer.extend(compressed);
+    for (i, &byte) in buffer.iter().enumerate() {
+        if i >= max_length {
+            break;
         }
+        unsafe { *ptr.add(i) = byte };
     }
 }