Native compilation

[lastmjs]

No description provided.

[bdemann]

What is the feature?

Does it have an issue?

No. It's just a sub point of #1691

What is it doing?

Explanation

• make --native azle flag for native compilation
  • should we add tests for native compilation? Maybe just choose a couple examples to do this with?

Questions

What on Earth is native compilation?

It looks like it is compiling the old way without podman/docker. I am guessing
we are adding this for people with computers that have docker limitations.
If that is incorrect then I'll have to redo this.

Expectations

Tests

I think I would expect the tests to be the same. I would just expect as the
explanation suggests that a handful of test would be selected to be compiled
natively.

Code

I would expect the code the just have an if statement based on the flag and
either run all of the command through podman or run all of the commands that
podman would normally run on the developers computer. I don't expect there to
be much complication with that. There are only two options, native or podman.

There might be better error handling, but I am guessing that would be a fair
deal of work and would be outside the scope of this feature.

I would expect it to look like what we had before podman.

This is from a bit of spoilers that I saw while I was trying to figure out what
this was but I would also expect now that the GitHub Actions would have to
install all of the dependencies again. I will be interested to see how that is
handled so that we aren't installing all of the dependencies all of the time. Or
maybe we are doing that. If so that would be maybe not the most efficient things
for our tests...

[bdemann]

Old Version

import { execSync, IOType } from 'child_process';
import {
    GLOBAL_AZLE_RUST_DIR,
    GLOBAL_AZLE_RUST_BIN_DIR,
    GLOBAL_AZLE_TARGET_DIR,
    time
} from './utils';

export function compileRustCode(
    canisterName: string,
    canisterPath: string,
    stdio: IOType
) {
    execSync(
        `cd ${canisterPath} && ${GLOBAL_AZLE_RUST_BIN_DIR}/cargo build --target wasm32-wasi --manifest-path canister/Cargo.toml --release`,
        {
            stdio,
            env: {
                ...process.env,
                CARGO_TARGET_DIR: GLOBAL_AZLE_TARGET_DIR,
                CARGO_HOME: GLOBAL_AZLE_RUST_DIR,
                RUSTUP_HOME: GLOBAL_AZLE_RUST_DIR,
                CARGO_REGISTRIES_CRATES_IO_PROTOCOL: 'sparse'
                // TODO this allows changing the stack size, could be useful for stack overflow or heap out of bounds errors
                // RUSTFLAGS: '-C link-args=-zstack-size=2000000000'
            }
        }
    );

    const wasmTargetFilePath = `${GLOBAL_AZLE_TARGET_DIR}/wasm32-wasi/release/canister.wasm`;

    execSync(`cp ${wasmTargetFilePath} ${canisterPath}`);

    execSync(
        `cd ${canisterPath} && ${GLOBAL_AZLE_RUST_BIN_DIR}/wasi2ic canister.wasm ${canisterName}.wasm`,
        {
            stdio,
            env: {
                ...process.env,
                CARGO_TARGET_DIR: GLOBAL_AZLE_TARGET_DIR,
                CARGO_HOME: GLOBAL_AZLE_RUST_DIR,
                RUSTUP_HOME: GLOBAL_AZLE_RUST_DIR
            }
        }
    );
}

So in this old version we have three commands we are running

1. cd <canister-path> && cargo build <with target, manifest, release, and env>
2. cp <wasm> <canister>
3. cd <canister-path> && wasi2ic canister.wasm <cansiterName>.wasm <with env>

There is a bunch more stuff to figure out all the right paths and file names but
that's about it.

[bdemann]

Results

There are a bunch of files that added target back to the .gitignore and added
the --native-compilation flag to the dfx.json. I don't think those files need to
be reviewed much.

test.ymal adds back all of the dependencies for native installation. It doesn't
look like it cares if the node is going to be running a native compilation test
or a podman test. Is it even possible to determine that before the test is running?

That leaves the meat of the review to pass through the rubric.

Rubric

1. Consistency (Same as the rest of the code and itself)
   1. No new files, new variable (nativeCompilation) looks good. New
      functions (compileRustCodeNatively and compileRustCodeWithPodman) look
      good. There isn't much more to look at here. It's just a couple of if
      statements and passing variables through.
2. Readability (Easy to understand)
   1. As far as readability goes I am finding it a bit confusing that we still
      use things based off of the dockerContainerName or dockerHash even if we are
      compiling natively. like here or here.
      I get that it's nice to have one function for compileRustCode and then it
      determines which way to compile it, but I would find it weird if I ever, for
      example, called it in a context where we were only compiling natively, and we
      don't even have a dockerContainerName. Realistically that will probably
      never happen...
3. Maintainability (modular, reusable, and simple)
   1. Looks good, not much has been added that would affect this, both
      compileRustCodeWithPodman and compileRustCodeNatively are encapsulated
      in their own functions, that's good.
4. Correctness (as bug free as reasonable)
   1. I think the only things that pertain to this category are more
      appropriately covered in the following.
5. Code level implications (code coverage, error coverage, conditionals coverage, efficiency, etc)
   1. I guess
      https://github.com/demergent-labs/azle/pull/1700/files#diff-bc0edaf897df65b29139d6cc26dfacd8dd366af89f38b99e6e22c672df55cfe6R72
      would raise the question of if there are other things we ought to ignore if
      we are doing native compilation. Which one of us has the burden to prove
      that it's just preparing the image and then the actual compilation?
   2. This looks like a different compilation process than before?
      1. Expected

          cd <canister-path> && cargo build <with target, manifest, release, and env>
          cp <wasm> <canister>
          cd <canister-path> && wasi2ic canister.wasm <cansiterName>.wasm <with env>

      1. Actual

          cargo build <with target, manifest, release>
          wasi2ic canister.wasm <cansiterName>.wasm

      We aren't cding or cping any more... what's up with that?
6. Quality of tests and documentation (thinking about how others will use it)
   1. No updates to the tests besides what I was expecting, (a few that
      have the addition of the native-compilation flag and that's it)
   2. Where would documentation for this feature be?
      1. We should at least add this flag to azle --help We should add
         azle --help. Could you make an issue to create azle --help that will
         explain all of our flags and usage to developers?

Code

Code is more or less as I was expecting, no big changes need.

Tests

Tests are mostly unchanged with just a few changing their compilation method,
just as expected.

The merge-base changed after approval.

The merge-base changed after approval.