Julia does not support cross-compilation to 32-bits

[tshort]

I'm trying to work out some kinks in the WebAssembly support in StaticCompiler. The main issue is addressing nested structs when I'm compiling on a 64-bit version of Julia. Here's an example that illustrates the problem:

# adapted from https://github.com/Alexander-Barth/FluidSimDemo-WebAssembly/blob/main/wasm_target.jl
# adapted from https://seelengrab.github.io/articles/Running%20Julia%20baremetal%20on%20an%20Arduino/

using GPUCompiler

struct WASMTarget <: GPUCompiler.AbstractCompilerTarget end


# GPUCompiler.llvm_triple(::WASMTarget) = "x86_64-pc-linux-gnu" # works
# GPUCompiler.llvm_triple(::WASMTarget) = "i686-pc-linux-gnu" # broken
GPUCompiler.llvm_triple(::WASMTarget) = "wasm32-unknown-unknown" # broken


GPUCompiler.runtime_slug(::GPUCompiler.CompilerJob{WASMTarget}) = "wasm-test"

struct WASMTargetParams <: GPUCompiler.AbstractCompilerParams end


module StaticRuntime
    # the runtime library
    signal_exception() = return
    malloc(sz) = C_NULL
    report_oom(sz) = return
    report_exception(ex) = return
    report_exception_name(ex) = return
    report_exception_frame(idx, func, file, line) = return
end

GPUCompiler.runtime_module(::GPUCompiler.CompilerJob{<:Any,WASMTargetParams}) = StaticRuntime
GPUCompiler.runtime_module(::GPUCompiler.CompilerJob{WASMTarget}) = StaticRuntime
GPUCompiler.runtime_module(::GPUCompiler.CompilerJob{WASMTarget,WASMTargetParams}) = StaticRuntime


function wasm_job(@nospecialize(func), @nospecialize(types))
    source = methodinstance(typeof(func), types)
    target = WASMTarget()
    params = WASMTargetParams()
    config = GPUCompiler.CompilerConfig(target, params, name = string(func), kernel = false)
    job = GPUCompiler.CompilerJob(source, config)
end

struct W{X,Y,Z} 
    x::X
    y::Y
    z::Z
end

w = W(W([10.0:-1:1;], [1.0:10;], 1.0), 1.0, 2.0)
WT = typeof(w)

f_y(w) = @inbounds w.x.y[1]

job = wasm_job(f_y, Base.to_tuple_type((WT,)))
GPUCompiler.code_llvm(job, dump_module = true)

Here's the result of that.

target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:10:11:12:13"
target triple = "wasm32-unknown-unknown"

;  @ /home/tshort/wasm/wasm-mwe.jl:52 within `f_y`
define double @f_y({ { {}*, {}*, double }, double, double }* nocapture noundef nonnull readonly align 8 dereferenceable(40) %0) local_unnamed_addr #0 {
top:
; ┌ @ Base.jl:37 within `getproperty`
   %1 = getelementptr inbounds { { {}*, {}*, double }, double, double }, { { {}*, {}*, double }, double, double }* %0, i32 0, i32 0, i32 2
   %2 = bitcast double* %1 to double***
   %3 = load atomic double**, double*** %2 unordered, align 8
; └
; ┌ @ essentials.jl:13 within `getindex`
   %4 = load double*, double** %3, align 8
   %5 = load double, double* %4, align 8
; └
  ret double %5
}

The problem is the getelementptr line. At the end, there is a i32 2. It should be a 1 instead of a 2 because it should be pointing at the second component of the nested struct using zero-based indexing.

I've tried some variations:

• It works fine when run in a 32-bit version of Julia.
• It works fine when I change the target to a 64-bit target. (And, it even works to compile that to WebAssembly.)
• In works fine if all three components of the first nested struct are arrays. It's only when the last one is a scalar that I get the error.

Anyone have hints or suggestions on how to track down the issue or correct an error in this?

[jpsamaroo]

I suspect running with opaque pointers in Julia 1.10 should probably resolve this, as the pointer size is not hard-coded into the IR.

[vchuravy]

Offsets will still be wrong... We may be able to fix this in Julia proper, but right now cross-compilation from 64bit to 32bit is not a supported feature.

[tshort]

I was wondering if that might be the answer (can't compile from 64 bit to 32 bit). For this case, I can make things work by setting the pointer size back to 64 bits in the data layout:

GPUCompiler.llvm_datalayout(::WASMTarget) = "e-m:e-p:64:32:32-i64:32:32-f64:32:64"

Then, Julia generates correct IR, and LLVM compiles to WebAssembly okay. That's probably a kludge and not a real solution.

[tshort]

When accessing fields, Julia can either:

1. Use a direct byte offset. This uses the host data layout.
2. Use a getelementptr with an LLVM structure representation. This leaves it up to LLVM to determine the index. This uses the target data layout.

The second is what you want for cross compiling. I've actually had better luck with the first for WebAssembly. I think the first is done for mutable structs, and the second is done for immutable structs. I haven't figured out the rules though (here, I think).

Using a 32-bit version of Julia doesn't solve the problem. A 32-bit version of Julia can have a different data layout than 32-bit WebAssembly, so the offsets are different.

Julia is just not set up well for cross compiling.

[vchuravy]

With opaque pointers gep will also use byte offsets, we haven't turned on opaque pointers, but eventually we must. Might be Julia 1.11