Unable to compile ibm-granite/granite-3b-code-instruct

[cjvolzka]

Issue

Attempting to compile ibm-granite/granite-3b-code-instruct from HuggingFace using the 0.4.2.0 release fails. I get the same error compiling for CPU or NNPA

Error

onnx-mlir --O3 --EmitLib --mtriple=s390x-ibm-loz --mcpu=z16 --maccel=NNPA ./model.onnx --onnx-op-stats TXT
Operations encountered:
-----------------------
   func.func                  , 1
   func.return                , 1
   onnx.Add                   , 165
   onnx.And                   , 1
   onnx.Cast                  , 67
   onnx.Concat                , 362
   onnx.Constant              , 94
   onnx.Cos                   , 1
   onnx.Dim                   , 277
   onnx.Div                   , 64
   onnx.Equal                 , 8
   onnx.Expand                , 8
   onnx.Gather                , 1
   onnx.Greater               , 1
   onnx.MatMul                , 64
   onnx.Max                   , 32
   onnx.Mul                   , 322
   onnx.Neg                   , 64
   onnx.NoValue               , 1
   onnx.RMSLayerNormalization , 1
   onnx.Range                 , 6
   onnx.ReduceMeanV13         , 64
   onnx.Reshape               , 197
   onnx.ScatterND             , 1
   onnx.Sin                   , 1
   onnx.Slice                 , 197
   onnx.Sqrt                  , 160
   onnx.Squeeze               , 7
   onnx.Transpose             , 161
   onnx.Trilu                 , 1
   onnx.Unsqueeze             , 11
   onnx.Where                 , 7
  zhigh.Add                   , 64
  zhigh.Div                   , 32
  zhigh.MatMul                , 227
  zhigh.Mul                   , 127
  zhigh.Sigmoid               , 32
  zhigh.Softmax               , 32
  zhigh.Stick                 , 164
  zhigh.StickifiedConstant    , 386
  zhigh.Unstick               , 290
onnx-mlir: /build/llvm-project/mlir/lib/Dialect/Affine/IR/AffineOps.cpp:2980: static void mlir::affine::AffineLoadOp::build(mlir::OpBuilder&, mlir::OperationState&, mlir::Value, mlir::AffineMap, mlir::ValueRange): Assertion `map.getNumInputs() == mapOperands.size() && "inconsistent index info"' failed.

Obtaining the model:

Using the Hugging Face optimum-cli:
optimum-cli export onnx --model ibm-granite/granite-3b-code-instruct --framework pt --atol 0.001 --opset 17 granite-3b-code-instruct-17

[imaihal]

Has anyone started to look into this? I created the onnx model to start investigation.

[imaihal]

@cjvolzka @mikeessen This compilation failure should be solved by PR #2853 .

[mikeessen]

@cjvolzka @mikeessen This compilation failure should be solved by PR #2853 .

@imaihal Thank you!