[DRAFT][compiler] Introduce FuseGRUPass

[BalyshevArtem]

This pr introduces FuseGRUPass for fusing gru pattern into single CircleGRU op.

for issues: #12263 and for #13365

ONE-DCO-1.0-Signed-off-by: Artem Balyshev [email protected]

[chunseoklee]

GRU_fusegru.zip

  import tensorflow as tf
  from tensorflow import keras
  from tensorflow.keras import regularizers
  import numpy as np

  normalization_layer = tf.keras.layers.Normalization()
  adapt_data = np.array([[0., 7., 4. , 0.5],
                         [2., 9., 6. , -0.5],
                         [0., 7., 4. , -0.5],
                         [2., 9., 6. , 0.5]], dtype='float32')
  #normalization_layer.adapt(adapt_data)
  classes = 4
  activation = 'tanh'
  model = tf.keras.models.Sequential([
      tf.keras.Input(shape=(10,4)),
      normalization_layer,
      tf.keras.layers.GRU(units=20, activation=activation),
      tf.keras.layers.Dense(classes, activation='softmax')
  ])

  model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001))

  model.summary()

  run_model = tf.function(lambda x: model(x))

  # This is important, let's fix the input size.
  BATCH_SIZE = 1
  X = 10
  Y = 4
  concrete_func = run_model.get_concrete_function(
      tf.TensorSpec([BATCH_SIZE, X,Y], model.inputs[0].dtype))

  # model directory.
  MODEL_DIR = "keras_model"
  model.save(MODEL_DIR, save_format="tf", signatures=concrete_func)

  converter = tf.lite.TFLiteConverter.from_saved_model(MODEL_DIR)
  converter.experimental_new_converter = True
  converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,
                                         ]
  #converter.optimizations = [tf.lite.Optimize.DEFAULT]
  converted_model = converter.convert()
  save_to = "GRU.tflite"
  if save_to is not None:
      with open(save_to, 'wb') as tf_lite_file:
          tf_lite_file.write(converted_model)

@BalyshevArtem After applying this pass to gru model, while_cond / while_body subgraph are still alive in resulting graph.

[BalyshevArtem]

@BalyshevArtem After applying this pass to gru model, while_cond / while_body subgraph are still alive in resulting graph.

Hmmm
For me it is worked (I used your python script to obtain model and then convert it into circle and then made fusing)

[BalyshevArtem]

@chunseoklee, I checked your models from GRU_fusegru.zip.
I did not immediately understand what you mean, but now I understand, yes, this pass does not remove additional subgraphs, a new pass is needed for this task (check #12319 (comment)), only fusing takes place here (which works as you can see in above screen).

[BalyshevArtem]

@chunseoklee, I have added the part that is responsible for removing dead graphs, please check :)

[seanshpark]

this is logically invalid case but we don't need to throw to stop the program.
just return false would be OK.

[BalyshevArtem]

Done

[chunseoklee]

Those two bias tensor are not well generated in the fused GRU.

-> since original model's FC does not have bias.

[BalyshevArtem]

-> since original model's FC does not have bias.

Yes, that is the reason, they are empty is FC doesn't have bias

[seanshpark]

To check pass(es) is(are) working correctly,

• add one more more tflite recipe(s) with decomposed GRU
• add to `compiler/circle2circle-dredd-recipe-test/test.lst
• add to compiler/luci-pass-value-py-test/test.lst
• update necessary modules

[BalyshevArtem]

To check pass(es) is(are) working correctly,

* add one more more tflite recipe(s) with decomposed GRU

* add to `compiler/circle2circle-dredd-recipe-test/test.lst

* add to `compiler/luci-pass-value-py-test/test.lst`

* update necessary modules

I added tflite decomposed GRU recipe and circle2circle-dredd-recipe-test.
For luci-pass-value-py-test we need support this op in the luci-interpreter, right? Can we postpone the supporting of this test for later, after completing the main work on supporting this kernel in onert-micro and its training?

[seanshpark]

For luci-pass-value-py-test we need support this op in the luci-interpreter, right?

Yes.

Can we postpone the supporting of this test for later, after completing the main work on supporting this kernel in onert-micro and its training?

How can you be certain without value check that the conversion pass is working correctly?
I think it's up to you for this. Anyway if you don't add the test, no one will.

@chunseoklee , I'd like to hear your opinion.

[BalyshevArtem]

How can you be certain without value check that the conversion pass is working correctly?

I checked manually the result for onert-micro. After finishing with onert-micro supporting we will ad it into luci-interpreter :)

[chunseoklee]

@chunseoklee , I'd like to hear your opinion.

I checked manually the result for onert-micro. After finishing with onert-micro supporting we will ad it into luci-interpreter :)

AFAIU, GRU kernel will be added into luci-interpreter after a while. Am I right ?

[seanshpark]

we will ad it into luci-interpreter :)

OK, adding a issue for this task will do for now.