Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with GRU simulation #20535

Open
kokhazade opened this issue Nov 22, 2024 · 0 comments
Open

Problem with GRU simulation #20535

kokhazade opened this issue Nov 22, 2024 · 0 comments
Assignees

Comments

@kokhazade
Copy link

Hello,

I am going to implement the GRU computations in my code and compare the results with the output of a GRU layer. To this end, I am using a same input for both cases and get the weights of GRU and pass it to my_GRU_simulation function. By using the following code for inner gates calculations, I get two different outputs for these two cases.
Is there any extra calculations (rather than following) that I have missed?
How can I create the exactly same output as a GRU layer?

        import tensorflow as tf
        from keras import backend

        GRU_layer_input = tf.squeeze(GRU_layer_input, axis=1)
        matrix_x = backend.dot(tf.convert_to_tensor(GRU_layer_input), tf.convert_to_tensor(gru_layer_weights[0]))
        matrix_x = backend.bias_add(matrix_x, gru_layer_weights[2][0])
        x_z, x_r, x_h = tf.split(matrix_x, 3, axis=1)

        matrix_y = backend.dot(tf.convert_to_tensor(previous_gru_output), tf.convert_to_tensor(gru_layer_weights[1]))
        matrix_y = backend.bias_add(matrix_y, gru_layer_weights[2][1])
        recurrent_z, recurrent_r, recurrent_h = tf.split(matrix_y, 3, axis=1)

        z = tf.sigmoid(x_z + recurrent_z)
        r = tf.sigmoid(x_r + recurrent_r)
        hh = tf.tanh(x_h + r * recurrent_h)

        h = z * h_tm1 + (1 - z) * hh
        new_state = [h] if tf.nest.is_nested(previous_gru_output) else h

previous_gru_output is all zero with GRU units size.
I am using tensorflow and keras version 2.11.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants