Wrong results for grad of grad.

[wangg12]

Following this doc pagehttps://minpy.readthedocs.io/en/latest/tutorial/autograd_tutorial.html, running the code

import minpy.numpy as np  # currently need import this at the same time
from minpy.core import grad

def foo(x):
    return x**2
d_foo = grad(foo)
d_2_foo = grad(d_foo)
d_3_foo = grad(d_foo)
x = np.linspace(-10, 10, 200)
# d_foo(x)
d_2_foo(x)
d_3_foo(x)

The first order gradient is right, but the second and the third order gradients are not right. My results for them are both 0.0(a single float number).

However, from the definition of foo, d_2_foo(x) should be an array of constants 2, and d_3_foo(x) should be an array of 0.

If my understanding is right, there must be some bug in the grad of grad mechanism, otherwise I would get my desired results.

[lryta]

The bug is confirmed. It is introduced by our latest performance refactoring. The fix will come soon.

[jermainewang]

@lryta do you know what causes the bug?

[lryta]

We added tape.start_recording and tape.end_recording in minpy.core. If two grad functions are stacked, tape will end recording earlier than expected.

[jermainewang]

I see. But the two tapes should be separated right? I remember the code is current_tape.end_recording.

[lryta]

I think current_tape is global. Im not 100% sure. I will check later.