Silently disable default saved tensor hooks during tracing (#123196)

Summary:
FIXES #113263. Same idea as in pytorch/pytorch#113417, but we need a more intrusive C API to silently nop default saved tensor hooks, in order to support user-code that use torch.autograd.disable_saved_tensors_hooks (see test_unpack_hooks_can_be_disabled). We mock the output of get_hooks while leaving push/pop untouched.

For compiled autograd, we're firing pack hooks once and unpack hooks twice right now, I'll look into this separately from this issue.

X-link: pytorch/pytorch#123196
Approved by: https://github.com/soulitzer

Reviewed By: izaitsevfb

Differential Revision: D58628125

Pulled By: xmfan

fbshipit-source-id: 399cba07d50610ba52da55951f7b2200eb9c217f

userbenchmark/dynamo/dynamobench/_dynamo/utils.py

@@ -2773,3 +2773,13 @@ def strip_color_from_string(text):
     # This regular expression matches ANSI escape codes
     ansi_escape = re.compile(r"\x1B[@-_][0-?]*[ -/]*[@-~]")
     return ansi_escape.sub("", text)
+
+
+@contextlib.contextmanager
+def _disable_saved_tensors_hooks_during_tracing():
+    # See NOTE: [Deferring tensor pack/unpack hooks until runtime]
+    try:
+        prior = torch._C._autograd._saved_tensors_hooks_set_tracing(True)
+        yield
+    finally:
+        torch._C._autograd._saved_tensors_hooks_set_tracing(prior)