Skip to content

Commit

Permalink
Minor fixes (microsoft#18949)
Browse files Browse the repository at this point in the history
### Description
<!-- Describe your changes. -->



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
  • Loading branch information
pengwa authored Dec 28, 2023
1 parent 3bbe4fe commit 998517b
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,6 @@ py::object finalize_training_mode_forward(
}

if (kernel_info.is_first_run) {
std::cout << "666666666666666666666666. py_fn->materialize_grads:" << py_fn->materialize_grads << std::endl;
get_materialize_grads_once(forward_output_tensors, py_fn->materialize_grads, kernel_info);

if (kernel_info.safe_run_enabled) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,10 @@ def _summarize_tensor(
min_buckets[i] = bucket.min()
max_buckets[i] = bucket.max()
mean_buckets[i] = bucket.sum()
std_buckets[i] = bucket.std()

# Only calculate std for float types, otherwise it will throw exception.
if bucket.dtype in [torch.float16, torch.float32, torch.float64]:
std_buckets[i] = bucket.std()

# Reduction across all buckets
num_nan = nan_buckets.sum()
Expand Down

0 comments on commit 998517b

Please sign in to comment.