Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a single point to process non-differentiable functions in the reverse mode #1161

Merged
merged 1 commit into from
Dec 3, 2024

Conversation

PetroZarytskyi
Copy link
Collaborator

Right now, we have a bunch of reasons to consider a function non-differentiable and not produce a pullback. In all cases, we clone the call and return early in different ways, almost all of which have flaws. For example:

Return on line 1488: it's not necessary to pass 0 as the adjoint of non-differentiable entities in the reverse mode. Also, cloning arguments is dangerous for multiple reasons (e.g. sometimes we introduce operators, brackets, etc. which are not generated when cloning) and doesn't account for their possible differentiable side effects.

Return on line 1520: Again, cloning is dangerous.

Return on line 1642: Cloning again. Also, providing the cloned call as the adjoint doesn't make much sense.

Return on line 1664: dfdx() of the call shouldn't be passed to its arguments.

This PR replaces all of these cases to the same place and fixes the mentioned issues.

@PetroZarytskyi PetroZarytskyi marked this pull request as ready for review December 3, 2024 16:40
Copy link
Owner

@vgvassilev vgvassilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link

codecov bot commented Dec 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.55%. Comparing base (6228385) to head (30c72ae).
Report is 1 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1161      +/-   ##
==========================================
- Coverage   94.55%   94.55%   -0.01%     
==========================================
  Files          51       51              
  Lines        8933     8923      -10     
==========================================
- Hits         8447     8437      -10     
  Misses        486      486              
Files with missing lines Coverage Δ
lib/Differentiator/ReverseModeVisitor.cpp 95.56% <100.00%> (-0.02%) ⬇️
Files with missing lines Coverage Δ
lib/Differentiator/ReverseModeVisitor.cpp 95.56% <100.00%> (-0.02%) ⬇️

@vgvassilev vgvassilev merged commit 65330b5 into vgvassilev:master Dec 3, 2024
89 of 90 checks passed
@PetroZarytskyi PetroZarytskyi deleted the no-diff-pull branch December 3, 2024 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants