Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix errors due to recursive calling of HandleTopLevelDecl #897

Merged
merged 1 commit into from
May 20, 2024

Conversation

vaithak
Copy link
Collaborator

@vaithak vaithak commented May 17, 2024

  • If the definition of the custom derivative is not found in the current translation unit and is linked from another,
    Adding it to the set of derivatives ensures that the custom derivative is not differentiated again using numerical differentiation due to an unavailable definition.
  • There can be cases where m_Multiplexer is not provided. Hence, we don't delay HandleTranslationUnit at the end and it is called repeatedly. This resulted in HandleTopLevelDecl being called recursively (from PerformPendingInstantiations). This PR adds conditional checks to ensure this doesn't perturb the execution of the differentiation plan.

Fixes: #890

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

Copy link

codecov bot commented May 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.15%. Comparing base (d1fec23) to head (a642784).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #897      +/-   ##
==========================================
- Coverage   94.82%   94.15%   -0.67%     
==========================================
  Files          52       53       +1     
  Lines        7708     7803      +95     
==========================================
+ Hits         7309     7347      +38     
- Misses        399      456      +57     
Files Coverage Δ
include/clad/Differentiator/DerivativeBuilder.h 100.00% <ø> (ø)
include/clad/Differentiator/DerivedFnCollector.h 100.00% <ø> (ø)
include/clad/Differentiator/DynamicGraph.h 98.07% <100.00%> (+0.03%) ⬆️
lib/Differentiator/DerivativeBuilder.cpp 99.01% <100.00%> (+0.01%) ⬆️
lib/Differentiator/DerivedFnCollector.cpp 100.00% <100.00%> (ø)
tools/ClangPlugin.cpp 95.51% <100.00%> (+0.01%) ⬆️
tools/ClangPlugin.h 92.64% <100.00%> (+0.67%) ⬆️

... and 2 files with indirect coverage changes

Files Coverage Δ
include/clad/Differentiator/DerivativeBuilder.h 100.00% <ø> (ø)
include/clad/Differentiator/DerivedFnCollector.h 100.00% <ø> (ø)
include/clad/Differentiator/DynamicGraph.h 98.07% <100.00%> (+0.03%) ⬆️
lib/Differentiator/DerivativeBuilder.cpp 99.01% <100.00%> (+0.01%) ⬆️
lib/Differentiator/DerivedFnCollector.cpp 100.00% <100.00%> (ø)
tools/ClangPlugin.cpp 95.51% <100.00%> (+0.01%) ⬆️
tools/ClangPlugin.h 92.64% <100.00%> (+0.67%) ⬆️

... and 2 files with indirect coverage changes

@vaithak
Copy link
Collaborator Author

vaithak commented May 17, 2024

To test this locally with Clang, we can declare a custom derivative (for a called function) but cannot offer its definition. Now, computing the hessian should result in a linker error instead of a compile-time error.
A similar scenario was observed in the cling and ROOT setup, where the definition of custom derivative was not found in some cases.

Minimal code:

#include "clad/Differentiator/Differentiator.h"

namespace clad{
namespace custom_derivatives {
  clad::ValueAndPushforward<double, double> f_pushforward(double x, double _d_x);
}
}

double f(double x) {
  return x*x;
}

double wrapper(double x) {
  return f(x);
}

int main() {
  clad::hessian(wrapper);
  return 0;
}

@vgvassilev
Copy link
Owner

Can we add this test to the PR?

@vaithak
Copy link
Collaborator Author

vaithak commented May 18, 2024

As mentioned above, the crude way to test this locally was seeing a linker error instead of a compile time error because of numerical differentiation. I don't think it would be good to have something like that in the CI check.

We can test this with ROOT + Cling infrastructure manually though.

@parth-07
Copy link
Collaborator

@vaithak We can test this by adding multiple files for the test, right? For example, we can have two files, CustomDerivativesDecl.C and CustomDerivativesDef.cpp. CustomDerivativesDecl.C will have declarations of custom derivatives and CustomDerivativesDef.cpp will have the definitions of custom derivatives. CustomDerivativesDecl.C will contain the actual test as well and it will utilize CustomDerivativesDef.cpp in the compile commands of the test.

@vgvassilev
Copy link
Owner

@vaithak We can test this by adding multiple files for the test, right? For example, we can have two files, CustomDerivativesDecl.C and CustomDerivativesDef.cpp. CustomDerivativesDecl.C will have declarations of custom derivatives and CustomDerivativesDef.cpp will have the definitions of custom derivatives. CustomDerivativesDecl.C will contain the actual test as well and it will utilize CustomDerivativesDef.cpp in the compile commands of the test.

That would be probably better in the unittests.

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@vaithak vaithak changed the title Add custom derivative to DerivativeSet Fix errors due to recursive calling of HandleTopLevelDecl May 19, 2024
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

lib/Differentiator/DerivativeBuilder.cpp Outdated Show resolved Hide resolved
tools/ClangPlugin.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@vgvassilev
Copy link
Owner

@vaithak, is it worth adding the test from #890 here?

@vaithak vaithak linked an issue May 20, 2024 that may be closed by this pull request
@vaithak
Copy link
Collaborator Author

vaithak commented May 20, 2024

@vaithak, is it worth adding the test from #890 here?

Added tests 👍🏼

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@vgvassilev
Copy link
Owner

I am wondering why the coverage was decreased when we added a test… quite strange. Can you fix the failures and squash into a single commit?

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@vaithak
Copy link
Collaborator Author

vaithak commented May 20, 2024

I am wondering why the coverage was decreased when we added a test… quite strange. Can you fix the failures and squash into a single commit?

I need to fix Kokkos test, not sure what is happening there.
For codecov, the lines we added were already covered (although not in the way we wanted).
I guess adding more lines only increased total lines without changing the total covered lines, hence decreasing the percent by a bit.
This is just a guess though :)

@vgvassilev
Copy link
Owner

I am wondering why the coverage was decreased when we added a test… quite strange. Can you fix the failures and squash into a single commit?

I need to fix Kokkos test, not sure what is happening there. For codecov, the lines we added were already covered (although not in the way we wanted). I guess adding more lines only increased total lines without changing the total covered lines, hence decreasing the percent by a bit. This is just a guess though :)

The kokkos failure looks like concurrency failure. We built this file before building clad.so?

- Add custom derivative to DerivativeSet: This is required if the definition of the custom derivative is not found in the current translation unit and is linked in from another.
Adding it to the set of derivatives ensures that the custom derivative is not differentiated again using numerical differentiation due to an unavailable definition.

- Fix recursive processing of DiffRequests: There can be cases where `m_Multiplexer` is not provided. Hence, we don't delay HandleTranslationUnit at the end and it is called repeatedly. This resulted in HandleTopLevelDecl being called recursively (from PerformPendingInstantiations). This commit adds conditional checks to ensure this doesn't perturb the execution of the differentiation plan.
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@vgvassilev vgvassilev merged commit 19c6573 into vgvassilev:master May 20, 2024
88 of 89 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Regression in support of functions with unknown definition
3 participants