Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix grant loader coeus SQL #115

Merged
merged 3 commits into from
May 24, 2024
Merged

Fix grant loader coeus SQL #115

merged 3 commits into from
May 24, 2024

Conversation

rpoet-jh
Copy link
Contributor

While working on updates to the JHU grant database SQL for Fibi, I realized an issue with the current SQL executing against Coeus. Luckily the impact is small.

The current SQL selects all records for a grant if one of the records has been updated if the last update timestamp is greater than the supplied filter timestamp. The issue was the set of grant records returned need to be only those with a Proposal Status of Funded. But with the current SQL, it was possible that grant records with Proposal Status of some other value would return too.

I fixed this issue by adding the AND A.PROPOSAL_STATUS = 'Funded' to the main SQL so only Funded grant records are returned.

The changes related to checking GRANT_NUMBER IS NOT NULL are also needed to ensure we don't get any records with NULL grant number. Turned out I had wrong the alias inside the EXISTS statement, but that actually works out in the end. But I changed the SQL to check in the EXISTS on the EXISTS alias and the outer query view as well to be complete and more understandable.

I did analysis since this SQL was deployed on 2024-03-28, and luckily there are only three records that were returned with a non-Funded Proposal Status for two grants. There were no records returned with a NULL GRANT_NUMBER. When this change is deployed in PROD, I will backdate to 2024-03-28 and run the grant loader to clean up the two grants.

@rpoet-jh rpoet-jh requested a review from markpatton May 24, 2024 14:46
@rpoet-jh rpoet-jh self-assigned this May 24, 2024
Copy link
Contributor

@markpatton markpatton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. This should get fixed.

@rpoet-jh
Copy link
Contributor Author

@markpatton I noticed that adding this to the outer query is needed to ensure we don't return grant records with award end dates older than 2011-01-01. 833e9f2

I did check in PROD, and there are no records that returned since 2024-03-28 with award end dates older than 2011-01-01, so there is no data impact. But this is good to add going forward.

@rpoet-jh rpoet-jh merged commit 33fffba into main May 24, 2024
2 checks passed
@rpoet-jh rpoet-jh deleted the russ-grant-loader-sql-cleanup branch August 1, 2024 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants