CAPT 983/slow loading payroll #3250
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rjlynch
force-pushed
the
CAPT-983/slow-loading-payroll
branch
6 times, most recently
from
7253886 to
479469c
Compare
rjlynch
commented
Oct 4, 2024
rjlynch
force-pushed
the
CAPT-983/slow-loading-payroll
branch
2 times, most recently
from
257cbc7 to
86ac927
Compare
rjlynch
force-pushed
the
CAPT-983/slow-loading-payroll
branch
from
86ac927 to
7958a55
Compare
slorek
approved these changes
Nov 8, 2024
| @@ -33,7 +33,7 @@ def create | |||
|
|
|||
| # NOTE: Optimisation - preload payments, claims and eligibility | |||
There was a problem hiding this comment.
Suggested change
| # NOTE: Optimisation - preload payments, claims and eligibility |
app/models/base_policy.rb
Outdated
| @@ -58,4 +58,12 @@ def further_education_payments? | |||
| def auto_check_student_loan_plan_task? | |||
| false | |||
| end | |||
|
|
|||
| def award_amount_column | |||
| if to_s == "StudentLoans" | |||
There was a problem hiding this comment.
Perhaps the StudentLoans model should override this method instead
There was a problem hiding this comment.
Yeah that's nicer, have fixed 👍
Rather than looping through all payments each time we can just loop through the payments for the provided policy.
Makes understanding what's going on easier
@Items doesn't need to be an instance variable
Doesn't speed things up as we're hitting the AR cache but cleans up the noise in the logs
We hit this line items method multiple times with the same policy, cache the results.
On my test data set this takes the page load from 7s to ~500ms
Refactors the CTE to build a generic eligibilities table we can join to. Now when we add a new policy the payroll query should automatically pick it up. Ideally we'd rename the "student_loan_repayment_amount" column in the DB but that's quite involved, so for the time being we define an 'award_amount_column' column on the policy which returns the name of the DB column we need to check to get the award amount.
Adds a new scope to claim `with_award_amount` allowing for querying claims in the db by their award amount.
rjlynch
force-pushed
the
CAPT-983/slow-loading-payroll
branch
from
7958a55 to
24dcf92
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Speed up payroll run queries.
In order to shift more of the payroll query work to the database we need to be able to join to the various eligibility tables. To make this trickier the column name for
award_amountis different on some policy's eligibilities (ie student_loans). This PR introduces a new claim scopewith_award_amountthat handles joining claims to the eligibilities tables and picking the correct column for the award amount.Of note this PR makes the following assumptions (which are also made on master but it's worth calling them out)
Benchmarks
Hitting the payroll index page
Master
:view_runtime=>6573.04, :db_runtime=>2113.22, :allocations=>13379102This branch
:view_runtime=>248.16, :db_runtime=>517.15, :allocations=>481363Payroll show for Nov 23, with 3k claims
Master
:view_runtime=>4479.93, :db_runtime=>325.29, :allocations=>10520859This branch
:view_runtime=>189.02, :db_runtime=>187.99, :allocations=>311803