perf(ScheduleController): Fetch schedules by individual stop for more cache hits #231
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KaylaBrady
added
the
deploy to dev-orange
KaylaBrady
force-pushed
the
kb-cacheable-scheds
branch
from
e0c279a
to
636c01d
Compare
… cache hits As implemented, if any request for a stop fails, the entire request fails. This is to preserve the existing all-or-nothing behavior for fetching schedules for all stops together. If we want to have partial failure states in the future, we'll want to adopt a frontend changes to represent the partial failure state.
KaylaBrady
force-pushed
the
kb-cacheable-scheds
branch
from
53d580f
to
66452db
Compare
KaylaBrady
had a problem deploying
to
dev-orange
GitHub Actions
Failure
KaylaBrady
force-pushed
the
kb-cacheable-scheds
branch
from
66452db
to
f1cb85e
Compare
KaylaBrady
had a problem deploying
to
dev-orange
GitHub Actions
Failure
KaylaBrady
commented
Nov 7, 2024
Comment on lines
+24
to
+25
| gc_interval: :timer.hours(2), | ||
| allocated_memory: 250_000_000 |
There was a problem hiding this comment.
Setting to 2 hours based on the recommendation here that ttl should be less than gc_interval
KaylaBrady
commented
Nov 7, 2024
lib/mbta_v3_api/repository.ex
Outdated
| @impl true | ||
| def alerts(params, opts \\ []), do: all(MBTAV3API.Alert, params, opts) | ||
|
|
||
| @impl true | ||
| @decorate cacheable(cache: MBTAV3API.RepositoryCache, on_error: :nothing, opts: [ttl: @ttl]) |
There was a problem hiding this comment.
pulled from dotcom
boringcactus
reviewed
Nov 7, 2024
Comment on lines
75
to
77
| |> Task.async_stream(fn filter_params -> | ||
| {filter_params, fetch_schedules(filter_params)} | ||
| end) |
There was a problem hiding this comment.
Would it make sense to call this with ordered: false?
boringcactus
approved these changes
Nov 8, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Ticket: Caching schedules data [backend only]
What is this PR for?
As implemented, if any request for a stop fails, the entire request fails. This is to preserve the existing all-or-nothing behavior for fetching schedules for all stops together.
During load testing (notes), there were still big discrepancies between the duration logged for a request from the V3 API vs the duration logged to receive on our backend (602ms vs 4.99s). This PR addresses that problem in the lowest effort way: Adding TTL-based caching to the other static endpoints too to limit the number of concurrent requests to the API.
With these changes (and load testing script changes from #236), there were a handful of failures when fetching schedules due to timeouts, which still represent a low % of overall requests. There is more optimization we can do here, but this PR feels like a meaningful enough improvement on its own.
