Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintainer page optimization #122

Open
4 tasks
AMDmi3 opened this issue Nov 25, 2024 · 0 comments
Open
4 tasks

Maintainer page optimization #122

AMDmi3 opened this issue Nov 25, 2024 · 0 comments
Labels
Component: webapp repology-webapp, repology web application Effort: intermediate A class or a huge function needs to be written Priority: someday Normal priority Topic: performance Both optimizations and perf monitoring Type: refactoring Internal improvements which do not change behavior

Comments

@AMDmi3
Copy link
Member

AMDmi3 commented Nov 25, 2024

P99 is above 1sec, should be optimized.

  • Parallel fetch of projects and similar maintainers
  • Measure projects and similar maintainers query time

It's already known though that similar maintainers query is rather slow since it needs to process a lot of maintainer_metapackages (to get all maintainers with intersecting projects list), then fetch maintainer info for all of these.

  • May try to optimize it by passing numeric maintainer id to the query to avoid InitPlans and possibly make it easier for an optimizer
  • Unwrap subqueries into CTEs (should have no effect, but improve query readability)

Real optimization would need prepared data, such as maintainer_id/project_id/num_maintainer_projects table which is smaller and does not require join with maintainers, or similar maintainers percomputed totally offline.

For now, reducing maintainer_metapackages should also help, such as by dropping unneeded maintainers (#121), and giving more memory to the database, as maintainer_metapackages is quite hot and should be well cached.

@AMDmi3 AMDmi3 added Component: webapp repology-webapp, repology web application Effort: intermediate A class or a huge function needs to be written Priority: someday Normal priority Type: refactoring Internal improvements which do not change behavior Topic: performance Both optimizations and perf monitoring labels Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: webapp repology-webapp, repology web application Effort: intermediate A class or a huge function needs to be written Priority: someday Normal priority Topic: performance Both optimizations and perf monitoring Type: refactoring Internal improvements which do not change behavior
Projects
None yet
Development

No branches or pull requests

1 participant