Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Trivia list loading #6336

Merged
merged 2 commits into from
Oct 29, 2024

Conversation

Jackenmen
Copy link
Member

Description of the changes

To support listing descriptions in [p]trivia list, there's a need for further optimizations in the list loading process.

Optimizations made:

  • Updated yaml.safe_load() calls to use of yaml.CSafeLoader (C implementation of the parser) when possible
  • Added a flag for disabling schema validation when it is not strictly needed

The latter change is supposed to help us support retrieval of the actually needed fields without having to validate schema of the whole file, for example:

def _get_list_description(path: pathlib.Path) -> str:
    """
    Returns a trivia list description from the given path.

    Raises
    ------
    InvalidListError
        Parsing of list's YAML file failed.
    """
    trivia_dict = get_list(path, validate_schema=False)
    return str(trivia_dict.get("DESCRIPTION", ""))

@Kreusada here's an example of how we can optimally get trivia descriptions for [p]trivia list output (10 per page in this example + cache for previously generated pages) when you work on it:

import pathlib
from typing import Dict, List

from redbot.vendored.discord.ext import menus as dpy_menus
from redbot.core.utils import views
from redbot.core.utils.chat_formatting import bold, italics


class _TriviasSource(dpy_menus.ListPageSource):
    def __init__(self, items: List[pathlib.Path]):
        super().__init__(items, per_page=10)
        self._cache: Dict[int, str] = {}

    async def get_page(self, page_number: int) -> str:
        cached = self._cache.get(page_number)
        if cached is not None:
            return cached

        categories = await super().get_page(page_number)
        entries = []
        for path in categories:
            description = _get_list_description(path)
            if len(description) > 100:
                description = f"{description[:99]}\N{HORIZONTAL ELLIPSIS}"
            elif not description:
                description = italics(_("No description provided for this category."))

            entries.append(f"- {bold(path.stem)} - {description}")

        ret = "\n".join(entries)
        self._cache[page_number] = ret
        return ret

    async def format_page(
        self, view: discord.ui.View, page: views._ACCEPTABLE_PAGE_TYPES
    ) -> views._ACCEPTABLE_PAGE_TYPES:
        return page


# ...

# in `[p]trivia list` implementation:
items = self._all_lists()
menu = views.SimpleMenu(items)
menu._source = _TriviasSource(items)
await menu.start(ctx)

Have the changes in this PR been tested?

Yes

Optimizations made:
- Usage of yaml.CSafeLoader when possible
- Schema validation skipped when not strictly needed
@Jackenmen Jackenmen added the Type: Optimisation Situations where too much time is necessary to complete the task. label Mar 31, 2024
@Jackenmen Jackenmen added this to the 3.5.9 milestone Mar 31, 2024
@Jackenmen Jackenmen requested a review from Kreusada March 31, 2024 01:34
@github-actions github-actions bot added the Category: Cogs - Trivia This is related to the Trivia cog. label Mar 31, 2024
@Kreusada Kreusada self-assigned this Apr 7, 2024
@Jackenmen Jackenmen modified the milestones: 3.5.9, 3.5.10 Apr 20, 2024
Copy link
Member

@Kreusada Kreusada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Kreusada Kreusada merged commit 4134881 into Cog-Creators:V3/develop Oct 29, 2024
17 checks passed
@red-githubbot red-githubbot bot added the Changelog Entry: Pending Changelog entry for this PR hasn't been added by repo maintainers yet. label Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Category: Cogs - Trivia This is related to the Trivia cog. Changelog Entry: Pending Changelog entry for this PR hasn't been added by repo maintainers yet. Type: Optimisation Situations where too much time is necessary to complete the task.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants