Generate GraphQL schema during startup

[ogenstad]

This PR aims to fix a timing issue when regenerating the GraphQL schema after the branch schema has been updated. I don't completely know the exact conditions that triggers the issue.

It could be that two requests are coming in at the same time before the GraphQL schema has been generated for an updating branch and they step on each others toes, or we start to generate the schema before the SchemaBranch has been properly processed so that the registry for this branch appears to be empty.

After making these changes where we generate the GraphQL schema upon startup and also generate the new schema within the task of refreshing all the branches I haven't been able to reproduce the error any more. On the stable branch it was quite easy to do it before with the procedure described below.

Unfortunately in order to be able to regenerate the GraphQL schema within the registry refresh function I had to introduce a circular import. I'd like to remove that part and change the way the tasks for the scheduler are managed. However, I don't want to do that within a hotfix release. Instead I want to revert back to that within the develop branch.

[codspeed-hq]

CodSpeed Performance Report

Merging #5164 will not alter performance

_{Comparing pog-generate-graphql-schema (0d43df1) with stable (0ced4f7)}

Summary

✅ 10 untouched benchmarks

[ogenstad]

Against stable I can get the problem to appear fairly easily by having a script that looks like this:

import asyncio

from infrahub_sdk import InfrahubClient


async def fetch_tags(client: InfrahubClient, iteration: int):
    """Fetch tags and print the result."""
    tags = await client.all(kind="BuiltinTag")
    print(f"iteration: {iteration} - tags: {tags}")


async def main():
    client = InfrahubClient()
    iteration = 0
    while True:
        tasks = [fetch_tags(client, iteration) for _ in range(5)]
        await asyncio.gather(*tasks)
        # await fetch_tags(client=client, iteration=iteration)
        await asyncio.sleep(0.2)
        iteration += 1


if __name__ == "__main__":
    asyncio.run(main())

Then on a fresh installation without having started touched anything else I run this from the command line:

infrahubctl schema load models/base && python ../run.py

Using this script I've not been able to reproduce the issue against any of the commits in this PR. While running the test developed during the offsite I've not personally been able to reproduce the issue with either of the commits but @wvandeun has seen a few errors with the first commit. I think the best option would be to combine them however the second commit introduces a circular import that looks quite ugly now. That could perhaps work as a hotfix solution but for a long term option we could look at implementing the same type of import logic as we have with the prefect workflow to get around these type of circular imports.

The potential error would be something like this:

[{'message': "Unable to find the schema 'InfraDevice' in the registry", 'extensions': {'code': 422}}]