feature: provide a way to use pagination concurrently to retrieve objects using the SDK

[wvandeun]

Component

Python SDK

Describe the Feature Request

When you execute a query that retrieves a large amount of nodes from the db, using the filters or all method, then the SDK will leverage pagination to break the query into smaller pages.

The retrieval of this pages happens in a serial way, which is ideal. We can do this better and faster if we retrieve the pages concurrently.

Pseud code of what it could look like

from infrahub_sdk import InfrahubClient

client = InfrahubClient()
resp = await client.get("query { LocationSuite { count }}")

count = int(resp["LocationSuite"]["count"])

batch = await client.create_batch()

num = 1
page_size=50
while num < count:
    batch.add(task=client.all, kind="nodekind", offset=num, limit=page_size)
    num += page_size

Describe the Use Case

Retrieval of a large number of nodes using a GraphQL query is taking some time, since we are retrieving the pages one by one in a serial way. Being able to get the pages in a concurrent way should improve the speed.

Additional Information

No response