Skip to content

Commit

Permalink
Docs: Add important note on using iterall and iterdict (aiidateam…
Browse files Browse the repository at this point in the history
…#6126)

Using the `all` and `dict` equivalents are very inefficient for large
query results and will lead to performance problems.
  • Loading branch information
sphuber authored Sep 21, 2023
1 parent a42e09c commit 0aea7e4
Showing 1 changed file with 11 additions and 2 deletions.
13 changes: 11 additions & 2 deletions docs/source/howto/query.rst
Original file line number Diff line number Diff line change
Expand Up @@ -74,12 +74,15 @@ There are several ways to obtain data from a query:
all_results_l = qb.all() # Returns a list of lists
In case you are working with a large dataset, you can also return your query as a generator:
.. tip::

If your query only has a single projection, use ``flat=True`` in the ``first`` and ``all`` methods to return a single value or a flat list, respectively.

You can also return your query as a generator:

.. code-block:: python
all_res_d_gen = qb.iterdict() # Return a generator of dictionaries
# of all results
all_res_l_gen = qb.iterall() # Returns a generator of lists
This will retrieve the data in batches, and you can start working with the data before the query has completely finished.
Expand All @@ -90,6 +93,12 @@ For example, you can iterate over the results of your query in a for loop:
for entry in qb.iterall():
# do something with a single entry in the query result
.. important::

When looping over the result of a query, use the ``iterall`` (or ``iterdict``) generator instead of ``all`` (or ``dict``).
This avoids loading the entire query result into memory, and it also delays committing changes made to AiiDA objects inside the loop until the end of the loop is reached.


.. _how-to:query:filters:

Filters
Expand Down

0 comments on commit 0aea7e4

Please sign in to comment.