Skip to content

Commit

Permalink
Correct usage for remote_cache_dir in the docs
Browse files Browse the repository at this point in the history
  • Loading branch information
pankajkoti committed Aug 15, 2024
1 parent 6ca7d26 commit 78f4727
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 11 deletions.
11 changes: 6 additions & 5 deletions docs/configuration/caching.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,15 +35,16 @@ This feature is on by default. To turn it off, export the following environment

(Introduced in Cosmos 1.6 - Experimental feature)

Starting with Cosmos 1.6.0, users can also set a remote path to store this cache instead of using Airflow Variables.
To do so, you need to configure a remote cache path. See :ref:`remote_cache_dir` and :ref:`remote_cache_dir_conn_id` for
more information. This is an experimental feature introduced in 1.6.0 to gather user feedback. The ``remote_cache_dir``
will eventually be merged into the :ref:`cache_dir` setting in upcoming releases.
Starting with Cosmos 1.6.0, users can also set a remote directory path to store this cache instead of using Airflow
Variables. To do so, you need to configure a remote cache directory. See :ref:`remote_cache_dir` and
:ref:`remote_cache_dir_conn_id` for more information. This is an experimental feature introduced in 1.6.0 to gather
user feedback. The ``remote_cache_dir`` will eventually be merged into the :ref:`cache_dir` setting in upcoming
releases.

**How the cache is refreshed**

If using the default Variables cache approach, users can purge or delete the cache via Airflow UI by identifying and
deleting the cache key. In case you're using the alternative approach by setting the ``remote_cache_path`` introduced
deleting the cache key. In case you're using the alternative approach by setting the ``remote_cache_dir`` introduced
in Cosmos 1.6.0, you can delete the cache by removing the specific files by identifying them using your configured path
in the remote store.

Expand Down
12 changes: 6 additions & 6 deletions docs/configuration/cosmos-conf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,10 @@ This page lists all available Airflow configurations that affect ``astronomer-co
.. _remote_cache_dir:

`remote_cache_dir`_:
The remote path to store the dbt cache. Starting with Cosmos 1.6.0, you can store the `dbt ls` output as cache in a
remote location (an alternative to the Variable cache approach released previously since Cosmos 1.5.0)
using this configuration. The remote cache dir supports all schemes that are supported by the
`Airflow Object Store <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/objectstorage.html>`_
The remote directory to store the dbt cache. Starting with Cosmos 1.6.0, you can store the `dbt ls` output as cache
in a remote location (an alternative to the Variable cache approach released previously since Cosmos 1.5.0)
using this configuration. The value for the remote cache directory can be any of the schemes that are supported by
the `Airflow Object Store <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/objectstorage.html>`_
feature introduced in Airflow 2.8.0 (e.g. ``s3://your_s3_bucket/cache_dir/``, ``gs://your_gs_bucket/cache_dir/``,
``abfs://your_azure_container/cache_dir``, etc.)

Expand All @@ -113,8 +113,8 @@ This page lists all available Airflow configurations that affect ``astronomer-co
.. _remote_cache_dir_conn_id:

`remote_cache_dir_conn_id`_:
The connection ID for the remote cache path. If this is not set, the default Airflow connection ID identified for
the scheme will be used.
The connection ID for the remote cache directory. If this is not set, the default Airflow connection ID identified
for the scheme will be used.

- Default: ``None``
- Environment Variable: ``AIRFLOW__COSMOS__REMOTE_CACHE_DIR_CONN_ID``
Expand Down

0 comments on commit 78f4727

Please sign in to comment.