Skip to content

Commit

Permalink
Fix dev/Dockerfile + Add uv pip install for faster build time (#997)
Browse files Browse the repository at this point in the history
## Fix dev Dockerfile

The Dockerfile was not building because `dbt-postgres==1.8.0` pinned
`psycopg2`.

To resolve this, you need to apt-get install `build-essential` and
`libpq-dev`. I added a `RUN` directive that does that. This is placed
near the top as it makes the most sense for layering purposes.

## Add uv

While I was at it, I added uv: https://github.com/astral-sh/uv tldr:
this makes the Dockerfile build a little faster. This speed is
especially important due to how the Dockerfile will typically be used:
if a user makes a change to the `cosmos/` directory and then wants to
test their code, due to how the layers are stacked, the pip install will
need to be re-run and resolved on over 100 Python packages on each code
change.
  • Loading branch information
dwreeves authored and tatiana committed Jun 6, 2024
1 parent df2b3ac commit 14291a3
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion dev/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,22 @@ FROM quay.io/astronomer/astro-runtime:11.3.0-base

USER root


# dbt-postgres 1.8.0 requires building psycopg2 from source
RUN /bin/sh -c set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends build-essential libpq-dev; \
apt-get clean; \
rm -rf /var/lib/apt/lists/*

RUN pip install -U uv

COPY ./pyproject.toml ${AIRFLOW_HOME}/astronomer_cosmos/
COPY ./README.rst ${AIRFLOW_HOME}/astronomer_cosmos/
COPY ./cosmos/ ${AIRFLOW_HOME}/astronomer_cosmos/cosmos/

# install the package in editable mode
RUN pip install -e "${AIRFLOW_HOME}/astronomer_cosmos"[dbt-postgres,dbt-databricks]
RUN uv pip install --system -e "${AIRFLOW_HOME}/astronomer_cosmos"[dbt-postgres,dbt-databricks]

# make sure astro user owns the package
RUN chown -R astro:astro ${AIRFLOW_HOME}/astronomer_cosmos
Expand Down

0 comments on commit 14291a3

Please sign in to comment.