Skip to content

Commit

Permalink
Install rearc requirements at build time instead of launch time... th…
Browse files Browse the repository at this point in the history
…is doesn't remove the check at runtime, to match cloud mwaa, but it does speed up restarts a LOT.
  • Loading branch information
scnerd committed Nov 22, 2023
1 parent d484b08 commit 604dc64
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 0 deletions.
5 changes: 5 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ RUN chmod +x /entrypoint.sh

EXPOSE 8080 5555 8793

# Rearc requirements
COPY config/rearc-constraints.txt /rearc-requirements/constraints.txt
COPY config/rearc-requirements.txt /rearc-requirements/requirements.txt
RUN pip3 install $PIP_OPTION -r /rearc-requirements/requirements.txt

#USER airflow
WORKDIR ${AIRFLOW_USER_HOME}
ENTRYPOINT ["/entrypoint.sh"]
Expand Down
2 changes: 2 additions & 0 deletions docker/config/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
rearc-constraints.txt
rearc-requirements.txt
1 change: 1 addition & 0 deletions docker/config/airflow.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -387,6 +387,7 @@ default_cpus = 1
default_ram = 512
default_disk = 512
default_gpus = 0
default_deferrable = True

# Default queue that tasks get assigned to and that worker listen on.
default_queue = default
Expand Down

0 comments on commit 604dc64

Please sign in to comment.