DAB RESOURCE SYNC flag on aap.yaml (#2302)

* DAB RESOURCE SYNC flag on aap.yaml

* More docs

---------

Co-authored-by: jctanner <[email protected]>

dev/compose/README.md

@@ -91,19 +91,78 @@ Ansible Hub UI can be started separately as a standalone `npm` run.
 
 
 ```console
+# Assuming galaxy_ng is running on community compose.
 
+$ git clone https://github.com/ansible/ansible-hub-ui ~/projects/ansible-hub-ui
+$ git clone https://github.com/ansible/galaxy_ng ~/projects/galaxy_ng
 ```
+Open 2 terminals:
 
-## Reload
+On the first terminal:
 
-Changing `.py` and `.yaml` files on any of the `DEV_SOURCE_PATH` directories will trigger reload of `api`, `worker`, and `content` services.
+```console
+$ cd galaxy_ng
+$ docker compose -f dev/compose/community.yaml up
+```
+
+On the second terminal:
+
+```console
+cd ansible-hub-ui
+npm install
+API_PROXY_PORT=5001 API_BASE_PATH=/api/galaxy/ npm run start-standalone
+```
+
+UI will be available on http://localhost:8002 and API on http://localhost:5001
 
 
+## Auto Reload and Local Checkouts
+
+To have the services `api`, `worker` and `content` to automatically reload when
+source code changes it is required to set which paths the `reloader` must watch for changes.
+
+Set the variable `DEV_SOURCE_PATH` with the packages you want to add to the reloader list.
+
+Those repositories must be local checkouts located on the same level of the `galaxy_ng` repository.
+
+Example:
+
+Get the repositories locally in the same root path.
+```console
+$ git clone https://github.com/dynaconf/dynaconf ~/projects/dynaconf
+$ git clone https://github.com/pulp/pulp_ansible ~/projects/pulp_ansible
+$ git clone https://github.com/ansible/galaxy_ng ~/projects/galaxy_ng
+```
+
+> **IMPORTANT** Ensure all the repos are checked out to compatible branches.
+> for example. you may be on galaxy_ng:master and checking out `setup.py` you
+> see that it requires `pulp_ansible>2.10,<3` then ensure you checkout `pulp_ansible`
+> to a compatible branch.
+
+Start the compose setting the desired editable paths.
+
+```console
+$ cd ~/projects/galaxy_ng
+$ export DEV_SOURCE_PATH="dynaconf:pulp_ansible:galaxy_ng"
+$ docker compose -f dev/compose/app.yaml up --build
+```
+
+Optionally it can be informed in a single line:
+
+```console
+$ DEV_SOURCE_PATH="dynaconf:pulp_ansible:galaxy_ng" docker compose -f dev/compose/app.yaml up --build
+```
+
+> **NOTE** if passing on the call line, remember to repass the same variable every time you interact with
+>`docker compose` for exec, logs, down etc.
+
+Now when changes are detected on `.py` and `.yaml` files on any of the `DEV_SOURCE_PATH`
+directories it will trigger reload of `api`, `worker`, and `content` services.
+
 ## Tips and Tricks.
 
 **TBD**
 
-
 ### Debugging
 
 ### Connecting to Database

dev/compose/aap.yaml

@@ -60,11 +60,24 @@ x-common-env: &common-env
  # role content workaround ..
  PULP_ANSIBLE_BASE_ROLES_REQUIRE_VIEW: 'false'
 
- # DEV EDITABLE STUFF
+ # Resource server
+ # This disables the attempt for resource syncing
+ PULP_RESOURCE_SERVER_SYNC_ENABLED: 'false'
+ # Set above to 'true' if/when RESOURCE_SERVER is configured
+ # The next variables must be configured to enable resource sync
+ # PULP_RESOURCE_SERVER__URL='https://localhost'
+ # PULP_RESOURCE_SERVER__SECRET_KEY='?'
+ # PULP_RESOURCE_SERVER__VALIDATE_HTTPS='false'
+
+ # Unpin dependencies on setup.py if set to 0
  LOCK_REQUIREMENTS: 0
- DEV_SOURCE_PATH: "galaxy_ng"
- # To enable editable installs of local checkouts change the variable above keeping the ordering as follows:
- # DEV_SOURCE_PATH: "dynaconf:pulpcore:galaxy_importer:pulp_ansible:pulp_container:galaxy_ng:django-ansible-base"
+
+ # DEV EDITABLE STUFF
+ # To enable editable installs of local checkouts set DEV_SOURCE_PATH keeping the ordering as follows:
+ # "dynaconf:pulpcore:galaxy_importer:pulp_ansible:pulp_container:galaxy_ng:django-ansible-base"
+ # This can be done as part of the `docker compose` call:
+ # $ DEV_SOURCE_PATH="pulp_container:galaxy_ng" docker compose -f dev/compose/aap.yaml up
+ DEV_SOURCE_PATH: ${DEV_SOURCE_PATH:-""}
 
 
 services:
@@ -257,8 +270,12 @@ services:
  /src/galaxy_ng/dev/compose/bin/wait /etc/pulp/certs/signing-secret.key;
  /src/galaxy_ng/dev/compose/bin/devinstall;
 
- # Schedule resource sync;
- pulpcore-manager task-scheduler --id dab_sync --interval 15 --path "galaxy_ng.app.tasks.resource_sync.run";
+ # Give some time for API to start;
+ sleep 5;
+
+ # Schedule resource sync if enabled;
+ /src/galaxy_ng/dev/compose/dab/schedule_resource_sync.sh;
+ echo 'Scheduled tasks';
  curl -s -u admin:admin http://api:24817/api/galaxy/pulp/api/v3/task-schedules/?name=dab_sync | python -m json.tool;
 
  # Keys are needed to register signing services;
@@ -267,6 +284,7 @@ services:
 
  # Setup signing services;
  /src/galaxy_ng/dev/compose/signing/setup_signing_services.sh;
+ echo 'Signing Services';
  curl -s -u admin:admin http://api:24817/api/galaxy/pulp/api/v3/signing-services/?fields=name,script,pubkey_fingerprint | python -m json.tool;
 
  # Setup repository gpgkey for upload verification;

dev/compose/bin/devinstall

@@ -1,9 +1,12 @@
 #!/usr/bin/env bash
+if [ -z "${DEV_SOURCE_PATH}" ]; then
+ echo "DEV_SOURCE_PATH is not defined, skipping editable installs."
+ exit 0
+fi
 
-local src_path_list
-IFS=':' read -ra src_path_list <<< "$DEV_SOURCE_PATH"
+IFS=':' read -ra SRC_PATH_LIST <<< "$DEV_SOURCE_PATH"
 
-for item in "${src_path_list[@]}"; do
+for item in "${SRC_PATH_LIST[@]}"; do
  src_path="/src/${item}"
  if [[ -d "$src_path" ]]; then
  echo "Installing path ${item} in editable mode."

dev/compose/bin/reloader

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 
 # List all files in the DEV_SOURCE_PATH and watch for changes and use entr to SIGKILL + restart
-echo "Watching $DEV_SOURCE_PATH"
+echo "Watching file changes on $DEV_SOURCE_PATH"
 
 find $(echo $DEV_SOURCE_PATH | tr ':' '\n' | while read item; do echo -n /src/$item\ ; done) \( -path /src/galaxy_ng/.venv -o -path /src/galaxy_ng/build -o -path /src/galaxy_ng/.eggs \) -prune -o -name '*.py' -o -name '*.yaml' | /src/galaxy_ng/dev/compose/bin/entr -n -r timeout -k 5 0 $1

dev/compose/dab/README.md

@@ -0,0 +1,6 @@
+# Django Ansible Base specific scripts and artifacts
+
+
+## schedule_resource_sync.sh
+
+Script will schedule a periodic resource sync when the stack is started with `RESOURCE_SERVER_SYNC_ENABLED` settings set to true.

dev/compose/dab/schedule_resource_sync.sh

@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+
+if [ "$PULP_RESOURCE_SERVER_SYNC_ENABLED" = "true" ]; then
+ pulpcore-manager task-scheduler --id dab_sync --interval 15 --path "galaxy_ng.app.tasks.resource_sync.run";
+else
+ echo 'Resource Sync is Disabled.';
+fi