Rethink pre/post deploy actions

[DuaelFr]

We currently have one script for predeploy actions and another for postdeploy and they can become quite complicated on real projects so they become hard to maintain.

It might be easier to get a collection of scripts for each of these "hooks". Combawa could run all scripts in a given directory instead of only run one shell file. Scripts would be ran in the alphabetical order, so devs could use some ordering like it's done in PHP configuration by prefixing the script name by a number (20-mything.sh).

This way, actions could be split in smaller chunks and each of them could have its own conditions inside instead of relying on the general structure of our templates.

We would have that kind of structure

scripts/combawa/
├─ pre_deploy/
│  ├─ 00-maintenance-mode-on.sh
│  ├─ 01-ensure-chmod.sh
│  ├─ 10-clean-files.sh
├─ post_deploy/
│  ├─ 00-post-update-1002.sh
│  ├─ 01-update-locale.sh
│  ├─ 10-create-test-accounts.sh
│  ├─ 20-create-default-content.sh
│  ├─ 50-search-index-rebuild.sh
│  ├─ 90-superadmin-login.sh
│  ├─ 99-maintenance-mode-off.sh
├─ install.sh
├─ update.sh

And scripts could look like this

# pre_deploy/00-maintenance-mode-on.sh
#!/bin/bash

if [ $COMBAWA_BUILD_MODE == "update" ]; then
  $DRUSH sset system.maintenance_mode 1
fi

or

# post_deploy/50-search-index-rebuild.sh
#!/bin/bash

if [ $COMBAWA_BUILD_ENV != "prod" ]; then
  $DRUSH sapi-r
fi
$DRUSH sapi-i

or ;)

# post_deploy/00-post-update-1002.sh
#!/bin/bash

UPDATE_PROCESSED=$($DRUSH eval "print (int) \Drupal::state('site_updates')->get('update_1002_reindexation_processed', 0);")
if [[ $UPDATE_PROCESSED == 0 ]]; then
  $DRUSH search-api:clear global
  $DRUSH search-api:clear mag_authors
  $DRUSH search-api:index global
  $DRUSH search-api:index mag_authors

  $DRUSH eval "\Drupal::state('site_updates')->set('update_1002_reindexation_processed', 1)"
fi

[Artusamak]

Splitting files this way could improve readability of each snippet. But... Define complicated? What is the longest file you have? Is it that bad? How often does it happen?

There is one layer missing from what i see in your example. It's the conditionality per environment. You could have your environment as a folder layer pre_deploy/dev pre_deploy/testing, etc. But this would force you to duplicate your snippet files. This would be a nightmare to figure out if each snippet is synced for its env when being updated.

The other way to introduce this env layer could be within the snippet itself to evaluate the environment and add conditions.

Both techniques would have the same issue, how do i easily embrace an overview of the steps executed for a given env? Which is in my opinion what matters in the end.

A thing that could be changed though in the current model, is to factorize fragments that are repeated on multiple environments by the usage of functions.

Instead of having:

#!/bin/bash
# Action to run after the main and shared deployment actions.
# It can be useful to enable specific modules for instance.

case $COMBAWA_BUILD_ENV in
  dev)
    if [ $COMBAWA_BUILD_MODE == "install" ] ; then
      $DRUSH migrate:import module_auteur
      $DRUSH migrate:import module_biblio
      $DRUSH migrate:import module_copyright_term
      $DRUSH migrate:import module_lieu
      $DRUSH migrate:import module_dispositifs
      $DRUSH migrate:import module_nature_images
      $DRUSH migrate:import module_objet
      $DRUSH migrate:import module_source_term
      $DRUSH migrate:import module_sujet_images
      $DRUSH migrate:import module_sujet_recherche
      $DRUSH migrate:import module_textes
      $DRUSH migrate:import module_series --continue-on-failure
      $DRUSH migrate:import module_serie_voc
      $DRUSH migrate:import module_notices --continue-on-failure --limit=1500
    else
      # Extra steps
    fi;
    # Extra steps
    ;;
  testing)
    if [ $COMBAWA_BUILD_MODE == "install" ] ; then
      $DRUSH migrate:import module_auteur
      $DRUSH migrate:import module_biblio
      $DRUSH migrate:import module_copyright_term
      $DRUSH migrate:import module_lieu
      $DRUSH migrate:import module_dispositifs
      $DRUSH migrate:import module_nature_images
      $DRUSH migrate:import module_objet
      $DRUSH migrate:import module_source_term
      $DRUSH migrate:import module_sujet_images
      $DRUSH migrate:import module_sujet_recherche
      $DRUSH migrate:import module_textes
      $DRUSH migrate:import module_series --continue-on-failure
      $DRUSH migrate:import module_serie_voc
      $DRUSH migrate:import module_notices --continue-on-failure --limit=1500
    else
      # Extra steps
    fi;
    # Extra steps
    ;;
  prod)
    if [ $COMBAWA_BUILD_MODE == "install" ] ; then
      $DRUSH migrate:import module_auteur
      $DRUSH migrate:import module_biblio
      $DRUSH migrate:import module_copyright_term
      $DRUSH migrate:import module_lieu
      $DRUSH migrate:import module_dispositifs
      $DRUSH migrate:import module_nature_images
      $DRUSH migrate:import module_objet
      $DRUSH migrate:import module_source_term
      $DRUSH migrate:import module_sujet_images
      $DRUSH migrate:import module_sujet_recherche
      $DRUSH migrate:import module_textes
      $DRUSH migrate:import module_series --continue-on-failure
      $DRUSH migrate:import module_serie_voc
      $DRUSH migrate:import module_notices --continue-on-failure --limit=1500
    else
      # Extra steps
    fi;
    # Extra steps
    ;;
  *)
    notify_error "Unknown environment: $COMBAWA_BUILD_ENV. Please check your name."
esac

We could have:

#!/bin/bash
# Action to run after the main and shared deployment actions.
# It can be useful to enable specific modules for instance.

case $COMBAWA_BUILD_ENV in
  dev)
    if [ $COMBAWA_BUILD_MODE == "install" ] ; then
      run_migrations()
    else
      # Extra steps
    fi;
    # Extra steps
    ;;
  testing)
    if [ $COMBAWA_BUILD_MODE == "install" ] ; then
      run_migrations()
    else
      # Extra steps
    fi;
    # Extra steps
    ;;
  prod)
    if [ $COMBAWA_BUILD_MODE == "install" ] ; then
      run_migrations()
    else
      # Extra steps
    fi;
    # Extra steps
    ;;
  *)
    notify_error "Unknown environment: $COMBAWA_BUILD_ENV. Please check your name."
esac

run_migrations() {
      $DRUSH migrate:import module_auteur
      $DRUSH migrate:import module_biblio
      $DRUSH migrate:import module_copyright_term
      $DRUSH migrate:import module_lieu
      $DRUSH migrate:import module_dispositifs
      $DRUSH migrate:import module_nature_images
      $DRUSH migrate:import module_objet
      $DRUSH migrate:import module_source_term
      $DRUSH migrate:import module_sujet_images
      $DRUSH migrate:import module_sujet_recherche
      $DRUSH migrate:import module_textes
      $DRUSH migrate:import module_series --continue-on-failure
      $DRUSH migrate:import module_serie_voc
      $DRUSH migrate:import module_notices --continue-on-failure --limit=1500
}

(It's on purpose a verbose example, we could use groups in this special use case :D)

[DuaelFr]

The longest I had was 200+ lines, mostly duplicated (triplicated ^^). The functions approach would reduce the size but I don't feel it's as effective as separate files.

In most projects now, we have not only conditions on environment but also on build mode or even on other parameters. This big case call was appropriate with our basic first uses but I feel it's getting less manageable with our recent projects and it's probably going to get worse if we start to implement post updates as we recently did.

Splitting this in multiple files have many advantages:

1. more readable and maintainable
2. easier to reorder
3. no need to duplicate/restrict on an environment if not needed
4. less chances of git conflicts
5. opens the door to other formats than bash (:grin:)

and only a few disadvantages:

1. harder to know what's executed on each environment

Upgrading to this new system could be quite easy because nothing would forbid to still use the current case on to put everything in the same file. It would only be needed to move the current pre/postdeploy_actions.sh file in the expected folder and that's it! Devs that would like to refactor these scripts to split them could do that afterwards.