Unable to build freshly scaffolded project in Lagoon due to missing drush aliases

[AlexSkrypnyk]

Summary

After recent changes to some scripts, Lagoon is "confused" by looking into files that no longer exist in the repository:

Site alias @govcms.prod not found.  

Details

New PaaS projects are provisioned using the code from the master branch of this project.

master branch contains Drush alias file as .docker/config/cli/ci.site.yml
develop branch contains Drush alias file as .docker/config/cli/govcms.site.yml

Both branches are using scaffold-tooling package, which has govcms-deploy file that expects .docker/config/cli/ci.site.yml to be present. This all should work fine, since master branch of this repo has .docker/config/cli/ci.site.yml. But Lagoon calls govcms-deploy command (/usr/bin/govcms-deploy) added to the CLI container in upstream image rather than govcms-deploy from scaffold-tooling package (./vendor/bin/govcms-deploy). This means that any script that calls govcms-deploy directly uses the old version of the script.

Another issue - .lagoon.yml in develop branch stopped using govcms-deploy altogether (🤦‍♂️). I'm not sure why this change was accepted as it now completely disconnects build processes in all 3 environments. The easiest way is to revert the latest commit 3bc0d35.

Locally, Ahoy calls the same outdated script as well in both master and develop.

In CI, GitLab config calls govcms-deploy from scaffold-tooling package.

So we have a situation where different govcms-deploy is called in different environments multiplied by having 2 branches (master and develop) that are not aligned.

Note that govcms-deploy is first identified script that has this issue, but there may be more, so it is important to resolve this in a way that can be applied to any future scripts.

Possible solutions

1. Update all references to govcms-deploy (located at the /usr/bin/govcms-deploy) to use ./vendor/bin/govcms-deploy.
   • Requires changes to .ahoy.yml
   • This would guarantee that there are no "hidden magic" scripts being executed. ./vendor/bin/govcms-deploy is available in the codebase and is not "baked" into container.
   • This will also require adding such script reference to the SaaS scaffold, but this can be done in a different time frame than PaaS.
     - Revert .lagoon.yml to use ./vendor/bin/govcms-deploy instead of using bunch of commands.

OR

2. Overwrite /usr/bin/govcms-deploy with ./vendor/bin/govcms-deploy file during Docker build in PaaS
   • Requires changes to Dockerfile.cli (RUN cp -f /app/vendor/bin/govcms-deploy /usr/bin/govcms-deploy).
   • Requires changes to .gitlab-ci-main.yml in scaffold-tooling repo to guarantee that the same file is called.
   • Not possible to change the version of the scaffold-tooling package and bring new govcms-deploy without rebuilding CLI container - not convenient for devops development.
   • Preserves "old" ways, but does not follow the "new" way, where all tools coming from ./vendor/bin
     - Revert .lagoon.yml to use ./vendor/bin/govcms-deploy instead of using bunch of commands.

The preferable option is Option 1, as it would follow how other scripts are called - from ./vendor/bin/.

Branches alignement solution

Once we solve the govcms-deploy script issue, we need to make a release of this project to even up develop with master so that new projects can get these new fixed changes.

[simesy]

Update all references to govcms-deploy (located at the /usr/bin/govcms-deploy) to use ./vendor/bin/govcms-deploy

This is the correct solution to continue to remove all the upstream drupal logic baked into govcms8lagoon.

This will also require adding such script reference to the SaaS scaffold, but this can be done in a different time frame than PaaS.

There is a PR for update SaaS to use the PaaS scaffold with a fairly large document that went to CAB to approve.

It should be govcms.site.yml to confirm.

Also to be really clear. A lot of people decided to update .lagoon.yml the way it is for the time being. This has been discussed a lot.

[AlexSkrypnyk]

@simesy
There will be another issue created about discussing govcms-deploy requirements - something that should have been done before we started changing any scaffolding. This time, we will capture all the requirements and add diagrams for the all possible variation of the logic that govcms-deploy script should have and if it is feasible to use variables to control the behaviour. But this is for another issue.

As for .lagoon.yml file - we should be sticking with govcms-deploy by default AND at the same time providing those commands as commented-out code, so that those developers that do not know/do not need to change deployment logic would use govcms-deploy that 100% tested and works, but those who need to modify the deployment logic - can do so by uncommenting the code and swapping govcms-deploy with any customisation they want (once they do that they are on their own with any deployment issues). What we have now in develop is the worst that could happen - local, CI and Lagoon are all using own custom logic for deployment, which is a cause of many issues to come (as soon as develop is released). There will be a situation where locally and CI passes and all looks good, but Lagoon cannot deploy. And this is C R A Z Y! Multiply this by the number of environments and number of clients - and the support desk will be buried under requests about problems with deployment.

[simesy]

I feel like you're reiterating the same point?

[AlexSkrypnyk]

@simesy
not quite. please see #47