Dev Tools

Tools used for development of Tide distribution and modules.

Prerequisites

1. Make sure that you have latest versions of all required software installed:

• Docker
• Pygmy
• Ahoy

2. Make sure that all local web development services are shut down (apache/nginx, mysql, MAMP etc).

Local environment setup

1curl https://raw.githubusercontent.com/dpc-sdp/dev-tools/master/install | bash 2ahoy build

Available ahoy commands

Run each command as ahoy <command>.

   build                Build or rebuild project.
   clean                Remove all build files.
   clean-full           Remove all development files.
   cli                  Start a shell inside CLI container or run a command.
   composer-merge       Merge composer files.
   doctor               Find problems with current project setup.
   down                 Stop Docker containers and remove container, images, volumes and networks.
   drush                Run drush commands in the CLI service container.
   info                 Print information about this project.
   install-dev          Install dependencies.
   install-site         Install site.
   lint                 Lint code.
   login                Login to a website.
   logs                 Show Docker logs.
   pull                 Pull latest docker images.
   restart              Restart all stopped and running Docker containers.
   start                Start existing Docker containers.
   stop                 Stop running Docker containers.
   test-behat           Run Behat tests.
   up                   Build and start Docker containers

Main ideas behind Dev Tools

• Do not add something that does not normally exist in the repos. Example: Drupal module repo should not have Docker configs.
• Fetch as much of dev config as possible from Dev Tools repository. Caveat: CircleCI config does not support inclusions (each project must commit it) or we would use a fetched version for it as well. Instead, the steps were abstracted into scripts, so that the main CI configuration file does not need to be changed.
• Abstract similar functionality into steps. This will allow to apply changes at the larger scale without the need to modify each project.
• Run the most of the code in the containers. This is to reduce required tools on the host machine. Caveat: due to some limitations, jq must be installed on the host to work on module projects. This may change in the future.

Features

• Supports project (full-site) and module projects
• Provides Docker stack based on Bay
• Allows to override configuration per-project
• Simple initialisation by CURLing install script
• Provides CI configuration
• Provides Behat configuration

Working with module projects

Dev Tools allows to add all required tools to module and site projects.

To support adding Dev Tools for development, it was designed to be fetchable from a separate repository. The main reason is to simplify the maintenance of module projects: remove the overhead of maintaining the same development files in all repositories.

While it is used only for development in module projects, for site builds it is advised to commit Dev Tools files to enable proper integration with hosting services (Lagoon) - cannot be hosted on Lagoon if there are no docker files!

To be able to support both module and site build projects, a series of configuration variables are available.

More specifically, for module projects, to enable local development (and CI) the Dev Tools needs to be fetched, composer configuration needs to be merged and a project needs to be built. All this is handled with a single ahoy build command. Dev Tools also supports symlinking of current module files into Drupal project to support development.

In order to allow this to happen, Dev Tools creates composer.build.json - config file with merged dependencies.

Then, we install a site and install current module.

To install current module, we export current directory as is and place it into a local composer dependency folder. Then, we provide a new location for this repository and require it. Since composer does not allow to symlink the files to the deps itself, we are working on the copy of files. Once the work is finished, we need to run a sync script to make these files overwrite actual files in the repository (this approach is very cubersome and not implemented yet, but it is the only one way to overcome Composer limitation).

Next, we assess if we need to install suggested packages - this is to ensure that optional configuration works as expected. We also enable them (as there is no way to specify optional packages in module files).

Finally, we install the module itself.

Releases and versions

• For projects and modules, we use SemVer 2.0.0:

    1.0.0
    1.2.0
    1.2.3
    1.2.3-alpha.1
    1.2.3-alpha.1.2
  

FAQs

How to use Xdebug?

1. Uncomment this line in docker-compose.yml:

#XDEBUG_ENABLE: "true"
#DOCKERHOST: "host.docker.internal"

2. Restart the stack: ahoy up.
3. Enable listening for incoming debug connections in your IDE.
4. If required, provide server URL to your IDE as it appears in the browser: http://<YOURSITE>.docker.amazee.io
5. Enable Xdebug flag in the request coming from your browser.
6. Set a breakpoint in your IDE and perform a request in browser.

How to specify custom branch for suggested module?

Just add it as a normal branch constraint to a name of the module:

    "suggest": {
        "dpc-sdp/tide_api:dev-ci": "Allows to use Drupal in headless mode"
    }

What is 'Suggested mode'?

This is how we test optional configuration. Modules may specify their optional dependencies using 'suggest' section in composer.json file. The CI process for each module runs build_suggest build, where these optional modules are enabled.

What is @suggest and @nosuggest tags in tests?

Tagging a test with @suggest means that it will run only for Suggested mode (see above). Tagging a test with @nosuggest means that the test will run only in normal mode. Not providing a tag will run test in both modes.

Workflow recipes

Using Dev Tools

Init Dev Tools in your project

curl https://raw.githubusercontent.com/dpc-sdp/dev-tools/master/install | bash or commit dev-tools.sh to your project and execute it as . dev-tools.sh. This will install the latest published release of Dev Tools.

Override Dev Tools files in your project

1. Remove lines that point to files being overridden from .git/info/exclude in your project repository.
2. Commit required files. They will not be overridden on next Dev Tools install.

Force update overridden Dev Tools files in your project

ALLOW_OVERRIDE=1 . dev-tools.sh

Use Dev Tools at specific commit

GH_COMMIT=COMMIT_SHA . dev-tools.sh

Branch-based development for module projects

Use development versions of dependencies

TBD

Maintenance

To update Dev Tools and release a new version, a special care should be taken as broken Dev Tools will break all project builds.

To make a change:

1. Create a separate branch in Dev Tools repository.
2. Make your changes, commit them and make sure the CI passes.
3. Use one of the tide_* modules to run the build. Create a branch in that module's repository.
4. Modify dev-tools.sh in that module and specify GH_COMMIT value of the commit from step 2.
5. Commit and make sure that the build passes.
6. If the build passes, merge the Dev Tools branch to master and make a release.
   • NOTE: This must be a release in GitHub, see https://github.com/dpc-sdp/dev-tools/releases/new
7. Remove the temporary branch in tide_* module from Step 3.

Unit test

The dev-tools supports unit tests.

Registering tests

Directory

1. The unit tests should go in tests\src\Unit direcotry.
2. The unit tests need to respect PSR-4 namespace naming convention, in our case, namespace Drupal\Tests\{module}\Unit

2 ways to run tests

1. commit to CI.
2. Run ./vendor/bin/phpunit inside CLI container locally.