Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Allow installing and testing with all database types, including MariaDB #34

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
27 changes: 22 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,21 @@ This is a DDEV addon for doing Drupal core development.

We're in #ddev-for-core-dev on [Drupal Slack](https://www.drupal.org/community/contributor-guide/reference-information/talk/tools/slack) (but please try and keep work and feature requests in Issues where it's visible to all 🙏)

`ddev drush` is fully supported, along with using or testing MariaDB, MySQL, and PostgreSQL databases (and Sqlite3)


```
git clone https://git.drupalcode.org/project/drupal.git drupal
cd drupal
ddev config --omit-containers=db --disable-settings-management
ddev start
ddev config --project-type=drupal
ddev get justafish/ddev-drupal-core-dev
ddev restart
ddev composer install
ddev config --update

# See included commands
ddev drupal list

# Install drupal
ddev drupal install
ddev drush si -y --account-pass==admin

# Run PHPUnit tests
ddev phpunit core/modules/sdc
Expand All @@ -26,6 +27,22 @@ ddev phpunit core/modules/sdc
ddev nightwatch --tag core
```

## Using various database types

By default, the DDEV default database type is used (MariaDB).

To use another supported database type,
`ddev delete -Oy` and `ddev config --database=mysql:8.0` or `ddev config --database=postgres:16` for example.

To use Sqlite,
```
ddev stop
ddev config --disable-settings-management --omit-containers=db
rm -rf web/sites/default/settings*.php web/sites/default/files
ddev start
ddev drupal install
```

## Nightwatch Examples

You can watch Nightwatch running in real time at https://drupal.ddev.site:7900
Expand Down
14 changes: 14 additions & 0 deletions commands/web/drush
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/bash

#ddev-generated
## Description: Run drush CLI inside the web container, installing it if necessary
## Usage: drush [flags] [args]
## Example: "ddev drush uli" or "ddev drush sql-cli" or "ddev drush --version"
## ProjectTypes: drupal7,drupal8,drupal9,drupal10,drupal,backdrop
## ExecRaw: true

if ! command -v drush >/dev/null; then
echo "drush is not yet installed. Installing it...'"
.ddev/core-dev/install_drush.sh
fi
drush "$@"
31 changes: 31 additions & 0 deletions config.core-dev.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# #ddev-generated
# This file is placed by the justafish/ddev-drupal-core-dev addon.

webimage_extra_packages: ["chromium-driver"]
ddev_version_constraint: '>=v1.23.0'
hooks:
post-start:
- exec: |
dburl='sqlite://localhost/sites/default/files/db.sqlite'
cp .ddev/core-dev/phpunit-chrome.xml core/phpunit.xml
cp .ddev/core-dev/.env core/.env
if ping -c 1 db >/dev/null 2>&1; then
case ${DDEV_DATABASE_FAMILY:-} in
mysql)
# the backslash here is to prevent perl from eating the @
dburl='mysql://db:db\@db/db'
;;
postgres)
dburl='pgsql://db:db\@db/db'
;;
esac
fi
perl -pi -e "s|SIMPLETEST_DB_VALUE|${dburl}|g" core/phpunit.xml

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rfay is it intentional that this is still using the sqlite DB for running PHPUnit tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this was using mariaDB by accident until a recent PR. But IMO it should use mariaDB by default and allow using any supported database. That's actually why I started working on this PR a bit, because I wanted to see "real" performance comparisons on Drupal, comparing DB types. I think it's possible that the use of sqlite3 "for performance reasons" may be obsolete.

perl -pi -e "s|DRUPAL_TEST_DB_URL_VALUE|${dburl}|g" core/.env
perl -pi -e "s|DRUPAL_CORE_DDEV_URL|${DDEV_PRIMARY_URL}|g" core/phpunit.xml

upload_dirs:
# The install technique tries to remove all of sites/default/files
# but with DDEV + mutagen that isn't possible.
# so just redirect the upload_dirs.
- .ddev/tmp
1 change: 0 additions & 1 deletion config.ddev-drupal-core-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

webimage_extra_packages: ["chromium-driver"]
ddev_version_constraint: '>=v1.23.1'
omit_containers: ["db"]
upload_dirs:
# The install technique tries to remove all of sites/default/files
# but with DDEV + mutagen that isn't possible.
Expand Down
2 changes: 1 addition & 1 deletion core-dev/.env
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ DRUPAL_TEST_BASE_URL=http://web

# By default we use sqlite as database. Use
# mysql://username:password@localhost/databasename#table_prefix for mysql.
DRUPAL_TEST_DB_URL=sqlite://localhost/sites/default/files/db.sqlite
DRUPAL_TEST_DB_URL=DRUPAL_TEST_DB_URL_VALUE

#############
# Webdriver #
Expand Down
3 changes: 2 additions & 1 deletion core-dev/gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@
/sites/simpletest
/sites/default/files
/vendor
test_output
test_output
*.orig
41 changes: 41 additions & 0 deletions core-dev/install_drush.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#!/bin/bash

#ddev-generated
# This script installs drush/drush without changing the git
# status of the Drupal checkout

set -eu -o pipefail

if command -v drush >/dev/null; then
echo "drush is already installed, taking no action. You can remove it if you want to reinstall"
exit
fi

echo "Installing drush without affecting git status"

# Make certain that we have something staged so we can create stash
touch .makedrush.txt
git add .makedrush.txt

# Save the stash, which will include anything people were doing
# plus the .makedrush.txt. This gets us back to "no changes"
# in `git status`
git stash

# Install drush
composer require drush/drush --with-dependencies

# Roll back to what we started with. Cleans up
# composer.* and anything else that the drush install changes
# But vendor directory is untouched since
# it's gitignored
git reset --hard

# Restore anything that might have been staged
# prior to the start
git stash pop

# Get rid of our dummy file
git rm -f .makedrush.txt

echo "drush/drush is installed in $(which drush)"
3 changes: 1 addition & 2 deletions core-dev/phpunit-chrome.xml
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@
<!-- Do not limit the amount of memory tests take to run. -->
<ini name="memory_limit" value="-1"/>
<env name="SIMPLETEST_BASE_URL" value="DRUPAL_CORE_DDEV_URL"/>
<!-- <env name="SIMPLETEST_DB" value="mysql://db:db@db/db"/> -->
<env name="SIMPLETEST_DB" value="sqlite://localhost/sites/default/files/db.sqlite"/>
<env name="SIMPLETEST_DB" value="SIMPLETEST_DB_VALUE"/>
<env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/var/www/html/test_output"/>
<!-- By default, browser tests will output links that use the base URL set
in SIMPLETEST_BASE_URL. However, if your SIMPLETEST_BASE_URL is an internal
Expand Down
3 changes: 1 addition & 2 deletions core-dev/phpunit-firefox.xml
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@
<!-- Do not limit the amount of memory tests take to run. -->
<ini name="memory_limit" value="-1"/>
<env name="SIMPLETEST_BASE_URL" value="DRUPAL_CORE_DDEV_URL"/>
<!-- <env name="SIMPLETEST_DB" value="mysql://db:db@db/db"/> -->
<env name="SIMPLETEST_DB" value="sqlite://localhost/sites/default/files/db.sqlite"/>
<env name="SIMPLETEST_DB" value="SIMPLETEST_DB_VALUE"/>
<env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/var/www/html/test_output"/>
<!-- By default, browser tests will output links that use the base URL set
in SIMPLETEST_BASE_URL. However, if your SIMPLETEST_BASE_URL is an internal
Expand Down
8 changes: 3 additions & 5 deletions install.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@
name: ddev-drupal-core-dev

project_files:
- config.ddev-drupal-core-dev.yaml
- config.core-dev.yaml
- docker-compose.core-dev-selenium.yaml
- core-dev/phpunit-firefox.xml
- core-dev/phpunit-chrome.xml
- commands/web/drupal
- commands/web/drush
- commands/web/phpunit
- commands/web/nightwatch
- core-dev/install_drush.sh
- core-dev/gitignore
- core-dev/.env
- core-dev/src/Command/AdminLoginCommand.php
Expand All @@ -20,13 +22,9 @@ project_files:
- core-dev/src/Command/UninstallCommand.php

post_install_actions:
- cp core-dev/phpunit-chrome.xml ../core/phpunit.xml
- perl -pi -e "s|DRUPAL_CORE_DDEV_URL|$DDEV_PRIMARY_URL|g" ../core/phpunit.xml
- cp core-dev/.env ../core/.env
- cp core-dev/gitignore ../.gitignore
- mkdir -p ../test_output
- chmod +w ../test_output
- cd ../core && ddev yarn

removal_actions:
- |
Expand Down
29 changes: 23 additions & 6 deletions tests/test.bats
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,40 @@ setup() {
set -eu -o pipefail
export DIR="$( cd "$( dirname "$BATS_TEST_FILENAME" )" >/dev/null 2>&1 && pwd )/.."
export TESTDIR=~/tmp/test-ddev-drupal-core-dev
mkdir -p $TESTDIR
export PROJNAME=ddev-drupal-core-dev
rm -rf ${TESTDIR}
mkdir -p ${TESTDIR}
export PROJNAME=test-ddev-drupal-core-dev
export DDEV_NON_INTERACTIVE=true
ddev delete -Oy ${PROJNAME} >/dev/null 2>&1 || true
curl -L -o /tmp/drupal.tar.gz https://ftp.drupal.org/files/projects/drupal-11.x-dev.tar.gz
tar --strip-components 1 -zxf /tmp/drupal.tar.gz -C ${TESTDIR}
cd "${TESTDIR}"
mv vendor /tmp/vendor.bak
git config --global user.email "[email protected]"
git config --global user.name "Example Example"
git init && git add . >/dev/null && git commit -m "current" >/dev/null
mv /tmp/vendor.bak vendor
ddev config --project-name=${PROJNAME} --upload-dirs=.ddev/tmp
ddev config --update
ddev start -y >/dev/null
ddev composer install >/dev/null
}

health_checks() {
ddev exec "curl -s chrome:7900" | grep "noVNC"
ddev exec "curl -s firefox:7901" | grep "noVNC"
base_checks() {
ddev exec "curl -s chrome:7900" | grep "noVNC" >/dev/null
ddev exec "curl -s firefox:7901" | grep "noVNC" >/dev/null
ddev phpunit core/tests/Drupal/Tests/Component/Datetime/DateTimePlusTest.php
}

drush_checks() {
# Make sure there's nothing in the git index before drush install
git diff --cached --quiet
ddev drush st
# Make sure there's nothing after the drush install
git diff --cached --quiet || (echo "git index has been touched" && exit 2)
ddev drush si -y --account-pass=admin
}

teardown() {
set -eu -o pipefail
cd ${TESTDIR} || ( printf "unable to cd to ${TESTDIR}\n" && exit 1 )
Expand All @@ -33,7 +49,8 @@ teardown() {
echo "# ddev get ${DIR} with project ${PROJNAME} in ${TESTDIR} ($(pwd))" >&3
ddev get ${DIR}
ddev restart
health_checks
base_checks
drush_checks
}

#TODO: Re-enable release tests after the add-on has a release with DDEV v1.23.0 support
Expand Down