dev/core#5611 Exclude tests and tools directories from exports

[colemanw]

Overview

Avoid distributing test files in packaged downloads.
See https://lab.civicrm.org/dev/core/-/issues/5611

Technical Details

I'm not sure if this only fixes composer-based installs and we also need to do something for our packaging script to avoid putting ext/*/tests directories in the zip/tar files.

[civibot]

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...

• If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
• A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
• A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
• If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
• Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
• A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
• To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
• For more information about contributing, see CONTRIBUTING.md.

Quick links for reviewers...

• ➡️ PR Demo site (can be rebuilt with the command test this please).
• 📖 Review standards
• 🗒️ Review template (brief or verbose)

➡️ Online demo of this PR 🔗

[civibot]

The issue associated with the Pull Request can be viewed at https://lab.civicrm.org/dev/core/-/issues/5611

[demeritcowboy]

I tried this out. When using --prefer-source, it does like it used to. When using --prefer-dist (or not specifying), it does exclude the top-level tests and tools but still installs the ext tests, so the pattern needs a little update.

I tested by composer init'ing a little project and then adding a vcs repo, like below, and then composer require civicrm/civicrm-core:dev-gitattributes [--prefer-dist|--prefer-source]. By the way prefer-source is way slower than dist.

  "minimum-stability": "dev",
    "repositories": [
        {
            "type": "vcs",
            "url": "https://github.com/colemanw/civicrm-core"
        }
    ],

[colemanw]

@demeritcowboy I've prepended them with a wildcard. See if that works.

[demeritcowboy]

Looks better. I'll put merge-ready because I don't know if there's a companion update needed for drupal 8+ tests, and I'm not sure where they run nowadays - I took a quick look at test.civicrm.org but it wasn't obvious.

[demeritcowboy]

I added the run-drupal-x tag - so I think it does need a corresponding buildkit or jenkins update since I see for example: Cannot open file "/home/homer/buildkit/build/build-1/vendor/civicrm/civicrm-core/ext/afform/core/tests/phpunit/bootstrap.php".

[totten]

By the way prefer-source is way slower than dist.

Yeah, at the end of MM discussion, I got a bit nervous about that extra load from getting all sources.

FWIW, the relevant files for test build are https://github.com/civicrm/civicrm-buildkit/blob/master/app/config/drupal10-dev/download.sh and https://github.com/civicrm/civicrm-buildkit/blob/master/app/config/drupal9-dev/download.sh

In playing with that on my local, I see differences in both run-time and disk-usage (source: 58s, 2132mb; dist: 35s, 894mb). For core/d10 test-jobs, I expect those stats should be representative of an average test-build. However, given the use of tmpfs and per-runner caches, the increased footprint would be quite visible (but not fatal).

Thinking outloud, another consideration of the extra work is the reliability of the operations:

• Network I/O (e.g. requests to github.com and civicrm.org) can randomly fail. (Call these... network-hiccups?) I don't have real stats, but they happen with some regularity. For an anecdotal ballpark -- call it 0.05% of requests.
• Say a typical build needs 100 packages.
• For --prefer-dist (ZIP) packages, the local cache is authoritative. You don't usually need any network I/O to read the artifacts in the cache. You're really only downloading (say) 5 packages. This is great because you zero-out the hiccup risk for the other 95 packages.
• For --prefer-source (VCS), composer resyncs the VCS cache (git fetch) for every package. That means all 100 packages are exposed to hiccup risk.
• The concern is that the exposure jumps significantly. (Say there are 500 jobs per day. 500*(0.9995^5) gives 498 success and 2 failures. But 500*(0.9995^100) gives 475 successes and 25 failures. That's a much more noticeable problem.)
• The theory of "hiccups" is an oversimplification, and all those numbers are fuzzy. So take that all with a big grain of salt.

Anyway, where I'm heading is that CI jobs probably shouldn't use broad options (--prefer-source vs --prefer-dist). Instead, there is a more targeted option:

https://stackoverflow.com/a/56602818/4195300

I'm gonna play with adding that to civibuild...

[totten]

I added the run-drupal-x tag - so I think it does need a corresponding buildkit or jenkins update since I see for example: Cannot open file "/home/homer/buildkit/build/build-1/vendor/civicrm/civicrm-core/ext/afform/core/tests/phpunit/bootstrap.php".

Arg. This feels kind of like a composer bug/quirk to me.

The distinctive feature of drupal10-dev is it builds a site with instructions like:

    "repositories": {
        "civicrm-packages": {
            "type": "path",
            "url": "./src/civicrm-packages",
            "options": {
                "symlink": ...
            }
        },
        "civicrm-drupal-8": {
            "type": "path",
            "url": "./src/civicrm-drupal-8",
            "options": {
                "symlink": ...
            }
        },
        "civicrm-core": {
            "type": "path",
            "url": "./src/civicrm-core",
            "options": {
                "symlink": ...
            }
        },
        "0": {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        }
    },

(Why: This helps with testing proposed revisions in ./src/civicrm-core/composer.json.)

The application of prefer-source / prefer-dist depends on how the path repository is used. These repositories operate in "copy" or "symlink" mode:

• With "symlink" mode, you get one symlink, and all the files come along with it. Not much else to be done. Effectively, it ignores gitattributes. (Which is good if your goal is run unit-tests.)
• With "copy" mode, I was expecting it to check the prefer-source or prefer-dist policy when deciding whether to apply the gitattributes filter. But in fact, it applies unconditionally. (Which is bad if your goal is to run unit-tests.)

A possible work-around is to enable symlink mode (civicrm/civicrm-buildkit#891). That nominally provides the files that we want, but then cv core:install fails. ☹️