quick and dirty cv core:download

[michaelmcandrew]

I wasn't sure why there is no core:download command for cv (hopefully, I didn't miss something).

wp-cli and drush have equivilant commands so I wrote this quick and dirty version for cv.

Initial thoughts? Happy to neaten up, refactor, etc.

[totten]

@michaelmcandrew This is a neat idea!

I don't have a beautiful message here. Let me just brain-dump a bit:

• In core:install and core:check-req, it autodetects the current CMS. This works because they use BootTrait's helper function _boot_cms_only(). It should work just as well here.
• There's another file UpgradeDlCommand.php (upgrade:dl) which is supposed to download/upgrade the core code. @agh1 did most of it. However, it's currently disabled because I haven't done enough testing.
• I wonder if it's better to have core:download and upgrade:dl as separate commands -- or to combine them?
  • Functionally, you'd want them to do a lot of the same things:
    • Figure out the available version and download URL.
    • Have support for downloading current stable, current RC, current master, or a specific number.
    • Cache the tarballs/zipballs.
  • The main differences are that:
    • With an install, you want to make a new folder. Sysadmins might have some discretion to choose the destination folder (sites/all/modules/civicrm, sites/foo.com/modules/civicrm, etc).
    • With an upgrade, you want to replace an old folder. The destination folder should be preserved (based on where civicrm is already installed).
• There's an innate complexity with respect to file-layouts on different CMSs:
  • (On one hand...) We can take a light/dumb approach which just downloads to CWD (like the PR for core:download). In that case, the complexity is transferred to sysadmins and to the sysadmin documentation. They have to understand the file-layout implications and navigate accordingly before calling cv core:download.
  • (On the other hand...) We can take a heavy/smart approach which picks a destination based on common practice (e.g. D6/7:sites/all/modules/civicrm; BD:'modules/civicrm'; WP:'wp-content/plugins/civicrm`). In that case, admins for vanilla systems are buffered from the complexity, but there's a bit more complexity in code, and advanced admins need a way to do things differently. Of course, "advanced" is subjective, and folks in "advanced" situations may not realize that they need to do things in an "advanced" way.
  • Maybe it's better to think of the "version" and "destination" as options with defaults -- defaults in which interactive users are required to make a choice.
• We don't have much of a "style guide" for CLI interaction in cv.
  • Originally, I had an unstated style: commands should be easily scriptable and therefore non-interactve; they should take command-line params and return structured output (json, csv, table, pretty, etc). Good examples are cv api, cv url, cv php:eval, cv ext:list, cv ang:html:list. These commands all end with a call to $this->sendTable(...) or $this->sendResult(...). The convention makes it easy to plug cv into external scripting/testing systems. (Think of Protractor, Codeception, Cucumber, Behat.)
  • However, as we've gotten into more sophisticated commands (ext:download, core:install, upgrade:db), there are more edge-cases where the scriptable UX feels awkward for a human user. (Example: If ext:download finds that an extension already exists, then a human user benefits from a prompt about whether to keep or overwrite or abort. Example: if upgrade:db finds pre-upgrade messages, then it should pause before running the upgrade.)
  • I've been dealing with this tension between scriptability and usability adhoc. We may eventually want some kind of style guide, but I'd be a little concerned enforcing that standard today.

I don't want to set the bar too high, but let me try sketching what I think would be a reasonable set of user-stories:

## Story 1 -- Interactive installation
me@~: cd /var/www
me@/var/www: cv core:download
Found CMS "Drupal" (/var/www)
The current stable CiviCRM version is 5.7.1. What version should we download?
  [s] Current stable (5.7.1) (default)
  [r] Current release candidate (5.8.beta1)
  [m] Current developmental master (5.9.alpha1)
  [o] Other (Specific version)
>
Where should CiviCRM be downloaded to?
  [1] /var/www/sites/all/modules/civicrm (default)
  [o] Other (Specify)
>
Downloading "https://download.civicrm.org/civicrm-5.7.1-drupal.tar.gz" 
to "/var/www/sites/all/modules/civicrm"

## Story 2 -- Automated installation. This is the same, but we run in a
## scripted/non-interactive context, so we use the defaults.
me@~: cd /var/www
me@/var/www: cv core:download -n
Found CMS "Drupal" (/var/www)
Downloading "https://download.civicrm.org/civicrm-5.7.1-drupal.tar.gz" 
to "/var/www/sites/all/modules/civicrm"

## Story 3 -- "Bare" download in current folder -- Does not autodetect or bootstrap CMS.
## All decisions based on command-line inputs. This is analogous to `cv ext:download -b`.
me@~: cv core:download -b --ver=5.7.1 --cms=Drupal

## Story 4 -- "Bare" download in alternate folder
me@~: cv core:download -b --to=$PWD/profiles/foobar/modules/civicrm --ver=5.7.1 --cms=Drupal

## Story 5 -- Automated installation... but with some non-default options
me@~: cd /var/www
me@/var/www: cv core:download --to=$PWD/profiles/foobar/modules/civicrm

We don't have to implement every user-story... but if the stories we do support the eventual contract, then it might be easier to make incremental progress?

[michaelmcandrew]

Thanks for your thoughts.

OK - so it has been thought about to some extent already - just within a different context, which i guess is why I didn't find it.

I wonder if it's better to have core:download and upgrade:dl as separate commands -- or to combine them?

From a usability perspective, for people that are looking to create initial set up workflows, having a command called core:download (even if it shares underlying implementation with upgrade:dl) would be good IMO

FWIW I used release rather than ver/version since version is kind of reserved in cli tools.

The stories look good. We probably want to make downloading l10n optional (and defaulted to on?)

Seems like upgrade is some combination of this command with a "force" to overwrite and some form of "auto detect location of existing CiviCRM download".

Seems like abstracting the download stage is also good. I didn't look at how the extensions downloading (and caching?) operates but maybe some refactoring of that would be good.

Story 3 looks like the easiest one to start with.

[michaelmcandrew]

PS. the context is that I am creating Dockerfiles for CiviCRM on WordPress and Drupal and I want commands to insert round about here:

https://github.com/michaelmcandrew/civicrm-docker/blob/master/wordpress/Dockerfile#L10
https://github.com/michaelmcandrew/civicrm-docker/blob/master/drupal/Dockerfile#L11

i.e. a non interactive scripted environment with uninstalled CMSes, which is the reason that I would be interested in starting with 3.

Story 3 looks like the easiest one to start with.

I probably should have said, looks like the most useful story for me :)

[michaelmcandrew]

fyi if you look at those files now, you will see that I have followed a different approach for now...

https://github.com/michaelmcandrew/civicrm-docker/blob/16ad299983d0fc5672b44bfbaba9bfd9bb783590/wordpress/Dockerfile#L16-L24

and

https://github.com/michaelmcandrew/civicrm-docker/blob/16ad299983d0fc5672b44bfbaba9bfd9bb783590/drupal/Dockerfile#L21-L29