Discuss tooling (i.e. Gulp) universal needs

[sghoweri]

@evanmwillhite originally wrote:

There's a lot of cross-over in terms of what projects like Pattern Lab Starter and Emulsify are trying to accomplish at a base-level when it comes to Sass, JS and Pattern Lab. Some of these are specific to company needs, but likely some of them can be put into a project we all use/support.

[EvanLovely]

I see this as taking a lot of what the Gulp theme tooling (i.e. Pattern Lab Starter uses p2-theme-core has in it and start to extract out lots of the small, focused functionality into dedicated npm modules.

I've got the functionality that turns scss files into json files up here for starter: https://github.com/EvanLovely/scsstojson (PS found out that someone wrapped it and turned it into a Pattern Lab node plugin today! https://github.com/cloudsociety/plugin-node-scsstojson)

We should identify what more of these types of thing are and then use those in p2-theme-core and emulsify-gulp

[a-fro]

I'm interested in simplifying/automating the pattern documentation and scaffolding for the "container" components and the variables that get mapped in the Drupal theme templates and passed to the presentational components.

My initial suggestion would be to add a step to the yeoman generator that would allow the developer to define a comma deleted list of variables for the pattern, add those as twig variables in the component file, and then add them also to the generated md file (assuming they've chosen to include the md file).

My assumption here is that it would make the step of creating container templates easier since a dev could then use the documentation in PatternLab. There could also be a scaffolding task to create a container template that uses either the data in the md file, or scans the component twig file for variable names and adds them to the file. Thoughts? I'm happy to work on a PR, though I realize that some of this might move from pattern-lab-starter to p2-theme-core, or a new pattern-lab-drupal-core repo.

[finteractive]

@EvanLovely - Regarding re-use of gulp/node tasks. Would we need to have some kind of "core" gulp task runner and an optional set of dependencies for specific tasks each in their own package?

Perhaps something like this file could be the core component?
https://github.com/fosterinteractive/mainspring/blob/master/gulpfile.js

• uses the require-dir to load all our optional gulp/node tasks
• allows a gulp.js.local to override pre-configured variables in npm packages

Then each specific Gulp Node Tasks is it's own package (kind of like these)

Example optional tasks that might be useful across a wide range of projects:

• PL complile
• Sass compiling / Sourcemaps / Post CSS
• Browsersync (Self contained server for PL)
• Browsersync (Local Proxy to allow it work in Drupal)
• SVG sprites (as Symbols tags for externally loaded Use tag loading)
• SVG inline backgrounds (with rendered CSS like in Emulsify)
• Image (lossless minifications)
• JS (ES6)
• JS concatinating / Minify
• Linting SCSS
• Linting JS / ES6

[sghoweri]

lol. @EvanLovely @finteractive beat me to the punch.

As far as the Gulp / task runner / universal needs initiative is concerned, what I'd strongly recommend is that we focus first and foremost (in this order):

Proposed Focuses:

A. Generic Gulp Scaffolding: a generic, simple, extendable setup / bootstrap code for Gulp that primarily just handles loading / orchestrating standalone Gulp tasks -- tasks which could be local to your codebase and/or mixed with community contributed npm-installed dependencies. At least this would give us a solid starting point we can work off of. Once we have that in place, THEN I think it'll make a helluva lot more sense to start looking into normalizing some of the more common Gulp tasks you might find in Drupal-friendly Pattern Lab projects. (Examples of this below).

B. Example Task "Packaging": provide an example way to package up / distribute your Gulp tasks. One great example of how this might be done is via a tool called Lerna (which can help organize, version and publish semi standalone parts of your code)

C. Normalized Gulp Task Examples: examples of actual Gulp tasks themselves - particularly the ones that wouldn't make sense to use if you were only using Gulp but not Pattern Lab.

The beauty of all this is we could focus on all three of these without necessarily requiring three separate standalone repos -- one mono-repo to maintain with separate consumables for end-users depending on what they need.

Why not just immediately jump right into the meaty Gulp tasks?

Just like CSS and JS frameworks, I feel like there's a LOT of differing views on what should go into your Gulp tasks + what tech stack is the best fit for your organization / your projects / your team's needs. Oh and how the heck we want to ship those parts independently from the rest of our codebase. It's complicated and everybody's got an opinion on how certain things should get handled.

That said, there's still probably a good bit that can be normalized / generalized in some of the most common tasks which we could add to a curated set of common task examples in addition to the common Gulp core + task "packaging mentioned above.

If we solve for these things separately (vs a single monolithic opinionated Gulp config / task) might at least provide some value for those that do have an opinion (especially a differing opinion) vs some one-size-fits-all solution.

Examples of Gulp Scaffolding / Standardized Task & Config Loading + Overrides

In terms of what I'm imagining might go in this sort of "extendable" gulp starting point, here's a couple examples of what I sort of had in mind to get the ball rolling:

Emulsify Gulp: lines 1 through 32

P2 Theme Core: https://github.com/phase2/p2-theme-core/blob/master/index.js

"PegaKit" Build Tools Config: https://github.com/pega-digital/patternkit-build-tools/blob/master/gulpfile.babel.js

Basically, use Gulp 4 to load up a bunch of separate tasks + extend config defaults / overrides.

[EvanLovely]

I agree with a lot of these thoughts. Over time of adding more and more functionality to the Gulp engine that powers Pattern Lab Starter, p2-theme-core I felt the desire to break it apart into modular parts. Then you could chose things like Libsass or Post CSS in your particular install without having to include dependencies for all.

I have an idea on how this could all come together in something that I hoped would supersede p2-theme-core one day: theme-core: I've already got the npm name snagged and it intentionally was hoped to work with the generator-theme name I snagged, mentioned and discussed in #5, that would let us use yo theme. The theme-core repo could become the monorepo that @sghoweri is suggesting that would let us npm install theme-core--libsass -S or something like that. I would be willing to contribute both the generator-theme and theme-core repos and npm names to this org if others are done for this approach.

As exciting as that is, I think we have one question to answer as a group that I already have opinions on:

How much to rely on the PHP tooling in Pattern Lab?

It has a generate, watch, and serve command already.... though I personally only use the generate command. The serve doesn't have the option to serve from a few directories up, so when I reference ../../../../dest/style.css it can't get to it, and the Browser Sync tool I use for serving brings better goodies like live reload and style injection.

So, do we decide to not suggest the Pattern Lab tools? If not, then are we good with choosing Gulp.

My suggestion is to only rely on the PL generate command, and recommend Gulp.

[sghoweri]

How much to rely on the PHP tooling in Pattern Lab?

I'd love to hear everyone else's thoughts but here's my (potentially controversial) opinion: I'm really not a fan of having Pattern Lab's built-in PHP tooling doing anything but compiling pattern templates and handling the internal data structure of those patterns. Plain and simple.

I've been using Gulp to handle all other built-in PHP tooling for at least a couple years now so everything can stay up to day when dependencies get updated or new options get released or when I feel like trying out a different solution.

Pattern Lab commands I actually use:

php core/console --generate (+ with the --patternsonly flag)

Commands I never use:

php core/console --config
php core/console --export    // written equivelents since this is half-baked imho
php core/console --server .  // never needed - use browsersync
php core/console --version   // never needed
php core/console --watch     // never use since it doesn't handle errors like gulp plumber. Use gulp watch + browsersync instead
php core/console --help        // Never use - use gulp help instead
php core/console --starterkit // Never once a project exists

**(used to but now have better ways of handling this)

[taivu]

As a Front Ender, I think its a good idea to rely on only the generate command for PL in php. I prefer the gulp/browsersync route. I also think going this way would remove alot of complexity for onboarding new FE devs into the workflow, who may or may be comfortable with PHP.

[evanmwillhite]

Trying to see where we stand on this. It sounds like we might have some agreement around a couple of things that would be in this:

1. Core Gulp Tasks with local config option - @finteractive recommended https://github.com/fosterinteractive/mainspring/blob/master/gulpfile.js and Emulsify has a more basic one here as well as some documentation around it here
2. Gulp Tasks to compile Pattern Lab
3. Additions such as the scsstojson plugin

Is this correct, and if so, should we go ahead and get signoff from everybody and break this out into 3 separate tasks?

[sghoweri]

@evanmwillhite I think that's pretty close. Have a couple things I want to add when I'm back at a computer + a couple implementation details, but that's the gist of it.

[sghoweri]

@EvanLovely @evanmwillhite @a-fro @taivu @finteractive How's this sound?

1. Core Gulp Tasks with local config option - @finteractive recommended https://github.com/fosterinteractive/mainspring/blob/master/gulpfile.js and Emulsify has a more basic one here as well as some documentation around it here

Edit: 1. Core Gulp 4 based gulpfile.babel.js that pulls in external Gulp-help friendly tasks + a default gulp-config.json config, docs with an example to extend via local.gulp-config.js, and a cooresponding package.json with only the required dependencies for this piece.

@finteractive recommended https://github.com/fosterinteractive/mainspring/blob/master/gulpfile.js and Emulsify has a more basic one here as well as some documentation around it here

2. Gulp Tasks to compile Pattern Lab

Edit: 2. Separate core tasks to compile PL, handle errors, watch files (Twig, json, yaml, markdown) for changes, and serve up Pattern Lab via browsersync.

These should be done in such a way so that they can also be used as examples when creating other Gulp tasks.

We should also handle this using Lerna so individual tasks can be created as a monorepo but versioned and included independently.

3. Additions such as the scsstojson plugin

Also using Lerna. 👍🏼

[evanmwillhite]

@sghoweri this sounds great to me. I haven't used Lerna, but at first glance it seems like it might be a great fit for this kind of thing (and straightforward to use).

[sghoweri]

@EvanLovely anything to add / thoughts?

Can we sign off and start breaking this out into smaller tasks?

[EvanLovely]

I'd like to own kicking this all off; I've got a pretty clear vision on how this could best come together and a test mono-repo already working locally. Once I get the basics set up, I'd love some help setting up other tasks and fleshing out edge cases from people.

[evanmwillhite]

Thanks @EvanLovely for taking this on!

[EvanLovely]

OK, I've got this mainly done! Anyone want to review this? EvanLovely/theme-tools#1

[legostud]

I'm a bit late to the conversation, but thought I would share the gulp solution we're using for Mass.gov's Mayflower Style Library. The gulp bundle was built by one of my co-workers here at Velir.com and is super flexible. The patternLab task however is a bit rough since I haven't found a good way to trigger browsersync to refresh when it's complete.

https://github.com/massgov/mayflower/tree/dev/styleguide/tools/gulp

[EvanLovely]

@legostud I took a look and I have a possible fix for you in a PR. Also, you should consider looking at the tools I made for this that are working very well for me: https://github.com/theme-tools/theme-tools

[cybtachyon]

Definitely a +1 for Gulp.

The only tidbit I'd like to add is:
I'd like to recommend we keep the tooling separate from any pattern libraries we ship by default.

From experience, tooling should be either a composer dependency or (if a custom setup like Gulp) part of the repo, and any shared pattern library(ies) should be separate repos.