Expanding scope of Picture Plugin or making it composable

[hpfast]

Hello @oscarotero !
I have been studying the picture plugin the last few days, and come to the conclusion (correct me if I'm wrong) that it supports one of multiple Responsive Images use cases:

a. resolution-switching, i.e. telling the browser about different sizes of images to use with media queries (not supported; needs an <img> with srcset and sizes but the plugin outputs <picture> elements)
b. multiple filetypes and pixel densities to allow the browser to choose the best one (supported)
c. art direction, i.e. directing the browser to use different versions of images with media queries, e.g. a cropped version on narrow screens (not supported; needs a media attribute on the <source> element in <picture> but the plugin doesn't support that);

(see MDN for an overview of these use-cases).

What I did for my immediate use-case ((a) above) was create a new plugin copying lots of the code, but only outputting <img>s. I also had to add a shared data file as well as another processing step to add the sizes property to images before my plugin ran and to handle some conditional cases. All in all, it was quite straightforward to adapt, and the flexibility of Lume's processing mechanism is very nice.

However, there is a bunch of helper code in the Picture plugin which it would be a pity to re-implement (creating the sources and transforms and such).

I was wondering:

1. would it be an idea to extend the Picture plugin to handle more use-cases? I would be willing to help with that, as time allowed.
2. alternatively, could some of the internal functions and interfaces get exported, so that another plugin for resolution switching could make use of that?

Cheers!
Hans

[oscarotero]

Hi @hpfast

The picture plugin outputs <picture> or <img>, it depends on the number of <source> elements needed. If the image is required only in one format, instead of creating a <picture> with just one <source>, it outputs a <img> with srcset attribute. Here's the code.

In short, the plugin only handles the combination of different formats and pixel densities. It doesn't handle media queries because it would be complicated and tricky (there are infinite combinations).

But I had a similar request the other day on discord and I'm thinking on implementing support for <source> elements in order to be able to do this:

<picture>
    <source srcset="/img/img.jpg" media="(min-width: 1920px)" transform-images="webp avif">
    <source srcset="/img/img.jpg" media="(min-width: 1280px)" transform-images="webp avif 1280">
    <source srcset="/img/img.jpg" media="(min-width: 480px)" transform-images="webp avif 480">
    <img alt="…" src="/img/img.jpg" transform-images="webp">
</picture>

Supporting this, you could create a <source> for each different media query you want to support (which would cover the art direction use case) and the plugin would complete it with the remaining source elements to support formats and/or pixel density.

What do you think? If you like the idea and want ho help with that, I'll appreciate a lot.

[hpfast]

Hi,
I thought I understood the various use-cases fully, but I've been reading a bit more and filling in some nuances. I now understand better your choice of focusing on the <picture> element by default.

If the image is required only in one format, ... it outputs a <img> with srcset attribute. Here's the code.

hmm. I think I tried this and it still was producing <picture>s. I'll have to see if I can reproduce that situation, as if this would work it would save me some trouble (that's what I thought would work initially).

I like your suggestion, and I would like to make an attempt at this.

Supporting this, you could create a <source> for each different media query you want to support

Is your code example the suggested input or output?

[The plugin] doesn't handle media queries because it would be complicated and tricky (there are infinite combinations).

I was wondering about the following approach: instead of one property image-transform, use several, one for each 'use-case'. The plugin could then run a slightly different pipeline for each set it encounters. This might make certain edge-cases hard to achieve, but that was already the case (hence my question) and given the infinite problem space, some simplification might be justified.

[oscarotero]

Is your code example the suggested input or output?

It's input. The idea is to be able to write this:

<picture>
    <source srcset="/img/img.jpg" media="(min-width: 1920px)" transform-images="webp avif">
    <source srcset="/img/img.jpg" media="(min-width: 1280px)" transform-images="webp avif 1280">
    <source srcset="/img/img.jpg" media="(min-width: 480px)" transform-images="webp avif 480">
    <img alt="…" src="/img/img.jpg" transform-images="webp">
</picture>

and the plugin outputs this:

<picture>
    <source srcset="/img/img.avif" type="image/avif" media="(min-width: 1920px)">
    <source srcset="/img/img.webp" type="image/webp" media="(min-width: 1920px)">
    <source srcset="/img/img-1280.avif" type="image/avif" media="(min-width: 1280px)">
    <source srcset="/img/img-1280.webp" type="image/webp" media="(min-width: 1280px)">
    <source srcset="/img/img-480.avif" type="image/avif" media="(min-width: 480px)">
    <source srcset="/img/img-480.webp" type="image/webp" media="(min-width: 480px)">
    <img alt="…" src="/img/img.webp">
</picture>

I was wondering about the following approach: instead of one property image-transform, use several, one for each 'use-case'.

I'm not sure to understand. Can you put a syntax example?

[hpfast]

now the Picture plugin selects images with attribute image-transform, (or in a container with that attribute).

You could add several, to support the various use-cases, with a different execution flow for each one:

<img art-direction="...">
<img image-types="...">
<img resolution-switching="...">

But now that I think about it, I think this wouldn't work. There's probably too much overlap in desired functionality, e.g. you might also want to specify multiple image types in the art-direction case.

To elaborate on your idea instead, what about this input:

<picture transform-images>
    <source <!-- optional 'srcset', 'media', 'sizes' and 'formats' attributes.
        'formats' replaces 'transform-images' in previous example -->
    >
    <source>
    ....
    <img alt="..." src="/img/img.webp">
</picture>

Reading MDN, the browser is smart enough to handle various combinations of srcset, media, and sizes. So wouldn't it be ok to just pass everything through that the user specifies? For example, sizes is 'Allowed if the parent of <source> is <picture>' but 'will take effect only if width descriptors are provided with srcset, not pixel density descriptors.'

P.S. I'm happy to continue back-and-forth design, but I'm also happy to think on it and come back with a fleshed-out proposal if you don't have the time.

[oscarotero]

Sizes attribute is already supported (see PR here: #482)

I like the idea of making srcset optional in the <source> (taking the value of the <img> as default). So the input code of my example would be simpler:

<picture>
    <source media="(min-width: 1920px)" transform-images="webp avif">
    <source media="(min-width: 1280px)" transform-images="webp avif 1280">
    <source media="(min-width: 480px)" transform-images="webp avif 480">
    <img alt="…" src="/img/img.jpg">
</picture>

I rather only one attribute (transform-images) instead of adding additional attributes because it's clearer, specially if you want to override them.

This attribute can handle values of different types that the plugin understand. For example in avif webp 300@2, avif and webp are formats and 300 is a image size (adding @2 to create a version for 2x density). Maybe we can include [number]w syntax to generate images of a specific width, without taking into account the density. I.e. avif webp 300w.

[hpfast]

The more I study it, the more it seems like my use-case should already be supported. Either I've found a bug or I'm doing something wrong. I'd like to extract this from my project and make an isolated test case, alongside a reference implementation of <picture>​ and <img>​, but I need to do some other things first. I'll get back with this soon

…

On Tuesday, July 30th, 2024 at 12:49, Óscar Otero ***@***.***> wrote: Sizes attribute is already supported (see PR here: [#482](#482)) I like the idea of making srcset optional in the <source> (taking the value of the <img> as default). So the input code of my example would be simpler: < picture > < source media =" (min-width: 1920px) " transform-images =" webp avif " > < source media =" (min-width: 1280px) " transform-images =" webp avif 1280 " > < source media =" (min-width: 480px) " transform-images =" webp avif 480 " > < img alt =" … " src =" /img/img.jpg " > </ picture > I rather only one attribute (transform-images) instead of adding additional attributes because it's clearer, specially if you want to override them. This attribute can handle values of different types that the plugin understand. For example in avif webp ***@***.***, avif and webp are formats and 300 is a image size (adding @2 to create a version for 2x density). Maybe we can include [number]w syntax to generate images of a specific width, without taking into account the density. I.e. avif webp 300w. — Reply to this email directly, [view it on GitHub](#637 (reply in thread)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AA35I2W6KD6W36UEZRVMVQDZO5VUHAVCNFSM6AAAAABLUDM2Y6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMJYHE2DGMI). You are receiving this because you were mentioned.Message ID: ***@***.***>

[hpfast]

hi @oscarotero, sorry for the silence. Life has been hectic. I hope to have time for this next week or the first week of september.