Proposal for conditional processing improvements to eXist's HTML Templating Library

[yamahito]

This proposal sets out the case for adding convenience features to eXist's HTML Templating Library which will both simplify conditionally including content in the templates themselves, as well as testing using frameworks such as XQsuite.

Current methods for conditionally including content:

Currently, it is possible to conditionally include content in templates by writing an XQuery function, and including it as a @data-template:

<p data-template="eg:when-link">This is text including a <a data-template="eg:link-href">generated link</a></p>

The conditional logic is then held in an XQuery function, alongside the function that actually generates the content:

declare %templates:wrap function eg:link-href($node, $model){
    if ($model?link)
    then (
        attribute href {$model?link},
        $node/node()
    )
    else ()
};

declare function eg:when-link($node, $model){
    if (exists(eg:link-href($node, $model)))
    then
        element {node-name($node)} {
            $node/@*,
            templates:process($node/node(), $model)
        }
    else ()
};

Note that it is necessary to recurse the templates processing using templates:process in the event that recursion is required. This makes writing tests cumbersome (particularly the first, below):

declare %test:assertEquals('true') function x:test-when-link-true(){
    let $config := map{
        $templates:CONFIG_FN_RESOLVER : function($functionName as xs:string, $arity as xs:int) {
            try {
                function-lookup(xs:QName($functionName), $arity)
            } catch * {
                ()
            }
        },
        $templates:CONFIG_PARAM_RESOLVER : map{}
    }
    let $model := map {
        $templates:CONFIGURATION : $config,
        "link": "https://example.com"
    }
    let $node := <p data-template="eg:when-link">Test</p>
    let $actual := eg:when-link($node, $model)     
    let $expected := $node
    return  if (deep-equal($expected, $actual)) then 'true' else <result><actual>{$actual}</actual><expected>{$expected}</expected></result>  
};

declare %test:assertEmpty function x:test-when-link-false(){
    let $model := map{}
    let $node := <p data-template="eg:when-link">Test</p>
    return eg:when-link($node, $model)
};

declare %test:assertEquals('true') function x:test-link-href-exists(){
    let $model := map{
        "link": "https://example.com"
    }
    let $node := <a>link test</a>
    let $expected := <a href="https://example.com">link test</a>
    let $actual := eg:link-href($node, $model)
    return  if (deep-equal($expected, $actual)) then 'true' else <result><actual>{$actual}</actual><expected>{$expected}</expected></result>  
};

declare %test:assertEmpty function x:test-link-href-empty(){
    let $model := map{}
    let $node := <a>link test</a>
    return eg:link-href($node, $model)
};

As well as the test for eg:when-link, I include the test for eg:link-href for reasons which will hopefully become obvious later on.

templates:if-parameter-set and templates:if-parameter-unset

The Templating library does include these tools for conditional processing already; the reader why ask, why these aren't sufficient?

There are two reasons:

1. These features only work on page parameters. Often the conditions we want to use are not parameters to the page, but features contained in the $model map, calculated using XQuery, or retrieved directly from the eXist database.
2. The ...parameter-set features are set using the @data-templates attribute; this means that an additional, redundant wrapping element is needed in addition to any content in the template that is populated using XQuery.

Proposed changes

We propose the addition of templating attributes @data-template-use-when and @data-template-use-unless to conditionally include or exclude content.

The attributes cause a function to be called using the same methods as @data-template; however, rather than use the results of the function to update the $model map or HTML page, the effective boolean value is used to control whether or not further processing takes place.

The use-when/use-unless attribute is used independantly of @data-template, so that either or both attributes may be specified on a template element. @data-template is only called where the condition is satisfied.

The example from the last session becomes:

<p data-template-when="eg:link-href">This is text including a <a data-template="eg:link-href">generated link</a></p>

The eg:when-link function is removed, and the testing burden is almost entirely eliminated: the existing testing for eg:link-href is sufficient.

declare %test:assertEquals('true') function x:test-link-href-exists(){
    let $model := map{
        "link": "https://example.com"
    }
    let $node := <a>link test</a>
    let $expected := <a href="https://example.com">link test</a>
    let $actual := eg:link-href($node, $model)
    return  if (deep-equal($expected, $actual)) then 'true' else <result><actual>{$actual}</actual><expected>{$expected}</expected></result>  
};

declare %test:assertEmpty function x:test-link-href-empty(){
    let $model := map{}
    let $node := <a>link test</a>
    return eg:link-href($node, $model)
};

Even if a different condition were needed instead of the function that we are using to populate, the tests for that conditional function would simply need to test the effective boolean value, and not the cumbersome resolver config.

Implementation suggestion

Implementation should be fairly straightforward; we can check for conditional attributes before during templates processing:

declare function templates:process($nodes as node()*, $model as map(*)) {
    let $config := templates:get-configuration($model, "")
    for $node in $nodes
    return
        typeswitch ($node)
            case document-node() return
                for $child in $node/node() return templates:process($child, $model)
            case element() return
                let $dataAttr := $node/@data-template
                let $dataWhen := ($node/@data-template-use-when ! templates:call(., $node, $model)) => fn:boolean()
                let $dataUnless := ($node/@data-template-use-unless ! templates:call(., $node, $model)) => fn:boolean() => fn:not()
                return
                    if (not($dataWhen) and $dataUnless) then ()
                    else
                        if ($dataAttr) then
                            templates:call($dataAttr, $node, $model)
                        else
                            let $instructions := templates:get-instructions($node/@class)
                            return
                                if ($instructions) then
                                    for $instruction in $instructions
                                    return
                                        templates:call($instruction, $node, $model)
                                else
                                    element { node-name($node) } {
                                        $node/@*, for $child in $node/node() return templates:process($child, $model)
                                    }
            default return
                $node
};

Further considerations

If desirable, it should be possible to allow multiple conditions as a space separated list of QNames for each of the proposed attributes. This could allow for some sophisticated mechanisms for choosing layouts based on more than simple binary choices.

[open-collective-bot]

Hey @yamahito 👋,

Thank you for opening an issue. We will get back to you as soon as we can. Have you seen our Open Collective page? Please consider contributing financially to our project. This will help us involve more contributors and get to issues like yours faster.

https://opencollective.com/existdb

We offer priority support for all financial contributors. Don't forget to add priority label once you become one! 😄

[line-o]

This is a valuable addition to the core templating library and will be included in the next major version.

[line-o]

The one thing we need to look at closely: @data-template-* are interpreted as parameters to template functions declared in @data-template. This might have undesirable side-effects when used together and needs therefore be tested thoroughly.
templates:parse-attr (in the next-branch) is the function doing this. We should think about a better test here than

let $key := substring-after(
    local-name($attr), $templates:ATTR_DATA_TEMPLATE || "-")

[line-o]

One alternative approach given that placeholders are expanded from model entries in attributes before the
template function is called:

  <a data-template="my:tpl-func" data-template-use-when="${link}" href="${link}">${text}</a>

[yamahito]

The one thing we need to look at closely: @data-template-* are interpreted as parameters to template functions declared in @data-template. This might have undesirable side-effects when used together and needs therefore be tested thoroughly. templates:parse-attr (in the next-branch) is the function doing this. We should think about a better test here than

let $key := substring-after(
    local-name($attr), $templates:ATTR_DATA_TEMPLATE || "-")

Agreed, or else decide to use a different @data- prefix to avoid the conflict altogether.

[line-o]

@yamahito Yes, do you have a proposal for an alternative?

What I would want to achieve is that filtering out all templating specific attributes is easy and reliable. All the while it is pleasant and understandable to edit those templates.
The more we escape the chatty $templates:ATTR_DATA_TEMPLATE prefix, as is the case with data-target, we pollute data-attribute space and it becomes harder to separate them from those that are application specific and needed at runtime.

[line-o]

Maybe it is safer to use non-HTML5-compliant attributes. If those are filtered out by default we achieve two goals:

• output remains valid HTML5 (for non-debug scenarios)
• we gain the freedom to shorten all templating specific attributes

Example:

Change the parameter attribute prefix to tpl-param- or similar. In this scenario your proposed additions can be expressed as tpl-use-when and tpl-use-unless.

<a tpl-func="my:tpl-func" tpl-use-when="${link}" href="${link}">${text}</a>
<span tpl-use-unless="${link}">${nolink-text}</span>

[yamahito]

If we leave the convention that all data attributes that become a function parameter e.g. x start e.g. data-template-x, perhaps we can add the convention that other templating options take the form data-option-template, e.g. data-when-template and data-unless-template.

My gut feeling is that if we are using non-HTML5-compliant attributes at all, it should be offered as an addition to the data- attributes currently used, as it is a pivot away from the current standard which is now in use.

[line-o]

With the approach to directly use model-properties we need to agree how to handle items that do not have an effective boolean value such as maps, functions and sequences.

[yamahito]

I am not in favour of the use of model properties directly: I think the strength of the templating system is that it refers to xquery functions, and this proposal attempts to build on that.

[line-o]

It is definitely more versatile and allows to specify arbitrary conditions.

[line-o]

I would still propose to add one or maybe two simple tests that are generally useful
Two immediately come to my mind:

<a tpl-func="my:tpl-func" tpl-if-exists="${link}" href="${link}">${text}</a>
<span tpl-if-empty="${link}">${nolink-text}</span>

[yamahito]

My gut feeling is that using non-HTML5-compliant attributes should maybe be offered as an addition to the data- attributes currently used, as it is a pivot away from the current standard which is now in use.

I have updated this paragraph above to clarify that I am reluctant to move away from data attributes - the wording as quoted was unclear!

The best alternative to data attributes, IMO, is to use namespaced elements in the templates XML format, and ensure that these are completely removed by the templating engine when it renders them as HTML...

[yamahito]

I would still propose to add one or maybe two simple tests that are generally useful Two immediately come to my mind:

<a tpl-func="my:tpl-func" tpl-if-exists="${link}" href="${link}">${text}</a>
<span tpl-if-empty="${link}">${nolink-text}</span>

If we had two tests, use-when, use-unless, we could allow either a reference to a function, or an effective boolean value. The third test if-empty or on-empty would then be surplus as it would be synonymous with use-unless.

[line-o]

If we had three tests, use-when, use-unless, we could allow either a reference to a function, or an effective boolean value. if-empty or on-empty would be synonymous with use-unless.

The placeholder (${link} in the above case) is replaced before the template-attribute is evaluated. It will thus be impossible to differentiate between a string value and a function reference.
Just to be sure you mean two tests correct? Otherwise, the third one is missing.

[yamahito]

Just to be sure you mean two tests correct? Otherwise, the third one is missing.

Yes! I had originally included on-empty in my list of three, then I realised that it would be the same as use-unless. I'll edit that.

I take your point about being unable to differentiate between string values and function references. It brings me back to feeling that we should stick to functions that return boolean values, particularly when considering effective boolean values of functions etc. If we allowed specification of arguments by function position as well as by argument name, could we do something like:

<a data-template="my:tpl-func" data-template-use-when="exists" data-arg1-template="${link}" href="${link}">${text}</a>

[line-o]

I like the general idea. Being able to use built-in functions can have great potential.
The necessity to open a separate set of parameter attributes next to the existing ones with data-template-<parameter-name> could be very confusing when trying to understand other peoples templates.

In order to be able to differentiate both the template function parameters from the test function parameters those attributes should clearly communicate what they are for, best be grouped by prefixes. data-arg1-template -> data-template-use-when-argument1.

With a template function/handler/renderer

declare function my:translated-label ($node, $model, $lang) {
  map:put($model, "text", "translated link text")
};

<a data-template-handler="my:translated-label" 
     data-template-handler-argument-lang="en"
     data-template-use-when="exists"
     data-template-use-when-argument-arg="${link}">${text}</a>

data-template-use-when now calls a function with any signature instead of the established pair of $node and $model. This has implications on function retrieval and is harder to communicate.

We are also deviating from your original proposal quite a lot, that was compelling because of its simplicity.
I just saw the added benefit in adding in two of the simplest tests (is this value set?). For the above it would yield

<a data-template-handler="my:translated-label" 
     data-template-handler-argument-lang="en"
     data-template-use-when-exists="${link}">${text}</a>

Since those tests are already part of the templating library, but only as template handling functions communicating this concept would be easier as we can refer to those and explain why it is
beneficial to use those instead. I hope we find a solution that allows us to deprecate all the conditional template handlers now or in the future as they can force you to have surplus markup in your template.

List of current conditional template handlers:

• templates:if-parameter-set
• templates:if-parameter-unset
• templates:if-attribute-set
• templates:if-attribute-unset
• templates:if-model-key-equals
• templates:unless-model-key-equals
• templates:if-module-missing

I oppose opening up data-template-use-when (and -unless) to arbitrary function calls.
The above list can, on the other hand, be a basis to form a set of builtin conditionals to use as the template test function. Some of them will need extra parameters so that data-template-use-when-argument-<name> does seem necessary.

[yamahito]

data-template-use-when now calls a function with any signature instead of the established pair of $node and $model. This has implications on function retrieval and is harder to communicate.

This is a very good point, I think I agree on the other points as well. Thanks for all of your consideration on this.