Math operators: stretchy or not stretchy

[Omikhleia]

Let's take again the Schwinger-Dyson equation, from some Joe's test page which gives both a TeX formula and a MathML one...

Expectation:

Current SILE results (SILE 0.15.5 + most of my previous math-related PRs, notably #2146 slightly extended)

MathML:

TeX-like = the TeX formula with the \left and \right commands set to nothing.

Analysis

On the bright side, we are quite close in both cases. But the some operators are not stretched (in both cases); and (in the TeX-like case) some are stretched while they shouldn't.

• MathML:

  • The operator that are stretched are those having an explicit "symbolDefaults" override in packages/math/unicode-symbols.lua (a small list)
    • The square brackets are there too, but stretching occurs within the context of the surrounding mrow, which is expected.
    • The vertical bar is here too but is not marked as stretchy
    • The angle brackets are missing
  • So our problem concerns the angle brackets and the vertical bar... and a large number of other symbols actually. The MathML guys were kind enough to provide a non-normative but good list of such symbols, see especially the table in §C.4....

• TeX-like

  • The issue is very similar and part of the solution is the same...
  • ... But with a few extra considerations if we target a reasonable support of TeX math
    • In TeX many of these symbols are not stretchy by default, and this is where \left and \right come into play.
    • TeX knows at parsing (i.e. using something similar to our "symbols" table) which atoms are mathopen/mathclose etc., and can use that to wrap the content in a box... And Pandoc knows that too, actually.
      • Running with -d texmath, we see that we converted the TeX to ...\mi{F}\mo{[}\mo{ϕ}\mo{]}...
        i.e. <mi>F</mi><mo>[</mo><mo>ϕ</mo><mo>]</mo>
      • Compare with Pandoc:
        <mi>F</mi><mrow><mo stretchy="true" form="prefix">[</mo><mi>ϕ</mi><mo stretchy="true" form="postfix">]</mo></mrow> -- see, besides the correct handling of stretchy etc., the extra <mrow> there?

Conclusion

To be, or not to be, that is the question:
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles (...)

I'd say it's a bug (= incompleteness of the implementation but leading to an unexpected output)

The steps would be clear:

• Properly implement the suggested "pre-defined" table from MathML app. C.4 --> see feat(math): Support the MathML operator dictionary and many TeX-like aliases #2167
• Also support explicit stretchy=bool on MathML elements... --> see Math more fixes and features #2151
• Improve our TeX-like parser to be closer to what TeX/Pandoc etc. understand. Without being exhaustive or fully right (it's just a quick analysis on this point:
  • Standalone atoms are forced to stretchy=false
  • Paired of open/close atom are forced in a mrow (and are possibly stretchy=true there) --> see Math more fixes and features #2151
  • Paired \left and \right might be considered too (and this does also affect the stretchy flag, and in addition allows for some of the left or right symbols to be something other than a natural mathopen/mathclose pair.) --> see Support TeX-like left..right construct #2179

EDIT: Not discussed here, but since it also occurs in the above examples, there's another topic on unary/binary operators (#1642)...

EDIT: The aforementioned MathML table also has indications for spacing around operators (which we implement with an hard-coded thin/med/thick TeX-inspired logic. It's not the topic here, but spacing issue can also be noticed in the above examples (... "How many issues do you want me to open?" 🌵 )

[Omikhleia]

Standalone atoms are forced to stretchy=false

Well, I don't think we should do this...
Right, TeX doesn't stretch parentheses on delimiters without a \left...\right construct.
But it does seem to me, reading carefully The TeXbook, that's is merely because TeX is not trying to re-pair (pun intended) these delimiters. It would require, as we do it, checking the inner content (= equivalent to some kind on look-ahead if it was done in the parsing syntax), and it's not that obvious to do with raw TeX construct or under the memory-limited constraints Knuth faced at the time... Our Lua implementation builds a lot of intermediary lists, loops over them at several places, so heh. We don't really have those constraints.
It's a small deviation vs. what TeX would generate for the same markup, but the result was bad anyhow, with too small delimiters (hence the introduction by Knuth of the \left...\right construct, very likely...). Our way of doing it is not only acceptable, it might even be better (... and you wouldn't see me saying this kind of things very often, I hold Knuth in high regards).

[Omikhleia]

See above comment - Considered closed.