Invalid HTML tag in hover response

[jwortmann]

I tried this language server in Sublime Text and I like it so far.

But I noticed that popups on mouse hover look broken, because the hover response from the server contains an unclosed and invalid HTML-tag <info:latex2e# [...] >. Sublime Text uses a custom HTML rendering engine for its hover popups, and so the Markdown content has to be converted to HTML by the client. Since HTML tags (including custom tags) are valid within Markdown, they are preserved when converting to HTML. However, Sublime Text can't handle unclosed custom tags, and afaik : and # are also invalid characters in tag names (see https://github.github.com/gfm/#tag-name).

Here is an example of the server response when I hover over the \LaTeX macro:

{
    "contents": {
        "value": "`\\LaTeX`\n\n# Documentation\n\n`\\LaTeX` is defined in the [latex](https://www.ctan.org/pkg/latex) package.\n\n- <info:latex2e#Text symbols>\n- [A (Not So) Short Introduction to LaTeX 2ε](https://www.tug.org/texlive/Contents/live/texmf-dist/doc/latex/lshort-english/lshort.pdf)\n- [LaTeX 2ε for authors](file://C:/Users/jwortmann/AppData/Roaming/MiKTeX/2.9/doc/latex/base/usrguide.pdf)\n- [LaTeX 2ε for class and package writers](file://C:/Users/jwortmann/AppData/Roaming/MiKTeX/2.9/doc/latex/base/clsguide.pdf)",
        "kind": "markdown"
    }
}

The popup cuts off after the first list item:

It would be nice if this HTML tag could be removed from the Markdown content, assuming it serves no purpose. In case it should be rendered literally, please consider to replace < by <.

[astoff]

I'm happy to find a way to accommodate this, but arguably our behavior is correct.

The snippet <info:latex2e#Text symbols> is not a tag. It's put there intentionally and is a hyperlink to an Info manual. If you have a local copy of the manual, you could set up your editor to take you to the relevant entry.

The angle brackets syntax for hyperlinks is standard markdown, see this. Could it be that the space is confusing Sublime? You could test this by hovering over the \arraycolsep command, which should display <info:latex2e#array>. (If this also also breaks Sublime, then I bet <mailto:[email protected]> will probably also break it, which you might agree is not really a good thing 😃)

[astoff]

PS: Did you need to do anything nontrivial to configure the server on Sublime? I'd be happy to add some instructions to the README or a link to some other place.

[jwortmann]

Oh, I totally forgot about autolinks in Markdown. I agree that it should be valid and recognized as a hyperlink. I think the space in the Text symbols example still isn't allowed though, see Example 611 in the GFM docs ("Spaces are not allowed in autolinks"), and it should probably be fixed in the server.

But the space is not the reason why the link conversion fails. The Sublime client uses a mdpopups dependency to convert Markdown to HTML, which in turn uses Python-Markdown. It seems that Python-Markdown can't handle autolinks with arbitrary schemes:

pip install markdown

>>> import markdown
>>> markdown.markdown('<http://foo.bar.baz>')
'<p><a href="http://foo.bar.baz">http://foo.bar.baz</a></p>'
>>> markdown.markdown('<info:latex2e#array>')
'<p><info:latex2e#array></p>'

I found the related issue Python-Markdown/markdown#1244, and from reading that it seems that preserving autolinks with unknown schemes as raw HTML is the intended behavior 🙄.

So if we want to mitigate the broken popups in Sublime Text, possible solutions could be

1. Parse and "patch" the Markdown content in the client
2. Pressure the mdpopups dependency to use another Markdown implementation, rather than Python-Markdown (Switch to CommonMark facelessuser/sublime-markdown-popups#103)
3. Use raw HTML links like <a href="info:latex2e#array">info:latex2e#array</a> instead of autolinks in the server response
4. Move the autolink to the very last list item

But since this is not really a bug in the server, feel free to close this issue.

---

PS: Did you need to do anything nontrivial to configure the server on Sublime? I'd be happy to add some instructions to the README or a link to some other place.

I use the following configuration for the LSP client in Sublime Text (LSP.sublime-settings):

{
    "clients": {
        "digestiff": {
            "enabled": true,
            "command": ["digestif"],
            "selector": "text.tex.latex"
        },
    }
}

Sublime Text prevents autocomplete in LaTeX files by default, and in order to have autocompletion to work you need to manually adjust the "auto_complete_selector" setting in Sublime's settings (Preferences.sublime-settings). I currently use

{
    "auto_complete_selector": "meta.tag, source - comment - string.quoted.double.block - string.quoted.single.block - string.unquoted.heredoc, text.tex constant.other.citation, text.tex constant.other.reference, text.tex variable.parameter.function, text.tex support.function",
}

which should enable completions for the relevant situations in LaTeX (commands, environment names, etc., but not plain text).

The installation of a recent Lua version and luarocks on Windows was a bit of pain though.

For instructions in the README, you could add

### Sublime Text with the [LSP](https://packagecontrol.io/packages/LSP) package

See instructions [here](https://lsp.sublimetext.io/language_servers/#digestif).

where the latter link does not yet exist, but I can add it later. (The reason is that there might possibly be a LSP-digestif helper package for Sublime Text some time in the future, which could simplify installation & updates. But probably not in the short term, because Sublime's LSP framework doesn't have automated support for Lua server implementations at the moment.)

[astoff]

Great, thanks for the pointers. Then I think I'll change the markdown link to [Info node (latex2e)array](info:latex2e#array) , and possibly something more user-friendly in the future (maybe "resolve" the info link to an http link if the document is not available locally). AFAICT this one is converted correctly by Python markdown.

As to Sublime, if you want to add the info the non-existing link, then I'll add your suggestion to the README. Just let me know!

[jwortmann]

Thanks, yes the regular Markdown link format should be sufficient.

By the way, the instructions link is ready now: https://lsp.sublimetext.io/language_servers/#digestif

[astoff]

Okay, I pushed some changes to the info link rendering, let me know if it improves the situation.

Improve compatibility of info hyperlionks with

Thanks a lot, I'll add a link to the readme soon.