Broken formatting in chat link

[tripleee]

My comment on https://metasmoke.erwaysoftware.com/post/383440 is oddly mixing HTML and Markdown formatting, where the latter is displayed verbatim.

Plagiarized from ethereum.stackexchange.com/questions/134094/… with spam link added. Please flag accordingly. — tripleee 27 secs ago – from a [chat message](https‍://chat.stackexchange.com/transcript/message/61837565) by [tripleee](https‍://chat.stackexchange.com/users/62118/tripleee)

Attempting to edit the message reveals that the working links are HTML and the non-working ones are Markdown.

[makyen]

I managed to mis-click and delete the comment on MS, and also mess up the onebox in the source chat message of your deleted comment on Ethereum. I, unfortunately, didn't check to see if your original comment on Ethereum had been deleted prior to attempting to recreate the MS comment. I'm sorry about that. I've returned the chat message to a reasonable semblance of what it was, but using Markdown. I was attempting to re-create the MS comment which I'd accidentally deleted.

It's strange that it's not accepting Markdown in this, but does accept Markdown in other such comments. After my unfortunate misadventure attempting to return things to the way they were, it's clear that the issue is when forwarding a oneboxed message. A onebox is moderately complex HTML, which MS somewhat renders. However, it doesn't render the Markdown that was also there.

Having the new attribution in Markdown works under other circumstances. We can try changing the attribution to being in HTML, rather than Markdown to see if that works for both oneboxes and Markdown based chat replies.

[makyen]

Another example of the issue for another forwarded onebox: https://metasmoke.erwaysoftware.com/post/383442

The source text of the comment on MS is:

<div class="onebox ob-message"><a rel="noopener noreferrer" class="roomname" href="/transcript/message/61837559#61837559">in Charcoal Test, <span title="2022-08-22 06:54:55Z">7 mins ago</span></a>, by <span class="user-name">Halflife</span> <br/><div class="quote">halflife:2022-08-22 06:54:39,491:<code>http://dlj.bz/7P3GpM</code> redirects to <code>https://2jewellery.com/</code></div></div> – from a [chat message](https://chat.stackexchange.com/transcript/message/61837598) by [tripleee](https://chat.stackexchange.com/users/62118/tripleee)

Which renders in HTML on MS as:

<a href="/transcript/message/61837559#61837559">in Charcoal Test, 7 mins ago</a>, by Halflife <br>halflife:2022-08-22 06:54:39,491:<code>http://dlj.bz/7P3GpM</code> redirects to <code>https://2jewellery.com/</code> – from a [chat message](https://chat.stackexchange.com/transcript/message/61837598) by [tripleee](https://chat.stackexchange.com/users/62118/tripleee)

So, MS strips out quite a bit of the HTML when rendering the comment and doesn't process the Markdown.

It looks like using HTML for the attribution will work in these cases, and should work in the cases where it's mixed with Markdown from forwarded non-oneboxed chat messages.

[makyen]

I updated the code to have the attribution in HTML instead of Markdown and did some testing here, which produced some comments on this MS post.

While the attribution is now displayed for a similar onebox of a chat comment, there was another issue which came up. This chat message didn't result in an MS comment being posted. It's unclear at this time why that is the case, or if a similar attempt would have produced an MS comment prior to this most recent change, or prior to adding the code for attribution.

[makyen]

The new issue is that

<div class="quote">This question is off-topic because it is not a practical programming problem. Thus, it's not within the scope of questions appropriate for this site, as defined in <a href="//stackoverflow.com/help/on-topic">What topics can I ask about here?</a> Please also see: <a href="//stackoverflow.com/help/dont-ask">What types of questions should I avoid asking?</a> You may be able to get help on <a href="//stackexchange.com/sites#name">another Stack Exchange site</a>. However, be sure to read the on-topic page for the site you select prior to posting. — <a rel="noopener noreferrer" href="https://stackoverflow.com/users/3773011/makyen">Makyen ♦</a> <a rel="noopener noreferrer" href="https://stackoverflow.com/questions/73158447/proper-vlan-isolation-for-exposed-server#comment129207586_73158447"><span title="2022-07-28 20:37:27Z" class="relativetime">Jul 28 at 20:37</span></a></div>

can't be added as a comment. It doesn't have anything to do with the new code. Trying to manually add that as a comment doesn't work either. The issue is that the ♦ causes the add comment operation to produce a Mysql2::Error: Incorrect string value error.

[teward]

Is the DB on MS using utf-8 collation or an ascii collation? UTF8 not being supported might be one of the string value errors causes.

[makyen]

Yes, what range of characters are supported in that field is almost certainly the issue. We can do any of:

• Solve in SD
  • Strip unsupported characters, perhaps with some specific ones, like ♦, converted into HTML entities
  • Convert all unsupported characters into HTML entities
• Solve in MS
  • Change the definition for that field in the database such that it accepts a wider range of characters.
  • Strip unsupported characters prior to adding the entry to, or updating the entry in, the database.
  • Convert unsupported characters to HTML entities.

We've had similar issues as this with other fields. In general, those have been resolved by changing the definition of the field in the database.

[makyen]

I've added a hack to SD to allow forwarding a oneboxed moderator comment into an MS comment by replying to a report in chat (example in this post below the comment with ♦ in code format). We should still implement a general solution.

[makyen]

I've created Issue #946 which covers the remaining general issue of Unicode characters in MS comments. As far as I'm aware, the problem which this issue reported has been addressed. It's that subsequent investigation highlighted another existing issue which has been present for a long time, but I don't recall having an Issue which addressed it.