Provide access to 'smartypantsu' somehow

[webketje]

In the marked implementation here: https://github.com/alphagov/govuk-design-system/blob/main/lib/marked/extension.js#L16-L25,
it became clear to me that users opting out of HTML entity encoding are forced to npm install smartypants separately to have backwards-compatible processing with pre-marked 5.0.0 smartypants option (and as in marked-smartypants-lite).

Perhaps add an option next to config: htmlEntities (default=true) - "HTML-encode special characters", or utf8 (default=false) "Do not HTML-encode special characters"?

Willing to contribute a PR when the direction is clear

[UziTech]

I think a better option would be to add config option to smartypants to output html entities or utf8. That way config is all in one place

[webketje]

True, but having the option is stil better than not having it and at least marked-smartypants can have all its config in one place + such a small effort.

For the users and depending packages of the most popular markdown js package that didn't have the time to upgrade 6 major versions in half a year and who relied on the smartypants option pre-5.x that was substituted for a similarly-named package with a silent but very impactful breaking change (multiplying output size and considerably degrading render perf cf #33).
Though I find the quality of its code dubious, I take it there was merit in depending on the smartypants package?

If anything the single method used should be smartypantsu, because why add kilobytes to your page when HTML entities are no longer required to render most UTF-8 chars since at least 2009? https://stackoverflow.com/questions/520236/should-i-still-use-html-entities-why. But with this smartypants dependency, smartypantsu function actually adds even more overhead instead of less 🤦‍♂️

[UziTech]

but having the option is stil better than not having it

Users still have the option to do it the same way you did in postprocess.

I would argue having a simple options for everything is not always better than not having it

I take it there was merit in depending on the smartypants package?

Just that it is the only npm package that matches the full smartypants spec. If you find (or create) a different package I have no problems with switching to that.

---

Honestly I think it would be easier and use less bites to not use smartypants

https://github.com/othree/smartypants.js#why-you-might-not-want-to-use-smart-quotes-in-your-weblog

[webketje]

I would argue having a simple options for everything is not always better than not having it
Honestly I think it would be easier and use less bites to not use smartypants

Totally agree and don't even use it personally. I'm coming at this from a point of view of minimizing breaking changes and user turnover as a major package maintainer. I commend the move to extensions but conclude that I am forced to include marked-smartypants-lite by default instead of marked-smartypants while upgrading @metalsmith/markdown to the latest marked, and adding a warning in the README discouraging use of this package until the issues with its dependency are adressed.

[UziTech]

If you don't want breaking changes than marked-smartypants-lite is the right way to go. That extension models the way marked did smartypants pre v5. It does a not follow the spec at all which is why we have two packages. Users can choose spec compliant or light weight.