Meta: changes to support a revamped developer's edition

[domenic]

This introduces a developers stylesheet derived from
https://github.com/benschwarz/developers.whatwg.org/tree/97ff943a8f5b8fe38f78e224b4b44b472ccefb57/sass,
including the web font. It also updates the source document slightly to
make the generated output work better for the dev edition.

---

Putting this up for review. Notable missing functionality from the old dev edition is noted in "TODO: things to possibly restore?". (It's in the stylesheet because I deleted the related CSS.) Also see below.

There are several TODOs in the stylesheet where we hide things via CSS where ideally Wattsi would omit them. Also it would help if Wattsi gave the index page a special class we could use. In general, peruse the TODOs.

Currently this is modeled as being hosted at https://html.spec.whatwg.org/dev, because that was easy, but we can also host it at https://developers.whatwg.org/ with a bit more work.

Relies on a small forthcoming html-build PR.

Live preview at https://ns-mumwzonpfy.now.sh/dev/ . Compare to https://developers.whatwg.org/ .

---

In general I'm wondering if we should try to stand something like this up sooner, perhaps as a "beta" we invite people to test, instead of getting it fully back to feature parity with the old one.

Here's a list of potential desired features. Those that require Wattsi support are marked with [W].

• [W] Service worker for offline support
  • Probably we should finish Add the ability for any spec to have a service worker resources.whatwg.org#53 first
  • Then we need Wattsi to generate a list of files; not difficult.
• Web app manifest?
  • See Add a manifest to specs so they can be pinned to the homescreen resources.whatwg.org#7; we would have to hide the URL bar. Not sure if worth it.
• Appcache for old browsers? (like the old version)
• Favicon that works in non-Firefox browsers?
• Favicon that is different from the WHATWG logo? (like the old verison)
• Search (like the old version)
• [W] Wattsi omitting stuff instead of CSS hiding it: Omit unnecessary elements in the dev edition wattsi#52
• [W] Wattsi giving the index page a special class: Add an "index" class to the multipage index.html wattsi#51
• [W] Footer, like the old version?
• "About this specification" on the index page, like the old version?
• [W] Expandable within-page TOCs ("in this section"), like the old version?
• [W] More eye-catching navigate-to-next-page footer, like the old version?
• An audit of the content to add w-nodev in more places (e.g. the WindowProxy exotic object stuff obviously shouldn't be there; the indices that refer to element interfaces shouldn't be there...)
• Syntax highlighting of HTML/CSS/JavaScript: Add syntax-highlighting of HTML/CSS/JS examples #2751

More suggestions for desired features welcome. Thoughts on which of those we should fix before releasing welcome.

Bugs noticed:

• self-link styling works badly with h2s fixed
• fingerprint styling needs to be ported fixed; fingerprinting should not be in dev edition at all
• /dev does not work, only /dev/. (Probably not an issue our actual server, just on now.sh?) But clicking on the logo takes you to /dev, not /dev/.
• <title> is just "HTML Standard" still Make <title> element content use split heading content wattsi#50
• index TOC styling doesn't work for large numbers like "15.11"; needs a bit more space fixed

[sideshowbarker]

In general I'm wondering if we should try to stand something like this up sooner, perhaps as a "beta" we invite people to test

I think we should. That will get more feedback and and more ideas more early

[sideshowbarker]

Alternately maybe we can make a list of desired features somewhere. Let me start...

• Search (like the old version)

I raised #2565 for that—though it’s not intended to necessarily be at all like the old version.

I don’t recall what that old search feature did/does, but as far as I know it only worked for searching within the current page?

If so I think that’s of pretty limited utility to users. Whatever search feature we might end up adding, a think a requirement needs to be that it is multipage-enabled, in that is allows searching for a string across all documents in the multipage output, and that it also allows searching for exact matches (by quoting the search term) as well as based on a list of terms (non-exact matching).

More suggestions for desired features welcome.

• Syntax highlighting of HTML/CSS/JavaScript

I raised #2751 for that. It does the syntax highlighting dynamically in the client side. Certainly it would seem better to instead implement it in the build—in the wattsi code—but I don’t forsee any of us implementing it there any time soon, nor can I imagine any other contributors stepping forward any time soon to do it.

Further comments about syntax highlighting

That #2751 is essentially just relying completely on https://highlightjs.org/. But that makes it uncomplicated as far as maintenance, while getting users what they want.

One thing that https://highlightjs.org/ doesn’t handle yet is WebIDL. But I think we could get it added with a fairly low level of effort, and give something back by contributing it upstream.

Alternatively (and preferably) we would implement https://www.w3.org/Bugs/Public/show_bug.cgi?id=26343 (“Wattsi: IDL parser in pipeline”) but again I don’t see any of us getting around to doing that any time soon, nor getting any contributors interested/able to work on it.

[sideshowbarker]

Bugs noticed:

• <title> is just "HTML Standard" still

That reminds me of something not closely related but maybe worth mentioning here; which is: we have a deficiency in the wattsi multipage-generating code in that it outputs the same <title>HTML Standard</title> for every multipage file. Google’s search indexer is smart enough to ignore that and build/show a useful/helpful title in its search results anyway (from the h2 content of the document) but for other search engines and use cases, we really should make the <title> content of the source actually be specific to the document.

[zcorpan]

The old version also has inlined references. See e.g.
https://developers.whatwg.org/infrastructure.html#infrastructure
vs
https://ns-uzpjigzezg.now.sh/dev/infrastructure.html#infrastructure

[zcorpan]

IMO...

Appcache for old browsers? (like the old version)

No.

Favicon that works in non-Firefox browsers?

No, we should apply pressure for non-Firefox browsers to support SVG favicons. ^_^

Favicon that is different from the WHATWG logo? (like the old verison)

I think the logo should fit with the WHATWG brand. WHATWG logo in a different color is OK.

Wattsi omitting stuff instead of CSS hiding it

Yes.

[domenic]

I don’t recall what that old search feature did/does, but as far as I know it only worked for searching within the current page?

No, it was more along the lines of what I've envisioned in #2565 (comment) ("my alternate approach"). Try it out at https://developers.whatwg.org/. That's still the direction I think would be best for #2565, instead of full-text search with an overlay...

The old version also has inlined references

Yeah, I somehow forgot that. I think it's quite nice, although it needs Wattsi support.

Let me reorganize the OP to mark things that need Wattsi support. Then maybe people could help with them?

[domenic]

New version with some fixes at https://ns-eukkeoznoe.now.sh/dev/ in theory although I have to run out the door and haven't tested it yet.

[sideshowbarker]

No, it was more along the lines of what I've envisioned in #2565 (comment) ("my alternate approach"). Try it out at https://developers.whatwg.org/

So what exactly is that searching? Just the section titles/headings?

That's still the direction I think would be best for #2565, instead of full-text search with an overlay...

Why? It’s not clear to me why readers would not want full-text search but would want it limited to something less than full text, and not clear to me why we would to limit them to something less.

I agree that some way to quickly navigate to specific sections based on their names is a useful feature, but to me that is something quite different than a search feature. It’s more a case of providing/exposing some kind of persistent outline/table-of-contents that users can access quickly.

[domenic]

Yes, section headers. We could additionally expand it to include definitions.

In my opinion for full text search readers can use a search engine. We don't need to create one for them. I'd rather provide something based on the more structured data we have in hand by virtue of deep knowledge of the document structure.

For another example, see the ECMAScript spec's built-in search feature.

[sideshowbarker]

In my opinion for full text search readers can use a search engine.

Of course they can. Or they can just use the single-page spec. They can do a lot of things without us adding features to try to make it easier for them.

Clearly none of this additional stuff for multipage is essential. We could choose to avoid the cost and time and maintenance of adding any special user-convenience features for the multipage spec at all. For example, the dfn-popup feature worked fine in the single-page spec and we could have just said users wanting to use it against the full spec should just use the single-page spec.

Similarly we can choose to just conclude we don’t want to provide full-text search in multipage.

But it seems pretty clear to me that a lot of users would appreciate having the choice of convenience at point of use of full-text search across the entire multipage spec, without us requiring them to load the full single-page spec to get that, or to go somewhere else to get that.

We don't need to create one for them. I'd rather provide something based on the more structured data we have in hand by virtue of deep knowledge of the document structure.

It’s not an either-or. I strongly agree that providing something based on data we have in hand by virtue of deep knowledge of the document structure would be great. But it doesn’t need to come at the exclusion of a convenient way to do full-text search in multipage at point of use.

[sideshowbarker]

In my opinion for full text search readers can use a search engine.

As far as third-party search engines go, another significant known problem for users we have there is the fact that results from html.spec.whatwg.org typically don’t show up high in the search results—sometimes not even at all in the first page of results.

Further thoughts on making users rely on third-party search, and user scenarios on mobile

What I mean is, if a user does a Google search for some term we know is defined in the HTML spec, in many or most cases what they’re likely to get back first in the results are hits to a copy of the spec somewhere at w3.org (and probably not even the most up-to-date w3.org copy), or hits from w3schools.com. The html.spec.whatwg.org hits will be buried further down, if they’re there at all.

Of course if these are users who’ve already been reading the html.spec.whatwg.org spec and are just going over to the search engine to find something specific in it, then they may know they just do site:html.spec.whatwg.org to limit the search to the actual HTML spec.

But all that seems to me like us making users jump through a lot of hoops just be able to do full-text search of the spec—making them open a separate tab and do foo bar site:html.spec.whatwg.org. when they already have the (multipage) spec open in a tab right in front of them.

And while the might not seem like such a big extra step for users of desktop browsers, consider the case of users reading the spec on mobile and how much more inconvenient the scenario above is.

Also I realize that on desktop, users can do something html.spec.whatwg.org-specific using an opensearch.xml file or whatever—or set up a special search browser extension maybe. But again I don’t know if/how they can do that on mobile. But at least as far as doing it with a browser extension, we know they couldn’t use an extension-based solution with Chrome on mobile, since it has no support for extensions.

Anyway from the above I hope it’s clear why I think there’s significant value to users in having full-text search using an overlay. Doing it with an overlay lets them initiate the search without leaving the spec they have open—and lets them if needed just drop right back to what they were reading, if the search turns out to not return any hits they want to browse to.

We don't need to create one for them.

Well my initial implementation at #2565 was an attempt to avoid creating a new one for them, and to instead embed a Google search at point of use with the significant convenience of an overlay. But the conclusion there seemed to be that we can’t use a third-party search service.

So I’ve instead provided the same functionality using a self-hosted search service—and in the process have come to realize that’s better for us than relying on third-party service anyway, because it gives us ourselves control over the formatting/look-and-feel and content of the search results (whereas we have no control at all over that with third-party service).

[domenic]

Sorry for replying here. Let's move this over to #2565.

[domenic]

Sent some Wattsi PRs that make the stylesheet cleaner; see latest commit and Wattsi PR list. Produced https://output-wtmypyildq.now.sh/dev/ from that.

[domenic]

What else should we work on before deploying this? To me the biggest outstanding question is whether we should make the URL https://html.spec.whatwg.org/dev/ or https://developers.whatwg.org/; if we decided on that I would be comfortable deploying right now, despite the loss of functionality. But if I got a signal about other things that were important, most of them are not that hard.

I guess also doing a pass through the output looking for out of place things, although the community can help with that.

[zcorpan]

"by" should be inside w-nodev

[zcorpan]

Do we want to inline this and host the fonts ourselves? I realize this uses different formats depending on UA, but woff2 is well supported (and woff has wider support still).

[xfq]

I don't have very strong opinions on this, but I personally lean towards hosting on whatwg.org, because I have trouble accessing *.googleapis.com. Google APIs are blocked in mainland China.

[domenic]

Great point; let's pull these out.

[zcorpan]

Generally looking good, I think we can ship and then iterate.

[domenic]

@zcorpan @sideshowbarker thoughts on the developers.whatwg.org vs. /dev issue?

Also need review on whatwg/html-build#112 (very quick)

[zcorpan]

I kinda like /dev/ better, since that makes it applicable to other specs as well, if we want to do that.

[sideshowbarker]

@zcorpan @sideshowbarker thoughts on the developers.whatwg.org vs. /dev issue?

I kinda like /dev/ better, since that makes it applicable to other specs as well, if we want to do that.

Agreed. I’d think anybody looking at the URL developers.whatwg.org who didn’t already know would assume to find something more general there, not a version of the HTML spec