Skewering HTTPS (mixed-mode loading problems)

[ecraven]

Is there a way to tell skewer to use https when skewering pages that are already https? Would make skewer-everything.js much more useful.

Thanks for creating this great emacs mode!

[skeeto]

This is one of the weak spots of Skewer, especially as the web moves
more and more towards HTTPS. Emacs has GnuTLS bindings but only for TLS
clients (i.e. url-retrieve). So at the moment there's no way to host an
HTTPS server within Emacs for Skewer short of writing an entire TLS
implementation in Emacs Lisp, which would be a far bigger project than
Skewer itself.

[ecraven]

Would it be possible to solve this by using something like socat to forward an ssl socket to emacs?

[skeeto]

That's very likely to be possible. When Emacs is compiled without GnuTLS
support, it uses the gnutls command line program instead in a similar
way.

Since I don't need this myself at the moment, it's not something I plan
to implement, though this doesn't preclude anyone else from trying.

[unhammer]

I've used hitch as a workaround.

Run this once:

mkdir -p ~/.emacs.d/skewer-https/
cd ~/.emacs.d/skewer-https/

openssl req -nodes -newkey rsa:2048 -keyout skewer-hitch.key -out skewer-hitch.csr \
    -subj "/C=GB/ST=London/L=London/O=Acme Widgets/OU=IT Department/CN=localhost"
openssl x509 -req -days 365 -in skewer-hitch.csr -signkey skewer-hitch.key -out skewer-hitch.crt
cat skewer-hitch.key skewer-hitch.crt >skewer-hitch.pem

sudo apt install hitch

and run this after run-skewer:

(unless (get-buffer-process "*skewer-hitch*")
      (start-process "skewer-hitch" "*skewer-hitch*"
                     "hitch"
                     (format "--backend=[localhost]:%s" httpd-port)
                     "--frontend=[localhost]:38443"
                     (expand-file-name "~/.emacs.d/skewer-https/skewer-hitch.pem")))

---

In your web browser, go to https://localhost:38443/skewer and accept the self-signed cert, then change your bookmarklet to have the right port:

javascript:(function(){var%20d=document;var%20s=d.createElement('script');s.src='https://localhost:38443/skewer';d.body.appendChild(s);})()

(Make sure to reload the page first if you used your old bookmarklet – it'll point to the wrong port.)

---

I suppose we could add this as a function run-skewer-https that checks if the .pem exists (creating it if not) and runs hitch (giving some helpful message if it's not found)?

[skeeto]

Since @unhammer has reminded me of this thread, I'll also mention that a couple months ago I experimented with using stunnel as a seemless front-end to Skewer. I captured the gist of it here: https://github.com/skeeto/dotfiles/blob/46c31aac041ee6f7b66083ed2512a21f02ce1987/_bashrc#L97 However its a bit flaky and I never quite sorted it out: * Sometimes stunnel prematurely terminates the connection and I don't know why. This happens with the Python server, too, so it's not just some issue with Skewer. * It relies specifically on stunnel3. Later (?) versions of stunnel (e.g. stunnel4) have a different interface and don't seem to support this mode. * Modern browsers *really* hate self-signed certs and no longer handle them gracefully (i.e. no more "store and allow this cert next time"). This is a messy issue to solve.

[unhammer]

Hm, I haven't noticed hitch doing any premature termination – maybe give that a try? It's quite small:

Installed-Size: 201 kB
Depends: libc6 (>= 2.14), libev4 (>= 1:4.04), libssl1.0.0 (>= 1.0.0), init-system-helpers (>= 1.18~), adduser

Why create a new cert on every startup? (Isn't that just creating more work for yourself?)
Firefox at least lets me save the cert when I go to localhost:38443/skewer and click Advanced, but since it's the same one each time, I only had to do it once (on that computer). Chrome I have no idea though.

[skeeto]

Hitch certainly seems like a cleaner solution than stunnel. I'll have to spend some time with it.

Why create a new cert on every startup?

That particular example is just for a throwaway sever. Plus, for documentation purposes, it captures the cert generation procedure (I can never remember). For Skewer it would probably make sense for a more persistent cert, hence it's important that the browser remembers the cert.

Firefox at least lets me save the cert

Oops, you're right. Upon closer examination I see that it's just my current configuration preventing the "permanently store" checkbox.

[unhammer]

Btw, for Chromium, I opened "Manage certificates → Authorities → Import" and selected my .pem file, ticking off "Trust this certificate for identifying websites". A bit more involved, but i can control Chromium too from Emacs now :)

Though for some reason, it sometimes first says GET https://localhost:38443/skewer/get net::ERR_INSECURE_RESPONSE in the console, but if I middle-click that (so it opens in a new tab) and click the bookmarklet again, it works. Very security 👍