Tracking issue: Nota installation and build pipeline

[willcrichton]

• nota package includes nota-editor frontend which bundles a bunch of things incl. all KaTeX fonts. Better way to reduce package size?
• need to simplify installation process, way too many pitfalls when installing node, npm, and global binaries from @nota-lang/nota
• SSR should run until all initial mobx reactions have executed so @Ref and @TableOfContents don't flicker on page load.
• Typescript should get moved into esbuild-utils (a la estrella) so it generates bindings without a separate command, and also works in watch mode
• SSR should pass noSandbox when running inside of a container

[jhvst]

Regarding installation process, I made a Dockerfile here: https://github.com/jhvst/dockerfiles/tree/main/nota

The point is that this would package nota and its dependencies into a container which one could invoke in folders where needed with a command such as podman run -v $PWD:/nota:z -w /nota nota build examples/index.nota. Here, the -v mounts the current folder to the container at /nota, and then changes the working folder within the container to /nota with the -w flag. Then, one can run the build command. Next, the output of nota appears on the host at the ./dist folder.

However, this was not as straightforward as usually for these cases:

1. there is a need for Chromium
2. since whatever goes with Chromium requires a sandbox, the user which runs the nota command cannot be root:

[core@localhost nota]$ podman run -v /var/mnt/nota:/nota:z nota build /nota/index.nota
✘ [ERROR] [plugin ssr] Failed to launch the browser process!
[0505/223244.655939:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.


TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md


/usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:1603
  let error = new Error(`${text}${summary}`);
              ^
Error: Build failed with 1 error:
error: Failed to launch the browser process!
[0505/223244.655939:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.


TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md

    at failureErrorWithLog (/usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:1603:15)
    at /usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:1131:18
    at /usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:1126:9
    at /usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:666:9
    at handleIncomingPacket (/usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:763:9)
    at Socket.readFromStdout (/usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:632:7)
    at Socket.emit (node:events:527:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Socket.Readable.push (node:internal/streams/readable:234:10) {
  errors: [
    {
      pluginName: 'ssr',
      text: 'Failed to launch the browser process!\n' +
        '[0505/223244.655939:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.\n' +
        '\n' +
        '\n' +
        'TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md\n',
      location: null,
      notes: [],
      detail: Error: Failed to launch the browser process!
      [0505/223244.655939:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
      
      
      TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
      
          at onClose (file:///usr/local/lib/node_modules/@nota-lang/nota/node_modules/puppeteer/src/node/BrowserRunner.ts:268:9)
          at Interface.<anonymous> (file:///usr/local/lib/node_modules/@nota-lang/nota/node_modules/puppeteer/src/node/BrowserRunner.ts:254:50)
          at Interface.emit (node:events:539:35)
          at Interface.close (node:internal/readline/interface:529:10)
          at Socket.onend (node:internal/readline/interface:258:10)
          at Socket.emit (node:events:539:35)
          at endReadableNT (node:internal/streams/readable:1345:12)
          at processTicksAndRejections (node:internal/process/task_queues:83:21)
    }
  ],
  warnings: []
}

This is about the user within the container which runs the command. I got around this by creating a user within the container, but this introduces other problems (most I think are podman centric, which is runs rootless containers):

[core@localhost nota]$ podman run -v $PWD:/nota:z -w /nota nota build examples/index.nota
✘ [ERROR] Failed to create output directory: mkdir /nota/dist: permission denied

[11:08:39 PM] info: Access to script at 'http://localhost:8000//index.mjs' from origin 'null' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
[11:08:39 PM] info: Failed to load resource: net::ERR_FAILED
✘ [ERROR] [plugin ssr] waiting for function failed: timeout 10000ms exceeded

  This error came from the "onEnd" callback registered here:

    /usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:885:10:
      885 │           await promise;
          ╵           ^

    at setup (file:///usr/local/lib/node_modules/@nota-lang/nota/node_modules/@nota-lang/esbuild-utils/lib/ssr.ts:148:11)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async handlePlugins (/usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:885:11)

/usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:1603
  let error = new Error(`${text}${summary}`);
              ^
Error: Build failed with 2 errors:
error: Failed to create output directory: mkdir /nota/dist: permission denied
error: waiting for function failed: timeout 10000ms exceeded
    at failureErrorWithLog (/usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:1603:15)
    at /usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:1249:28
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  errors: [
    {
      detail: undefined,
      location: null,
      notes: [],
      pluginName: '',
      text: 'Failed to create output directory: mkdir /nota/dist: permission denied'
    },
    {
      pluginName: 'ssr',
      text: 'waiting for function failed: timeout 10000ms exceeded',
      location: null,
      notes: [Array],
      detail: TimeoutError: waiting for function failed: timeout 10000ms exceeded
          at new WaitTask (file:///usr/local/lib/node_modules/@nota-lang/nota/node_modules/puppeteer/src/common/DOMWorld.ts:813:28)
          at DOMWorld.waitForFunction (file:///usr/local/lib/node_modules/@nota-lang/nota/node_modules/puppeteer/src/common/DOMWorld.ts:728:22)
          at Frame.waitForFunction (file:///usr/local/lib/node_modules/@nota-lang/nota/node_modules/puppeteer/src/common/FrameManager.ts:1375:28)
          at Page.waitForFunction (file:///usr/local/lib/node_modules/@nota-lang/nota/node_modules/puppeteer/src/common/Page.ts:3358:29)
          at file:///usr/local/lib/node_modules/@nota-lang/nota/node_modules/@nota-lang/esbuild-utils/lib/ssr.ts:173:20
          at processTicksAndRejections (node:internal/process/task_queues:96:5)
          at async Promise.all (index 0)
          at async file:///usr/local/lib/node_modules/@nota-lang/nota/node_modules/@nota-lang/esbuild-utils/lib/ssr.ts:181:7
          at async /usr/local/lib/node_modules/@nota-lang/nota/node_modules/esbuild/lib/main.js:1040:15
    }
  ],
  warnings: []
}

On podman systems this requires running podman unshare chown $uid:$gid -R $folder to make the system work. Eventually one should have:

[core@localhost nota]$ ls ./dist/
[core@localhost nota]$ podman run -v $PWD:/nota:z -w /nota nota build examples/index.nota
[11:09:29 PM] info: Built in 1.68s.
[core@localhost nota]$ ls dist/
index.html  index.mjs

My workflow requires containers for a self-inflected reason, but I'm willing to help if this seems something that you see would simplify the process.

As a takeaway, I think that if the Chromium dependency could removed, then I don't see a reason why this wouldn't work well as a VSCode extension. Otherwise, I think the VSCode extension might run into similar problems as podman.

[willcrichton]

@jhvst thanks for the heads up about containerizations. The Chromium dependency is important, because we use Chrome (via Puppeteer) for server-side rendering. But I could change the SSR plugin to pass --no-sandbox to Puppeteer when inside of a container.

[jhvst]

Thanks, that could help in part. Though I feel like containers might not be the way the go here. I pushed some code of mine to https://github.com/jhvst/jhvst.github.io but as you can see, it became rather hacky with bash scripts involved. I think containers in general have a lot of painpoints when the host filesystem has to be modified. Though I hope this report could help anyone who wants to look into that.

That being said, do you happen to have plans to integrate Nota with VSCode via LSP? If Puppeteer is used for server-side rendering, I wonder could one use the WebView feature in VSCode for a more streamlined setup?

[willcrichton]

Puppeteer shouldn't be part of the editing loop -- it's only for generating final files that will be shipped to clients. So that shouldn't be an issue for editor integration.

For now, I am building my own Overleaf-like editor (@nota-lang/nota-editor) based on Codemirror 6. If Nota gets popular enough, I will look to add editor integrations.