Interactive playground for einops

[arogozhnikov]

I want to use interactive notebooks as an intro to einops, currently there are two versions: on github, and converted version in docs.

Only two notebooks work in jupyterlite (basic operations, and pack/unpack), but that's already enough for an intro.

Trying to figure out a simple setup to

• keep notebooks in github
• do not conflict with gh-pages branch
• auto-update to latest pushed version on github
• provide utils / npy together with tutorial

[zachlipp]

Hey @arogozhnikov! I would love to help contribute to this if you're open to it

[arogozhnikov]

@zachlipp yes, some help would be great.

FYI I expect this to be challenging: I didn't see jupyterlite and mkdocs being co-deployed.

Prefect solution is to have jupyterlite and notebooks being stored in different repos: after loading in browser jupyterlite would download fresh notebooks from einops repository. This would be very reliable and simple to maintain.

[arogozhnikov]

I think other projects would also be interested in just relying on some static server (e.g. hosted at github pages) if they could just provide a folder with notebooks.

[zachlipp]

This sounds interesting. If I understand right, what you're looking for is:

• A separate repository consisting of a jupyterlite app
• On load, this repository fetches the current notebooks from this repository
• Deploying this app on github pages so it can run in a browser

Does this sound about right? If so, I can get started on a proof of concept in a new repo under my username and we can figure out where it lives in the long term later

[arogozhnikov]

@zachlipp yes, this would be great setup if it works

[zachlipp]

@arogozhnikov I'm making some progress here! One thing to think about: When should we overwrite the files the user has downloaded? I don't think overwriting the notebooks each time the user refreshes the page is a good user experience.

Maybe we get the git hash of the latest commit when we download the files, store the hash, then prompt the user to update if we see there's a new hash on the main branch?

[arogozhnikov]

Wow, great to hear that.

I normally don't expect js playgrounds to store my results, I think this would work.

Maybe we get the git hash of the latest commit when we download the files, store the hash, then prompt the user to update if we see there's a new hash on the main branch?

Yes, that sounds a a good tradeoff.
NB: if user reloads page, and you keep user's previous result, it worth saying this to avoid cases when users complain about broken example notebooks (broken because user previously played with it, but didn't realize that's their changes).

[zachlipp]

I've spent a few more hours on this, and you're right, it's pretty complicated. I am writing a lot of javascript and I am far from a javascript expert. One thing to flag: Most of the existing tutorials cannot work because pytorch isn't yet supported in pyodide. This is an area of active discussion here pyodide/pyodide#1625

Once I have a hosted environment that can execute some of the examples (I'm close!), I'll let you know and we can decide on appropriate steps to take from there

[arogozhnikov]

pytorch isn't yet supported in pyodide.

yes, we should only plan for two notebooks with numpy (basic into and pack/unpack). Completely fine for playground, and I don't expect DL frameworks to target WASM anytime soon

[zachlipp]

I totally agree :)

[arogozhnikov]

heyhey @zachlipp how is the progress?

i've tried this link: zachlipp.github.io/einops-documentation/lab/index.html

that's what appears in chrome console:

Error fetching file 2-einops-for-deep-learning.ipynb: DOMException: Failed to execute 'transaction' on 'IDBDatabase': One of the specified object stores was not found.

[zachlipp]

Hey @arogozhnikov! Yeah, it's not quite there. I could use some assistance from someone who knows frontend better than I do (e.g. how to work with IndexedDB and the like). I'll keep trying in the mean time

[zachlipp]

Here's what I see as the next steps:

• Finishing the JS to properly download the files and store them in IndexedDB
• Setting up some sort of limits on when the files get redownloaded. I don't have a great answer here.
• Including the subdirectories in the download
• Excluding files with torch dependencies in the download
• Getting the notebooks we download ready to work with einops out of the gate. That could mean:
  • Pre-installing einops with emscripten forge (instructions here)
  • Do something clever to run micropip.install("einops") before/on kernel load
• Replace the hacky JS + awk pipeline I have now to an actual pyodide module

I am trying to complete these in roughly this order; I definitely welcome help from folks with more frontend experience