Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NNotepad sample #243

Merged
merged 4 commits into from
Jun 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions nnotepad/.eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
module.exports = {
env: {'es6': true, 'browser': true, 'jquery': false, 'node': true},
parserOptions: {ecmaVersion: 2021, sourceType: 'module'},
};
9 changes: 9 additions & 0 deletions nnotepad/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.PHONY: clean

all: res/docs.html

res/docs.html: README.md
bin/makedocs

clean:
rm -f res/docs.html
106 changes: 106 additions & 0 deletions nnotepad/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# What is this?

**NNotepad** is a browser-based playground for experimenting with [WebNN](https://webmachinelearning.github.io/webnn/) expressions without boilerplate code. As of mid-2024, WebNN is available as a prototype in Chromium-based browsers, but requires launching the browser with particular flags enabled.


# Usage

Type assignments like `foo = 1 + 2` or expressions like `2 * foo`. The result of the last assignment or expression is shown. Some examples:

```
1 + 2
# yields 3

a = 123
b = 456
a / b
# yields 0.2697368562221527

A = [[1,7],[2,4]]
B = [[3,3],[5,2]]
matmul(A,B)
# yields [[38,17],[26,14]]
```

**NNotepad** translates what you type into script that builds a WebNN graph, evaluates the script, then executes the graph. Click 🔎 to see the generated script.

Expressions can use:

* Operators `+`, `-`, `*`, `/`, `^`, `==`, `<`, `<=`, `>`, `>=`, `!` with precedence, and `(`,`)` for grouping.
* Function calls like `add()`, `matmul()`, `sigmoid()`, and so on.
* Numbers like `-12.34`.
* Tensors like `[[1,2],[3,4]]`.
* Dictionaries like `{alpha: 2, beta: 3}`, arrays like `[ A, B ]`, strings like `"float32"`, and booleans `true` and `false`.

Functions and operators are turned into [`MLGraphBuilder`](https://webmachinelearning.github.io/webnn/#mlgraphbuilder) method calls.

Array literals (`[...]`) and number literals (`12.34`) are interpreted contextually:

* In assignments, they are intepreted as tensor/scalar constant [`MLOperand`](https://webmachinelearning.github.io/webnn/#mloperand)s, e.g. `alpha = 12.34` or `T = [1,2,3,4]`.
* In most function calls, they are interpreted as tensor/scalar constant [`MLOperand`](https://webmachinelearning.github.io/webnn/#mloperand)s, e.g. `neg(123)` or `neg([1,2,3])`.
* In some function calls, they are interpreted as arrays/numbers for some positional parameters, e.g. `concat([A,B,C],0)`. This includes: [`concat()`](https://webmachinelearning.github.io/webnn/#dom-mlgraphbuilder-concat), [`expand()`](https://webmachinelearning.github.io/webnn/#dom-mlgraphbuilder-expand), [`pad()`](https://webmachinelearning.github.io/webnn/#dom-mlgraphbuilder-pad), [`reshape()`](https://webmachinelearning.github.io/webnn/#dom-mlgraphbuilder-reshape), [`slice()`](https://webmachinelearning.github.io/webnn/#dom-mlgraphbuilder-slice), [`split()`](https://webmachinelearning.github.io/webnn/#dom-mlgraphbuilder-split).
* In dictionaries, they are interpreted as arrays/numbers, e.g. `linear(123, {alpha: 456, beta: 789})` or `transpose(T, {permutation: [0,2,1]})`. To pass a tensor/scalar constant in a dictionary, use a variable or wrap it in [`identity()`](https://webmachinelearning.github.io/webnn/#dom-mlgraphbuilder-identity) e.g. `gemm(A, B, {c:identity([4])})` or `gemm(A, B, {c:identity(4)})`.

The default [data type](https://webmachinelearning.github.io/webnn/#enumdef-mloperanddatatype) for scalars and tensors is [`float32`](https://webmachinelearning.github.io/webnn/#dom-mloperanddatatype-float32). To specify a different data type, suffix with one of `i8`, `u8`, `i32`, `u32`, `i64`, `u64`, `f16`, `f32`, e.g. `123i8` or `[1,2,3]u32`.


# Helpers

In addition to WebNN [`MLGraphBuilder`](https://webmachinelearning.github.io/webnn/#mlgraphbuilder) methods, you can use these helpers:

* **load(_url_, _shape_, _dataType_)** - fetch a tensor resource. Must be served with appropriate [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) headers. Example: `load('https://www.random.org/cgi-bin/randbyte?nbytes=256', [16, 16], 'uint8')`


# Details & Gotchas

* [`float16`](https://webmachinelearning.github.io/webnn/#dom-mloperanddatatype-float16) support (and the `f16` suffix) is experimental.
* Whitespace including line breaks is ignored.
* Parsing around the "unary minus" operator can be surprising. Wrap expressions e.g. `(-a)` if you get unexpected errors.
* If output is a constant, it will be wrapped with [`identity()`](https://webmachinelearning.github.io/webnn/#dom-mlgraphbuilder-identity) if your back-end supports it. Otherwise, you must introduce a supported expression.

What ops are supported, and with what data types, depends entirely on your browser's WebNN implementation. Here be dragons!


# Parsing & Grammar

```
Anything after # or // on a line is ignored (outside other tokens)

{} means 0-or-more repetitions
[] means 0-or-1 repetitions
() for grouping
| separates options
'' is literal
// is regex

program = line { line }
line = assigment | expr
assigment = identifier '=' expr

expr = relexpr
relexpr = addexpr { ( '==' | '<' | '<=' | '>' | '>=' ) addexpr }
addexpr = mulexpr { ( '+' | '-' ) mulexpr }
mulexpr = powexpr { ( '*' | '/' ) powexpr }
powexpr = unyexpr { '^' unyexpr }
unyexpr = ( '-' | '!' ) unyexpr
| finexpr
finexpr = number [ suffix ]
| array [ suffix ]
| string
| boolean
| dict
| identifier [ '(' [ expr { ',' expr } ] ')' ]
| '(' expr ')'

string = /("([^\\\x0A\x0D"]|\\.)*"|'([^\\\x0A\x0D']|\\.)*')/
number = /NaN|Infinity|-Infinity|-?\d+(\.\d+)?([eE]-?\d+)?/
boolean = 'true' | 'false'
identifier = /[A-Za-z]\w*/
suffix = 'u8' | 'u32' | 'i8' | 'i32' | 'u64' | 'i64' | 'f16' | 'f32'

array = '[' [ expr { ',' expr } ] ']'

dict = '{' [ propdef { ',' propdef } [ ',' ] ] '}'
propdef = ( identifier | string ) ':' expr
```

17 changes: 17 additions & 0 deletions nnotepad/TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# To-Do

## Basics

* Style to match rest of webnn-samples.
* Improve default text.
* Consider incorporating [WebNN Polyfill](https://github.com/webmachinelearning/webnn-polyfill).
* Make input/output areas resizable.
* Add to `../README.md` once we're happy with it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Style to match rest of webnn-samples.
  • Consider incorporating WebNN Polyfill.

I can take these two TODOs.

## WebNN Support

* Allow size-0 dimensions in tensors per [#391](https://github.com/webmachinelearning/webnn/issues/391).

## Advanced

* Show line/col in parse error messages, and line numbers in textarea.
53 changes: 53 additions & 0 deletions nnotepad/bin/makedocs
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/usr/bin/env python3

import markdown

with open('README.md', 'r', encoding='utf-8') as input_file:
text = input_file.read()

html = markdown.markdown(text, extensions=['extra'])

with open('res/docs.html', 'w', encoding='utf-8', errors='xmlcharrefreplace') as output_file:
output_file.write('''<!doctype html>
<meta charset=utf8>
<title>NNotepad</title>
<!--

THIS IS A GENERATED FILE.

DO NOT EDIT.

Edit README.md instead, then run: ./makedocs

-->
<style>
body {
font-family: sans-serif;
font-size: 16px;
line-height: 30px;
}
code {
font-family: "Consolas", "Lucida Console", monospace;
}
code {
display: inline-block;
background-color: #eee;
border-radius: 0.25lh;
padding: 0 0.25lh;
}
pre code {
display: inline;
background-color: inherit;
border-radius: initial;
padding: initial;
}
pre {
background-color: #eee;
border-radius: 1lh;
padding: 1lh;
}
</style>

''');
output_file.write(html)

128 changes: 128 additions & 0 deletions nnotepad/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
<!doctype html>
<meta charset=utf-8>
<title>NNotepad</title>
<link rel=icon href="res/webml.png">
<link rel=manifest href="res/manifest.json">
<style>
html { height: 100%; }
body {
margin: 0; padding: 0;
width: 100%; height: 100%;
overflow: hidden;
}
pre {
white-space: pre-wrap;
}
#input {
box-sizing: border-box;
position: absolute;
margin: 0;
left: 0; right: 0; top: 0; bottom: 200px;
padding: 10px;
border: none;
outline: none;
resize: none;
}
#output {
box-sizing: border-box;
position: absolute;
margin: 0;
left: 0; right: 0; height: 200px; bottom: 0;
padding: 10px;
border: none;
background-color: #eee;
overflow: auto;
}
#watermark {
position: absolute;
right: 15px; top: 5px;
color: #61BAFB;
color: #4777C0;
font-family: sans-serif;
font-size: 32px;
font-style: italic;
font-weight: bold;
user-select: none;
}
#watermark img {
height: 40px;
vertical-align: bottom;
}
#toolbar {
position: absolute;
right: 15px;
top: 50px;
}
#toolbar button {
background-color: transparent;
border: none;
font-size: 40px;
}

#srcDialog {
max-width: calc(100vw - 80px);
max-height: calc(100vh - 80px);
}
#srcText {
position: relative;
box-sizing: border-box;
border: 20px solid #eee;
max-height: calc(100vh - 200px);
overflow: auto;
background-image: linear-gradient(#eee 50%, #e4e4e4 50%);
background-size: 100% 2lh;
}
dialog {
font-family: sans-serif;
}
code {
font-family: "Consolas", "Lucida Console", monospace;
background-color: #eee;
border-radius: 0.25lh;
padding: 0.25lh;
}
#helpDialog {
max-width: calc(100vw - 80px);
max-height: calc(100vh - 80px);
}
#helpText {
position: relative;
box-sizing: border-box;
border: none;
width: calc(100vw - 200px);
height: calc(100vh - 200px);
}
</style>

<script src="js/float16arraypolyfill.js"></script>
<script src="js/util.js" type="module"></script>
<script src="js/nnotepad.js" type="module"></script>
<script src="js/index.js" type="module"></script>

<textarea id=input autofocus cols=80 rows=24 placeholder="Enter code here..." spellcheck="false">
</textarea>

<pre id=output>
Results will show here
</pre>

<div id=watermark>
<img src="res/webml.png" alt="">NNotepad
</div>

<div id=toolbar>
<button id=peek title="Show generated code">&#x1F50E;</button><br>
<button id=help title="Show documentation">&#x1F6C8;</button><br>
<select id=device title="MLContext deviceType hint"><option value=cpu>CPU</option><option value=gpu>GPU</option><option value=npu>NPU</option></select>
</div>

<dialog id=srcDialog>
An <code>MLGraphBuilder</code> is passed as <code>_</code>
<pre id=srcText></pre>
<button id=srcClose autofocus>Close</button>
</dialog>

<dialog id=helpDialog>
<iframe id=helpText src="res/docs.html"></iframe><br>
<button id=helpClose autofocus>Close</button>
</dialog>
Loading