Dex Tutorial #443

srush · 2021-01-09T01:26:44Z

An attempt at writing a Dex starter tutorial.

Goal was to start and focus on the that makes Dex unique (for / indices / type system) as opposed to starting with the syntax. Also assumes that the user is coming from python.

Open questions:

What are the standard names for things. Arrays? Tables? Tensors?
Is plotting / html stable? Probably would look better with plots and things like that as a running example.

For #425

google-cla · 2021-01-09T01:26:48Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

dan-zheng

Excellent language introduction and persuasive literate programming, thank you!

No surprise given your history of great notebook essays:

Tensors Considered Harmful (named tensors): http://nlp.seas.harvard.edu/NamedTensor
The Annotated Transformer: https://nlp.seas.harvard.edu/2018/04/03/attention.html

Would you mind if I pushed a copy-editing commit to your branch? I thought that might be more efficient than leaving a bunch of small comments.

Edited: I created a PR srush#1 to your srush:tutorial branch with some tutorial edits, in addition to merging google-research:main into the branch. The current srush:tutorial branch seems ~5 months outdated.

@srush: please take a look! I wanted to share as early as possible to prevent conflicting/duplicate edits.

examples/tutorial.dx

srush · 2021-01-09T14:11:57Z

Thanks so much Dan! I will update right away.

Definitely a first draft, so I very much appreciate all the edits. Happy for any feedback.

dan-zheng · 2021-01-09T14:30:42Z

Sure thing! It also looks like you need to sign the Contributor License Agreement (CLA).

google-cla · 2021-01-09T14:47:25Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

srush · 2021-01-09T15:31:10Z

@googlebot I signed it!

google-cla · 2021-01-09T15:31:15Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

dan-zheng · 2021-01-09T15:37:19Z

ℹ️ Googlers: Go here for more info.

The internal link says that the email address [email protected] is missing a signed CLA. I think there are two ways to resolve it:

Signing a CLA with that email address. I think that involves linking the email address to your GitHub account.
Amending the commit 3bcda85 to use a different email instead.
- Instructions here. Something like git rebase -i ... and git commit --amend --author="Sasha Rush <[email protected]>" --no-edit should do the trick.

google-cla · 2021-01-09T15:52:15Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

srush · 2021-01-09T15:58:03Z

@dan-zheng Errors fixed.

I do want to note though that this is still a draft. I want to make most of these example run on images (maybe mnist?) to make it a bit more engaging.

dan-zheng · 2021-01-09T16:01:23Z

Visualization via images sounds great!
I didn't mean to dominate, sorry. I'd like to read your tutorial again when it's ready!

examples/tutorial.dx

oxinabox · 2021-01-09T17:04:47Z

examples/tutorial.dx

+
+' Next: `\`. This symbol is the lambda sigil in Dex. It is analogous to the
+  `lambda` keyword in Python, and starts the definition of a function value
+  (i.e. closure). In `arrSum` above: the lambda takes an argument named `acc`


Technically speaking it may or may not be a closure.
Idk if that kind of pedantry is useful in the tutorial though.

It might be nice to mention in this section that this can also be used to define functions, like the def keword above.

It also might be nice to mention the symmetry between for i. and \i.;
though I thing that might be awkard so maybe not worth interupting flow that much.

Technically speaking it may or may not be a closure.
Idk if that kind of pedantry is useful in the tutorial though.

Good note. From my Swift background, I use "closure" as a generic name for "function values", including both function references (trivial closures, capturing no context) and closure expressions (\x. blah x, possibly capturing values).

It might be nice to mention in this section that this can also be used to define functions, like the def keword above.

I think omitting this (explanation of first-class functions, lambdas can be assigned to variables) is reasonable, given that Python programmers are familiar with it (Python also has first-class function values).

It also might be nice to mention the symmetry between for i. and \i.;
though I thing that might be awkard so maybe not worth interupting flow that much.

I agree, expanding a bit on "arrays are like functions" would be neat.

We discussed a bit more about "arrays as functions" in a different thread. Sasha pointed out that "for as a list comprehension" is more user-friendly than "arrays as function types/lambdas" (which is more programming-language-theoretic) - and this tutorial seems catered to folks coming from Python.

I'm fine with closure. This is not supposed to be a super formal introduction, and in most informal settings lambda ≈ closure.

oxinabox · 2021-01-09T17:13:23Z

What are the standard names for things. Arrays? Tables? Tensors?

Tables I think. That was what the old tutorial used; iirc.
When introducing them it might also be worth saying something like "Traditional arrays are the common special case of a Table where the index is Fin _, but dex tables can (as we will later see) be more exotic"
(We could probably implement some thing that a physicist would be happy to call a Tensor with up and down indexes etc. That would be cool)

Is plotting / html stable? Probably would look better with plots and things like that as a running example.

Nothing is stable, e.g. #370 like completely changed the way type classes are used and constructed.
But we should totally use plotting and html in this tuitorial and should just maintain the tutorial when/if they change.

apaszke

This is amazing, thanks for writing that!

examples/tutorial.dx

apaszke · 2021-01-09T18:17:44Z

examples/tutorial.dx

+
+' Next: `\`. This symbol is the lambda sigil in Dex. It is analogous to the
+  `lambda` keyword in Python, and starts the definition of a function value
+  (i.e. closure). In `arrSum` above: the lambda takes an argument named `acc`


I'm fine with closure. This is not supposed to be a super formal introduction, and in most informal settings lambda ≈ closure.

examples/tutorial.dx

google-cla · 2021-01-10T00:53:50Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2021-01-10T02:24:20Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

srush · 2021-01-10T02:32:54Z

Did another pass today to remove some sections and added some visual examples.

Here's the running version: https://srush.github.io/dex-lang/tutorial.html

@dougalm I love this multiple image function!

@dan-zheng feel free to take over, your changes were great. I probably don't have too much more time to work on this in the next couple days.

todo

@apaszke comments
conclusion - (point people to read prelude / examples)
something about laziness. (I need to test it out to understand when that is okay to assume)

I think I will defer advanced topics to a second doc. These include:

Randomness
Records / lenses
List

dan-zheng · 2021-01-10T13:03:47Z

Thanks @srush! I'm happy to continue editing the tutorial today, making progress towards the todos you listed above.

google-cla · 2021-01-10T22:37:12Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

…h#445) The bottom padding adds unnecessary empty bottom scrolling, which slightly hurts UX.

…#444) This involves only web frontend changes via KaTeX CSS and JS: https://katex.org. I chose KaTeX over MathJax because KaTeX seems more modern, performant, and prettier.

Remove `-threaded` option to fix warning: ``` Warning: 'ghc-options: -threaded' has no effect for libraries. It should only be used for executables. ```

- Rebase on top of main branch. - Some code snippets do not yet compile due to syntax changes. This should be straightforward to fix. - Stylistic edits: use consistent terminology and voice. - Remove some second person "you" references. - Use consistent Dex and programming languages terminology and spellings. - Consistently use "array" everywhere: no mentions of "table" - Consistently use "`for` comprehension" - Consistent formatting, punctuation, heading casing (first word uppercase).

apaszke

This is awesome thanks! I left a few general comments, but it looks good.

One big blocker for merging though might be the MNIST binary file. I'd be kind of reluctant to include this because of licensing issues, and git also being really bad when it comes to handling binary files in the history. Can we just include a line such as "oh btw line wget <url> if you want to continue with the tutorial"?

examples/tutorial.dx

srush · 2021-01-14T15:14:57Z

Sure @apaszke I will do that and your other changes.

dan-zheng · 2021-01-14T15:34:53Z

Can we just include a line such as "oh btw line wget <url> if you want to continue with the tutorial"?

Could dataset downloading be a Dex command too, via IO effect and shell command support by @dougalm? "wget to ~/.dex/datasets/mnist if it does not already exist"

Shouldn't be much work to create a basic datasets library.

I filed a feature issue for a Dex datasets library, hope that's not controversial: #458

srush · 2021-01-14T20:34:45Z

Okay, think this mostly complete.

Should I send a PR to the GitHub Pages branch as well or is that automated?

dan-zheng · 2021-01-19T14:50:53Z

Should I send a PR to the GitHub Pages branch as well or is that automated?

My opinion: I think we're happy to handle the GitHub Pages branch regeneration!

Perhaps the tutorial is in a ready state for final review and merging. I'll take my final pass now.

static/dynamic.js

examples/latex.dx

…orial

- Consistent space: no triple newline. - Spell out `tabXX` function names to `tableXX`. - Rename dummy `Add`` and `VSpace`` interface requirements. - Prevents clash with Prelude interfaces and requirements of the same name. - Rename title: "Dex Tutorial" → "Introduction to Dex" - Fix English typos.

google-cla · 2021-01-19T22:22:57Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

…to tutorialfinal

srush · 2021-01-19T22:38:24Z

Thanks @dan-zheng . Think everything checks from my side.

apaszke

Sorry for the delays. It's a busy week, and for some reason the email notifications still had the "[Preliminary]" prefix in your PR title, so that didn't help.

Anyway, this looks good, so I'll go ahead and merge!

apaszke · 2021-01-20T10:39:30Z

examples/tutorial.dx

+  rewrite all operations in terms of batched tensor interactions, but
+  allows for a range of interactions. Put more simply, when learning
+  MATLAB students are told repeatedly to "avoid for loops". *Dex gives
+  for loops back*.


I love this pitch!

google-cla bot added the cla: no label Jan 9, 2021

dan-zheng reviewed Jan 9, 2021

View reviewed changes

examples/tutorial.dx Outdated Show resolved Hide resolved

dan-zheng mentioned this pull request Jan 9, 2021

Edit tutorial by @srush. srush/dex-lang#1

Merged

dan-zheng reviewed Jan 9, 2021

View reviewed changes

examples/tutorial.dx Outdated Show resolved Hide resolved

oxinabox reviewed Jan 9, 2021

View reviewed changes

examples/tutorial.dx Outdated Show resolved Hide resolved

oxinabox reviewed Jan 9, 2021

View reviewed changes

apaszke approved these changes Jan 9, 2021

View reviewed changes

srush and others added 7 commits January 10, 2021 17:47

tutorial

f814a6c

Remove extra CSS bottom padding in HTML-rendered Dex. (google-researc…

a4299a8

…h#445) The bottom padding adds unnecessary empty bottom scrolling, which slightly hurts UX.

Add LaTeX rendering for HTML-rendered Dex via KaTeX. (google-research…

87cf010

…#444) This involves only web frontend changes via KaTeX CSS and JS: https://katex.org. I chose KaTeX over MathJax because KaTeX seems more modern, performant, and prettier.

Fix stack build warning for dex executable target. (google-research#446)

3b8b9e7

Remove `-threaded` option to fix warning: ``` Warning: 'ghc-options: -threaded' has no effect for libraries. It should only be used for executables. ```

Consistently spell "n-D" as "nD".

2c99e48

Update tutorial to use the latest syntax for typeclasses

e9368b1

srush force-pushed the tutorial branch from 8b5d67c to 902a68b Compare January 10, 2021 22:48

google-cla bot added cla: yes and removed cla: no labels Jan 10, 2021

Update tutorial to use string literals

1cbd5b1

apaszke requested changes Jan 14, 2021

View reviewed changes

examples/tutorial.dx Outdated Show resolved Hide resolved

examples/tutorial.dx Outdated Show resolved Hide resolved

examples/tutorial.dx Outdated Show resolved Hide resolved

srush requested a review from apaszke January 14, 2021 20:34

dan-zheng force-pushed the tutorial branch 2 times, most recently from da92945 to 13d9651 Compare January 15, 2021 07:47

srush changed the title ~~[Preliminary] Dex Tutorial~~ Dex Tutorial Jan 15, 2021

dan-zheng approved these changes Jan 19, 2021

View reviewed changes

static/dynamic.js Outdated Show resolved Hide resolved

examples/latex.dx Outdated Show resolved Hide resolved

srush and others added 5 commits January 19, 2021 17:21

Merge branch 'main' into tutorial

937e84e

Merge branch 'tutorial' of https://github.com/srush/dex-lang into tut…

be6a2ae

…orial

remove binary files and update Adam comments

93a7aac

Adam comment

255ccbe

google-cla bot added cla: no and removed cla: yes labels Jan 19, 2021

srush added 2 commits January 19, 2021 17:23

Merge branch 'main' of https://github.com/google-research/dex-lang in…

8a17996

…to tutorialfinal

delete trailing whitespace

d7e872f

srush force-pushed the tutorial branch from b90137d to d7e872f Compare January 19, 2021 22:25

google-cla bot added cla: yes and removed cla: no labels Jan 19, 2021

apaszke approved these changes Jan 20, 2021

View reviewed changes

apaszke merged commit 8aeaf90 into google-research:main Jan 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dex Tutorial #443

Dex Tutorial #443

srush commented Jan 9, 2021 •

edited

Loading

google-cla bot commented Jan 9, 2021

dan-zheng left a comment •

edited

Loading

srush commented Jan 9, 2021

dan-zheng commented Jan 9, 2021

google-cla bot commented Jan 9, 2021

srush commented Jan 9, 2021

google-cla bot commented Jan 9, 2021

dan-zheng commented Jan 9, 2021 •

edited

Loading

google-cla bot commented Jan 9, 2021

srush commented Jan 9, 2021

dan-zheng commented Jan 9, 2021 •

edited

Loading

oxinabox Jan 9, 2021

dan-zheng Jan 9, 2021 •

edited

Loading

apaszke Jan 9, 2021

oxinabox commented Jan 9, 2021 •

edited

Loading

apaszke left a comment

apaszke Jan 9, 2021

google-cla bot commented Jan 10, 2021

google-cla bot commented Jan 10, 2021

srush commented Jan 10, 2021 •

edited

Loading

dan-zheng commented Jan 10, 2021

google-cla bot commented Jan 10, 2021

apaszke left a comment

srush commented Jan 14, 2021

dan-zheng commented Jan 14, 2021 •

edited

Loading

srush commented Jan 14, 2021

dan-zheng commented Jan 19, 2021 •

edited

Loading

google-cla bot commented Jan 19, 2021

srush commented Jan 19, 2021 •

edited

Loading

apaszke left a comment

apaszke Jan 20, 2021

Dex Tutorial #443

Dex Tutorial #443

Conversation

srush commented Jan 9, 2021 • edited Loading

google-cla bot commented Jan 9, 2021

What to do if you already signed the CLA

Individual signers

Corporate signers

dan-zheng left a comment • edited Loading

Choose a reason for hiding this comment

srush commented Jan 9, 2021

dan-zheng commented Jan 9, 2021

google-cla bot commented Jan 9, 2021

What to do if you already signed the CLA

Individual signers

Corporate signers

srush commented Jan 9, 2021

google-cla bot commented Jan 9, 2021

dan-zheng commented Jan 9, 2021 • edited Loading

google-cla bot commented Jan 9, 2021

srush commented Jan 9, 2021

dan-zheng commented Jan 9, 2021 • edited Loading

oxinabox Jan 9, 2021

Choose a reason for hiding this comment

dan-zheng Jan 9, 2021 • edited Loading

Choose a reason for hiding this comment

apaszke Jan 9, 2021

Choose a reason for hiding this comment

oxinabox commented Jan 9, 2021 • edited Loading

apaszke left a comment

Choose a reason for hiding this comment

apaszke Jan 9, 2021

Choose a reason for hiding this comment

google-cla bot commented Jan 10, 2021

google-cla bot commented Jan 10, 2021

srush commented Jan 10, 2021 • edited Loading

dan-zheng commented Jan 10, 2021

google-cla bot commented Jan 10, 2021

apaszke left a comment

Choose a reason for hiding this comment

srush commented Jan 14, 2021

dan-zheng commented Jan 14, 2021 • edited Loading

srush commented Jan 14, 2021

dan-zheng commented Jan 19, 2021 • edited Loading

google-cla bot commented Jan 19, 2021

srush commented Jan 19, 2021 • edited Loading

apaszke left a comment

Choose a reason for hiding this comment

apaszke Jan 20, 2021

Choose a reason for hiding this comment

srush commented Jan 9, 2021 •

edited

Loading

dan-zheng left a comment •

edited

Loading

dan-zheng commented Jan 9, 2021 •

edited

Loading

dan-zheng commented Jan 9, 2021 •

edited

Loading

dan-zheng Jan 9, 2021 •

edited

Loading

oxinabox commented Jan 9, 2021 •

edited

Loading

srush commented Jan 10, 2021 •

edited

Loading

dan-zheng commented Jan 14, 2021 •

edited

Loading

dan-zheng commented Jan 19, 2021 •

edited

Loading

srush commented Jan 19, 2021 •

edited

Loading