Edit tutorial by @srush. #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Dougal Maclaurin <[email protected]>
Merging dev back into main
(Rather than lumping them all together under `examples/`.)
…filter Particle filter example
…y3` etc. Now that we have pattern matching on table constructors, this is just as convenient to use, because the pattern tells the function how many keys to produce. Also add an `Arbitrary` type class for creating dummy data at any type.
…examples Reorganize examples/tests/libs and update to "modern Dex"
Use consistent formatting and style.
…oken. The goal is to let us sequence effectful FFI calls, without worrying about reordering and DCE, and then wrap the sequence in `unsafePerformIO` to expose it as a pure function.
There's still a lot to do around errors and thread safety.
This just forces us to explicitly wrap with `unsafeIO` when that's what we mean. Previously it was too easy to forget the IO tag.
The API is still far from ergonomic, because it returns raw function pointers, with signatures derived from pretty obscure Haskell code. The next step will be to clean up the expected signatures and handle all the destination allocation and dereferencing internally. This is a big-ish change, mostly because it required a further refactor of our compilation pipeline, and a few additions to the llvm-hs APIs (non-bracketed context and module management). Until those changes are merged upstream, I changed `stack.yaml` to point to the PR branch in my fork.
The memory discipline is C-style: the user is responsible for freeing what they allocate. I added some bracketed functions like `withAlloc` to make this easier. This lets us retire `%getPtr` which was always a bit dodgy.
This patch extends the export functionality with the ability to generate
a description of how the user-facing arguments and results map to the
exported native function. Then, I created a mini-language that can
faithfully encode this description, and is exposed in the foreign API
via the `dexGetFunctionSignature` function.
On the Python side, I have implemented a parser for that language, which
turns it into a few `NativeType` objects that describe how the arguments
are to be (de)serialized when interfacing with the native function. This
completely automates the manual labor of creating ctype wrappers and
allows used to call the compiled function as if it was any other Python
callable.
Note that this interface also supports zero-copy bidirectional
conversion of NumPy arrays and Dex tables with `Fin` index sets.
For example:
```py
import dex
import numpy as np
from textwrap import dedent
sigmoid = dex.eval(r"\x:Float. 1.0 / (1.0 + exp(-x))").compile()
print(sigmoid(-1.0), sigmoid(0.0), sigmoid(1.0))
transpose = dex.Module(dedent("""
def myTranspose (n: Int) ?-> (m: Int) ?->
(x : (Fin n)=>(Fin m)=>Float) : (Fin m)=>(Fin n)=>Float =
for i j. x.j.i
""")).myTranspose.compile()
example = np.arange(25, dtype=np.float32).reshape((5, 5))
print(transpose(example)) # NB: Implicit arguments get inferred from shapes
print(example.T)
```
Remove unused dependencies, group the ones we actually need in categories. Also, remove some files that have bitrotted.
Use a stable timer, and take a single measurement for a whole bunch of runs, to amortize the measurement cost (~1us). Thanks to this, the lowest numbers we can possibly benchmark are ~150ns on my machine (this is for running a `1 + 1` program). This is still suspiciously high, but I think it might be connected to the fact that the program we emit actually does contain a call to `posix_memalign`, and so it might just be the cost of memory allocation (especially that we never free the result!).
We introduce a special global variable, `OUT_STREAM_PTR`, which at runtime resolves to a C stdio `FILE*` pointer. The runtime creates a fresh pipe every time it runs an LLVM program. It puts the write end in `OUT_STREAM_PTR`, and captures what comes out the other end in a separate thread while the LLVM program runs. The Haskell thread that reads the output needs to be able to concurrently with the LLVM program in case the pipe fills up. To ensure this can happen, we use the `-threaded` GHC option.
We can still create fixed-sized tables of characters using `['a', 'b', 'c']`. We should do the same on the pretty-printing side.
It looks like a bigger change than it is because I had to manually re-toposort the prelude.
Disable `-Wnonportable-include-path`. I am not sure what earlier change surfaced this macOS build error or why macOS CI does not encounter it.
Turns out that the inlining pass had a pretty big gaping hole previously: the inlineable tables originate not only from the `for` loops, but also from applications with table lambdas! Previously we completely ignored the second case, potentially failing to fully reducing a table application, and blowing up the run-time complexity as in google-research#346.
Who knew that zero-extending two's complement integers could go badly.
We end up committing a whole bunch of warnings all the time, which create unnecessary noise for others. This turns them into errors, to make sure that none of those slip through the cracks. Also, any segmentation faults or aborts didn't cause the tests to fail in the past, which should hopefully be fixed now.
Of course the segfaults are due to google-research#348, but I'd like to keep main green. Let's resubmit this once the underlying issue is fixed.
tematic -> thematic
Previously, a downstream commit at apaszke/llvm-hs was referenced. That commit has now been merged into llvm-hs/llvm-hs@llvm-9.
Make it possible to wrap Dex function atoms in JAX primitives. For now the only supported JAX functionality is abstract evaluation (shape inference). Next step will be to make it possible to JIT the functions as XLA custom calls.
This makes it possible to use Dex functions in self-contained XLA executables. Unfortunately, Python is still in the loop, as we use ctypes to generate a trampoline that adapts the Dex calling convention to the one expected by XLA. Still, I don't expect it to be too difficult to generate the right signature from Dex, so we should do that at some point in the future.
- Address some hlint warnings. - Add doc comments to `Dag` in LiveOutput.hs. - Improve dex help option printing.
Rename `dex --backend interp` flag to `--backend interpreter`. `interpreter` is a noun and reads more naturally.
Ignore "Use fmap": some use sites (e.g. of `liftM`) are intentional and more
readable.
Consider adding more cases in the future, based on coding style preferences and
noise level from hlint-aware IDEs:
```
- ignore: {name: "Use <$>"}
- ignore: {name: "Eta reduce"}
```
It seems standard for CLIs to use lowercase option names. Changed option names: - `--backend llvm` (was LLVM) - `--backend llvm-cuda` (was LLVM-CUDA) - `--backend llvm-mc` (was LLVM-MC) - `--outfmt html` (was HTML) - `--outfmt json` (was JSON)
Disable `-Wnonportable-include-path` for executable dex in dex.cabal. I am not sure caused this issue to surface. The same issue previously occurred for other targets in dex.cabal. Workaround suggested by discussion: haskell/cabal#4739 (comment)
Also add tests for this bug and for google-research#348.
The new Imp check would have caught the bug from the previous commit.
This reverts commit 9e6e470.
`make watch` invokes `stack build $(STACK_FLAGS) --file-watch`, which watches for source file changes and automatically rebuilds.
…h#445) The bottom padding adds unnecessary empty bottom scrolling, which slightly hurts UX.
…#444) This involves only web frontend changes via KaTeX CSS and JS: https://katex.org. I chose KaTeX over MathJax because KaTeX seems more modern, performant, and prettier.
Remove `-threaded` option to fix warning: ``` Warning: 'ghc-options: -threaded' has no effect for libraries. It should only be used for executables. ```
- Rebase on top of main branch.
- Some code snippets do not yet compile due to syntax changes.
This should be straightforward to fix.
- Stylistic edits: use consistent terminology and voice.
- Remove some second person "you" references.
- Use consistent Dex and programming languages terminology and spellings.
- Consistently use "array" everywhere: no mentions of "table"
- Consistently use "`for` comprehension"
- Consistent formatting, punctuation, heading casing (first word uppercase).
dan-zheng
force-pushed
the
tutorial-edits
branch
from
3a925d0 to
f521e30
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Some edits for the Dex tutorial added in google-research#443.
google-research:mainbranch.forcomprehension" and"forconstructor"Known issues: a LaTeX rendering bug appears in the current version of this tutorial.LaTeX reference should not be done for HTML-rendered code snippetsLaTeX rendering bug fixed in google-research#447.