Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Edit tutorial by @srush. #1

Merged
merged 120 commits into from
Jan 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
120 commits
Select commit Hold shift + click to select a range
945aa65
sketching out a particle filter example
mattjj Aug 26, 2020
b6d3636
Finish particle filter and move some shared utilities to the prelude.
dougalm Aug 27, 2020
673b5fb
Strengthen warning about experimental status
dougalm Dec 4, 2020
f8f9282
Merge pull request #353 from google-research/dev
dougalm Dec 18, 2020
16a698d
Make separate directories for tests and libraries.
dougalm Dec 18, 2020
45e5454
Fix GHC warnings.
dougalm Dec 18, 2020
d53771b
Update examples and prelude to "modern Dex".
dougalm Dec 18, 2020
62dd839
Merge branch 'updating-examples' into particle-filter
dougalm Dec 19, 2020
6e2d625
Merge pull request #358 from google-research/particle-filter
dougalm Dec 19, 2020
7ce56f7
Delete duplicate prelude left over from merge
dougalm Dec 19, 2020
b367ddc
Make `splitKey` produce a table instead of a pair, subsuming `splitKe…
dougalm Dec 19, 2020
8646638
Merge pull request #357 from google-research/updating-examples
dougalm Dec 19, 2020
d854025
[README] Edit build dependency instructions. (#360)
dan-zheng Dec 19, 2020
5dc6a6f
Add an IO effect, modeled as `{State World}` with a special `World` t…
dougalm Dec 20, 2020
6acdf92
Expose some unix file operations using the IO effect.
dougalm Dec 21, 2020
433916d
Treat all FFI functions as potentially IO-causing.
dougalm Dec 21, 2020
1d81135
Make it possible to export JITed Dex functions to Python
apaszke Dec 11, 2020
a1982fb
Use the IO effect to expose malloc/free/load/store.
dougalm Dec 21, 2020
e9847eb
Generalize load/store/alloc/free to `Ptr a` for `Storable a`.
dougalm Dec 21, 2020
067ba2f
Fix atan2 bug and add more tests.
duvenaud Dec 21, 2020
fc825c1
Automate wrapping of JITed function pointers in Python callables
apaszke Dec 21, 2020
2212cc7
Updated fluidsim example to have a gradient check, and be more general.
duvenaud Dec 22, 2020
a61f1fa
Clean up the Cabal file
apaszke Dec 23, 2020
75c4cb6
Improve the benchmark harness
apaszke Dec 23, 2020
d83d756
Make a "virtual stdout" stream that we can write to using the IO effect.
dougalm Dec 23, 2020
98c0977
Merge branch 'main' into IO-effect
dougalm Dec 23, 2020
459da30
Parse string literals as lists instead of tables.
dougalm Dec 23, 2020
cf8b260
Add strings to runtime error messages.
dougalm Dec 24, 2020
e74e306
Fix `stack build` on macOS. (#378)
dan-zheng Dec 24, 2020
56b128e
Add an IO-based dynamically expanding buffer and use it to implement …
dougalm Dec 24, 2020
2043334
Add animations by shelling out to imagemagick to turn PNGs into a GIF.
dougalm Dec 24, 2020
5b2eb9b
Merge branch 'main' into IO-effect
dan-zheng Dec 25, 2020
9674910
Make show identity on strings
oxinabox Dec 23, 2020
269c26c
Refactor diagram.dx
oxinabox Dec 24, 2020
bdf1df1
Make changes suggested in review.
dougalm Dec 26, 2020
ea4d91c
Merge branch 'main' into IO-effect
dougalm Dec 26, 2020
5eb53d7
Update MCMC example
devmotion Dec 26, 2020
91c526b
Merge pull request #383 from devmotion/mcmc
dougalm Dec 26, 2020
70a645f
Fix `stack build` on macOS again. (#381)
dan-zheng Dec 26, 2020
55df897
Tweak MCMC example to fix quine test and make it faster to run.
dougalm Dec 26, 2020
e15835e
Extend rejection sampling example
devmotion Dec 27, 2020
49c0411
Treat effect rows as sets of effects. Fixes #267.
dougalm Dec 28, 2020
e0742a5
Add a special case to effect inference to accept subsets of allowed e…
dougalm Dec 28, 2020
afcddfe
Make a simpler iteration combinator that lets effects carry any neede…
dougalm Dec 29, 2020
e138971
Add `do <expr>` as syntactic sugar for `\(). <expr>`.
dougalm Dec 29, 2020
42193e3
Add shell.nix.
athas Dec 28, 2020
dab1c4f
Add cabal.project.
athas Dec 28, 2020
c595c6a
Print floats with fewer decimal places to make quine tests less brittle.
dougalm Dec 29, 2020
11e2b53
Merge branch 'main' into polymorphic-effects
dougalm Dec 29, 2020
09509aa
Merge branch 'rejection' of https://github.com/devmotion/dex-lang int…
dougalm Dec 29, 2020
28b96bd
Add rejection sampler to tests and update it to use iteration combina…
dougalm Dec 29, 2020
be7de05
Improve HTML doc generation.
dan-zheng Dec 26, 2020
ab33856
Remove css minification.
dougalm Dec 29, 2020
3467bd0
Add a `getEnv` wrapper around POSIX `getenv`.
dougalm Dec 29, 2020
4ee5f7a
Use `DEX_TEST_MODE` flag to choose different params when testing.
dougalm Dec 29, 2020
ed270a1
Add Text support to Diagram
oxinabox Dec 23, 2020
d4d6ec5
center text element
oxinabox Dec 29, 2020
db019ee
Made diagram demos literate comments
oxinabox Dec 29, 2020
078e7dc
Delete stale tests
dougalm Dec 29, 2020
69ceb2f
Delete stale code. Fixes #342.
dougalm Dec 29, 2020
4a1fe72
Update or delete stale examples
dougalm Dec 29, 2020
affa9fe
Merge branch 'ox/text' of https://github.com/oxinabox/dex-lang into o…
dougalm Dec 29, 2020
f836ba5
Improved linear algebra example.
duvenaud Dec 23, 2020
a3d65fe
Start adding an exception effect - syntax and parser.
dougalm Dec 30, 2020
d449ebf
Add `throw` and `catch` primitives and lower them away in simp pass.
dougalm Dec 30, 2020
003f1e2
Add a helper for emitting case expressions on `Maybe a` scrutinees.
dougalm Dec 30, 2020
2c09957
Handle `for` expressions when lowering away exceptions.
dougalm Dec 30, 2020
5be588f
Expose `view`, a "lazy" variant of `for`.
dougalm Dec 30, 2020
dadfaec
Merge branch 'exception-effect'
dougalm Dec 31, 2020
0ecf6bb
Start a pure-Dex parser combinator library
dougalm Dec 30, 2020
9ffb81f
Add some more parser combinators.
dougalm Dec 30, 2020
59a507d
Simplify `while`. It doesn't need separate `cond` and `body` functions.
dougalm Dec 31, 2020
4425cb3
Handle exceptions under `while` and `runState`.
dougalm Dec 31, 2020
9da58a1
Make a newtype wrapper for `Parser a`.
dougalm Dec 31, 2020
18926b5
Add a parser for integers. Fixes #312.
dougalm Dec 31, 2020
cf0f508
Fix Imp code I accidentally left commented out.
dougalm Dec 31, 2020
58c2e79
New `run/with/yield` naming convention for effect runners.
dougalm Dec 31, 2020
0b76c12
clang install instructions
joaogui1 Dec 31, 2020
37e31a1
Make the `else` clause of `if..then..else` optional.
dougalm Dec 31, 2020
d0f4702
Update README.md
apaszke Jan 2, 2021
c858f8e
clang install instructions (#405)
apaszke Jan 2, 2021
c004ceb
Make README a little nicer, add pointers to new label system
apaszke Jan 4, 2021
66d225f
Add completions to REPL (#407)
oxinabox Jan 4, 2021
cd7cd09
Add new syntax for type class constraints on `def` decls. Fixes #370.
dougalm Dec 31, 2020
599e2ee
Overhaul new-style interface/instance decls.
dougalm Jan 4, 2021
567faa2
Enable BlockArguments by default (we use the bracketing pattern a lot)
dougalm Jan 4, 2021
68cefc8
Infer types of implicit implicit arguments.
dougalm Jan 4, 2021
dba12d0
Make fixes suggested in review.
dougalm Jan 5, 2021
8283de6
Make scoping of binders in instance declarations more explicit.
dougalm Jan 5, 2021
6b73c50
Add some comments suggested by Adam.
dougalm Jan 5, 2021
baa7589
Remove `Storable` instance for pairs due to alignment concerns.
dougalm Jan 6, 2021
be2f552
Make `IO` its own distinct effect instead of using `State World`.
dougalm Jan 6, 2021
3d1146e
Continue inlining after application
apaszke Jan 6, 2021
72f19dc
Fix the stupid zext bug
apaszke Jan 6, 2021
de34e9f
Harden CI checks
apaszke Jan 6, 2021
9e6e470
Revert #376 to fix segfaults in main
apaszke Jan 6, 2021
43f7758
[README] Fix typo. (#436)
dan-zheng Jan 7, 2021
4002ba7
Use llvm-hs/llvm-hs@llvm-9 in cabal.project.
dan-zheng Jan 7, 2021
e17e584
Basic JAX integration
apaszke Jan 6, 2021
8282129
Add a CPU XLA translation rule for the dex_call primitive (JAX interop)
apaszke Jan 7, 2021
4036c60
[NFC] Gardening.
dan-zheng Jan 7, 2021
cf06f3d
Gardening.
dan-zheng Jan 7, 2021
0451c7c
Add .hlint.yaml to match repository coding style preferences.
dan-zheng Jan 7, 2021
37e4f50
Make `--backend` and `--outfmt` options be consistently lowercase.
dan-zheng Jan 8, 2021
5af3a29
Merge pull request #437 from dan-zheng/gardening
dan-zheng Jan 8, 2021
08db4cf
Fix `stack build` on macOS. (#441)
dan-zheng Jan 8, 2021
1c0a417
Put `SumAsProd` accesses under switches to avoid illegal dereferencin…
dougalm Jan 6, 2021
ae2492d
Small changes that were blocked on #348.
dougalm Jan 7, 2021
e979cae
Fix a shadowing bug by making `emitBlock` freshen its binders.
dougalm Jan 7, 2021
d37b9cb
Strengthen the Imp type checks and add a debugging printf to JIT.
dougalm Jan 7, 2021
d3fd463
Fix conversion of Word8 to hex string for diagram colors.
dougalm Jan 7, 2021
113bdf7
Revert "Revert #376 to fix segfaults in main"
dougalm Jan 7, 2021
f461093
Changes suggested in review.
dougalm Jan 8, 2021
f713b3d
Add `make watch` command for automatic rebuilding. (#442)
dan-zheng Jan 8, 2021
0dab3fa
Remove extra CSS bottom padding in HTML-rendered Dex. (#445)
dan-zheng Jan 9, 2021
c3f6a10
Add LaTeX rendering for HTML-rendered Dex via KaTeX. (#444)
dan-zheng Jan 9, 2021
f7459d6
Fix stack build warning for dex executable target. (#446)
dan-zheng Jan 9, 2021
14a1dfb
Merge branch 'main' of github.com:google-research/dex-lang into tutor…
dan-zheng Jan 9, 2021
f521e30
Edit tutorial by @srush.
dan-zheng Jan 9, 2021
1288790
Consistently spell "n-D" as "nD".
dan-zheng Jan 9, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,12 @@ jobs:
run: rm -rf ~/.stack/setup-exe-cache
if: runner.os == 'macOS'

- name: Build, treating warnings as errors
run: make build-ci
if: runner.os == 'Linux'

- name: Build
run: make
run: make build

- name: Run tests
run: make tests
2 changes: 2 additions & 0 deletions .hlint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
- arguments: [--color]
- ignore: {name: "Use fmap"}
36 changes: 22 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,25 +12,33 @@ To learn more, check out our
or these example programs:

* [Dex prelude](https://google-research.github.io/dex-lang/prelude.html)
* [Mandelbrot set](https://google-research.github.io/dex-lang/mandelbrot.html)
* [Ray tracer](https://google-research.github.io/dex-lang/raytrace.html)
* [Estimating pi](https://google-research.github.io/dex-lang/pi.html)
* [Hamiltonian Monte Carlo](https://google-research.github.io/dex-lang/mcmc.html)
* [ODE integrator](https://google-research.github.io/dex-lang/ode-integrator.html)
* [Sierpinski triangle](https://google-research.github.io/dex-lang/sierpinski.html)
* [Basis function regression](https://google-research.github.io/dex-lang/regression.html)
* [Brownian bridge](https://google-research.github.io/dex-lang/brownian_motion.html)

Please note that Dex is an experimental research project at an early stage of
development. Contributions welcome!
* [Mandelbrot set](https://google-research.github.io/dex-lang/examples/mandelbrot.html)
* [Ray tracer](https://google-research.github.io/dex-lang/examples/raytrace.html)
* [Estimating pi](https://google-research.github.io/dex-lang/examples/pi.html)
* [Hamiltonian Monte Carlo](https://google-research.github.io/dex-lang/examples/mcmc.html)
* [ODE integrator](https://google-research.github.io/dex-lang/oexamples/de-integrator.html)
* [Sierpinski triangle](https://google-research.github.io/dex-lang/examples/sierpinski.html)
* [Basis function regression](https://google-research.github.io/dex-lang/examples/regression.html)
* [Brownian bridge](https://google-research.github.io/dex-lang/examples/brownian_motion.html)

🚨 **Dex is an experimental research project at an early stage of
development. Expect monstrous bugs and razor-sharp edges!**

🤝 **Contributions welcome!** See our issue tracker for [good first issues](https://github.com/google-research/dex-lang/labels/good%20first%20issue), or browse by [thematic labels](https://github.com/google-research/dex-lang/labels).

## Dependencies

* Install [stack](https://www.haskellstack.org)
* Install LLVM 9
* `apt-get install llvm-9-dev` on Ubuntu/Debian,
* `brew install llvm@9` on macOS, and ensure it is on your `PATH` e.g. via `export PATH="$(brew --prefix llvm@9)/bin:$PATH"` before building.
* Install libpng (often included by default in *nix)
* Ubuntu/Debian: `apt-get install llvm-9-dev`
* macOS: `brew install llvm@9`
* Make sure `llvm@9` is on your `PATH` before building. Example: `export PATH="$(brew --prefix llvm@9)/bin:$PATH"`
* Install clang (may be installed together with llvm)
* Ubuntu/Debian: `apt-get install clang`
* macOS: installs with llvm
* Install libpng (often included by default in *nix platforms)
* Ubuntu/Debian: `apt-get install libpng-dev`
* macOS: `brew install libpng`

## Building

Expand Down
6 changes: 3 additions & 3 deletions benchmarks/dexbench.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,15 +55,15 @@ def restore_machine():
def run_benches(lang, backend):
if lang == "dex":
if backend == "CPU":
backend_args = ["--backend", "LLVM-MC"]
backend_args = ["--backend", "llvm-mc"]
env = {}
elif backend == "GPU":
backend_args = ["--backend", "LLVM-CUDA"]
backend_args = ["--backend", "llvm-cuda"]
env = {"CUDA_LAUNCH_BLOCKING":"1"}
else:
raise Exception
command = (["stack", "exec", "dex", "--"] + backend_args +
["script", "--outfmt", "JSON", dex_microbench_file])
["script", "--outfmt", "json", dex_microbench_file])
elif lang == "jax":
if backend == "CPU":
env = {"CUDA_VISIBLE_DEVICES":""}
Expand Down
13 changes: 13 additions & 0 deletions cabal.project
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
packages: dex.cabal

source-repository-package
type: git
location: https://github.com/llvm-hs/llvm-hs
tag: llvm-9
subdir: llvm-hs

source-repository-package
type: git
location: https://github.com/llvm-hs/llvm-hs
tag: llvm-9
subdir: llvm-hs-pure
66 changes: 32 additions & 34 deletions dex.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ name: dex
version: 0.1.0.0
author: Dougal Maclaurin
maintainer: [email protected]
license-file: LICENSE
build-type: Simple

flag cuda
Expand All @@ -32,17 +33,22 @@ library
exposed-modules: Env, Syntax, Type, Inference, JIT, LLVMExec,
Parser, Util, Imp, Imp.Embed, Imp.Optimize,
PPrint, Algebra, Parallelize, Optimize, Serialize
Actor, Cat, Flops, Embed,
Actor, Cat, Embed, Export,
RenderHtml, LiveOutput, Simplify, TopLevel,
Autodiff, Interpreter, Logging, PipeRPC, CUDA,
Autodiff, Interpreter, Logging, CUDA,
LLVM.JIT, LLVM.Shims
build-depends: base, containers, mtl, binary, bytestring,
time, tf-random, llvm-hs-pure ==9.*, llvm-hs ==9.*,
aeson, megaparsec >=8.0, warp, wai, filepath,
parser-combinators, http-types, prettyprinter, text,
blaze-html, cmark, diagrams-lib, ansi-terminal,
transformers, directory, mmap, unix,
process, primitive, store, dex-resources, temporary,
build-depends: base, containers, mtl, bytestring,
llvm-hs-pure, llvm-hs,
-- Parsing
megaparsec, parser-combinators,
-- Text output
prettyprinter, text,
-- Portable system utilities
filepath, directory, ansi-terminal, process, temporary,
-- Serialization
store,
-- Notebook support
warp, wai, blaze-html, aeson, http-types, cmark, binary
if !os(darwin)
exposed-modules: Resources
hs-source-dirs: src/resources
Expand All @@ -51,11 +57,12 @@ library
build-depends: dex-resources
default-language: Haskell2010
hs-source-dirs: src/lib
ghc-options: -Wall -fPIC
ghc-options: -Wall -fPIC -threaded -optP-Wno-nonportable-include-path
cxx-sources: src/lib/dexrt.cpp
cxx-options: -std=c++11 -fPIC
default-extensions: CPP, DeriveTraversable, TypeApplications, OverloadedStrings,
TupleSections, ScopedTypeVariables, LambdaCase, PatternSynonyms
TupleSections, ScopedTypeVariables, LambdaCase, PatternSynonyms,
BlockArguments
pkgconfig-depends: libpng
if flag(cuda)
include-dirs: /usr/local/cuda/include
Expand All @@ -71,42 +78,33 @@ executable dex
main-is: dex.hs
other-extensions: OverloadedStrings
build-depends: dex, base, haskeline, prettyprinter, mtl,
optparse-applicative, unix, store, bytestring, directory
optparse-applicative, ansi-wl-pprint,
unix, store, bytestring, directory
if os(darwin)
build-depends: dex-resources
default-language: Haskell2010
hs-source-dirs: src
default-extensions: CPP, LambdaCase
default-extensions: CPP, LambdaCase, BlockArguments
ghc-options: -optP-Wno-nonportable-include-path
if flag(optimized)
ghc-options: -O3
else
ghc-options: -O0

foreign-library Dex
type: native-shared
other-modules: API
build-depends: base, dex, dex-resources, mtl
hs-source-dirs: src/foreign
c-sources: src/foreign/rts.c
other-modules: Dex.Foreign.API, Dex.Foreign.Util, Dex.Foreign.JIT
, Dex.Foreign.Context, Dex.Foreign.Serialize
build-depends: base, mtl, containers, llvm-hs, dex, dex-resources
if os(darwin)
build-depends: dex-resources
hs-source-dirs: src/
c-sources: src/Dex/Foreign/rts.c
cc-options: -std=c11 -fPIC
ghc-options: -Wall -fPIC
default-language: Haskell2010
default-extensions: TypeApplications, ScopedTypeVariables, LambdaCase
if flag(optimized)
ghc-options: -O3
else
ghc-options: -O0

Test-Suite test-dex
type: exitcode-stdio-1.0
main-is: PropTests.hs
build-depends: dex, base, prettyprinter, containers,
hedgehog, microlens-platform, mtl
other-modules: GenExpr, TestPass
ghc-options: -Wall -fPIC -optP-Wno-nonportable-include-path
default-language: Haskell2010
hs-source-dirs: tests
ghc-options: cbits/libdex.so
-Wall
default-extensions: TypeApplications, ScopedTypeVariables, LambdaCase,
BlockArguments
if flag(optimized)
ghc-options: -O3
else
Expand Down
56 changes: 0 additions & 56 deletions examples/ad-tests-interp.dx

This file was deleted.

48 changes: 0 additions & 48 deletions examples/aspirational.dx

This file was deleted.

3 changes: 0 additions & 3 deletions examples/bad-binary-file.dxbo

This file was deleted.

26 changes: 13 additions & 13 deletions examples/brownian_motion.dx
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@

include "plot.dx"

UnitInterval = Float

bmIter : (Key & Float & Float & UnitInterval) -> (Key & Float & Float & UnitInterval) =
\(key, y, sigma, t).
(kDraw, kL, kR) = splitKey3 key
t' = abs (t - 0.5)
y' = sigma * randn kDraw * (0.5 - t')
key' = select (t > 0.5) kL kR
(key', y + y', sigma / sqrt 2.0, t' * 2.0)
def bmIter ((key, y, sigma, t):(Key & Float & Float & UnitInterval)) :
(Key & Float & Float & UnitInterval) =
[kDraw, kL, kR] = splitKey key
t' = abs (t - 0.5)
y' = sigma * randn kDraw * (0.5 - t')
key' = select (t > 0.5) kL kR
(key', y + y', sigma / sqrt 2.0, t' * 2.0)

sampleBM : Key -> UnitInterval -> Float =
\key t.
(_, y, _, _) = fold (key, 0.0, 1.0, t) \i:(Fin 10). bmIter
y
def sampleBM (key:Key) (t:UnitInterval) : Float =
(_, y, _, _) = fold (key, 0.0, 1.0, t) \i:(Fin 10). bmIter
y

xs = linspace (Fin 1000) 0.0 1.0
ys = map (sampleBM (newKey 0)) xs

-- :plot zip xs ys
-- > <html output>
:html showPlot $ xyPlot xs ys
> <html output>
35 changes: 0 additions & 35 deletions examples/bugs.dx

This file was deleted.

Loading