Merge pull request #192 from willow-ahrens/wma/docs-again

Wma/docs again

docs/make.jl

@@ -17,10 +17,9 @@ makedocs(;
     pages=[
         "Home" => "index.md",
         "Array Formats" => "fibers.md",
-        "The Deets" => "listing.md",
-        "Embedding" => "embed.md",
         "Custom Functions" => "algebra.md",
         "Tensor File I/O" => "fileio.md",
+        "C, C++, ..." => "interop.md",
         "Development Guide" => "development.md",
     ],
 )

docs/src/algebra.md

@@ -6,7 +6,8 @@ CurrentModule = Finch
 
 ## User Functions
 
-Finch supports arbitrary Julia Base functions over [`isbits`](@ref) types.  You
+Finch supports arbitrary Julia Base functions over
+[`isbits`](https://docs.julialang.org/en/v1/base/base/#Base.isbits) types.  You
 can also use your own functions and use them in Finch! Just remember to define
 any special algebraic properties of your functions so that Finch can optimize
 them better. You must declare the properties of your functions before you call
@@ -85,7 +86,9 @@ the behavior of Finch in different ways, and call those Finch functions during
 precompilation, the resulting behavior is undefined.
 
 There are several packages that take similar, but different, approaches to
-allow user participation in staged Julia programming (not to mention Base `eval` or `@generated`): [StagedFunctions.jl](https://github.com/NHDaly/StagedFunctions.jl),
+allow user participation in staged Julia programming (not to mention Base `eval`
+or `@generated`):
+[StagedFunctions.jl](https://github.com/NHDaly/StagedFunctions.jl),
 [GeneralizedGenerated.jl](https://github.com/JuliaStaging/GeneralizedGenerated.jl),
 [RuntimeGeneratedFunctions.jl](https://github.com/SciML/RuntimeGeneratedFunctions.jl),
 or [Zygote.jl](https://github.com/FluxML/Zygote.jl).

docs/src/development.md

@@ -138,7 +138,7 @@ TODO more on the way...
 Every virtual tensor must be in one of two modes: read-only mode or update-only mode. The following functions may be called on virtual tensors throughout their life cycle.
 
 ```@docs
-initialize!
+declare!
 get_reader
 get_updater
 freeze!
@@ -151,7 +151,7 @@ Fiber levels are implemented using the following methods:
 
 ```@docs
 default
-initialize_level!
+declare_level!
 assemble_level!
 reassemble_level!
 freeze_level!

docs/src/fibers.md

@@ -171,6 +171,7 @@ of supported formats is described below:
 @fiber
 fiber
 fiber!
+fiber_abbrev
 ```
 
 ### Level Constructors
@@ -181,59 +182,4 @@ ElementLevel
 SparseListLevel
 SparseCOOLevel
 SparseHashLevel
-```
-
-## 0-Index Compatibility
-
-Julia, Matlab, etc. index arrays [starting at
-1](https://docs.julialang.org/en/v1/devdocs/offset-arrays/). C, python, etc.
-index starting at 0. In a dense array, we can simply subtract one from the
-index, and in fact, this is what Julia will does under the hood when you pass a
-vector [between C to
-Julia](https://docs.julialang.org/en/v1/manual/embedding/#Working-with-Arrays). 
-
-However, for sparse array formats, it's not just a matter of subtracting one
-from the index, as the internal lists of indices, positions, etc all start from
-zero as well. To remedy the situation, Finch defines a handy zero-indexed integer
-type called `CIndex`. The internal representation of `CIndex` is one less than the
-value it represents, and we can use `CIndex` as the index or position type of
-a Finch array to represent arrays in other languages.
-
-For example, if `idx_c`, `ptr_c`, and `val_c` are the internal arrays of a CSC
-matrix in a zero-indexed language, we can represent that matrix as a one-indexed
-Finch array without copying by calling
-```@meta
-DocTestSetup = quote
-    using Finch
-    using Finch: Cindex
-end
-```
-```jldoctest example2
-julia> m = 4; n = 3; ptr_c = [0, 3, 3, 5]; idx_c = [1, 2, 3, 0, 2]; val_c = [1.1, 2.2, 3.3, 4.4, 5.5];
-
-julia> ptr_jl = unsafe_wrap(Array, reinterpret(Ptr{Cindex{Int}}, pointer(ptr_c)), length(ptr_c); own = false)
-4-element Vector{Cindex{Int64}}:
- Cindex{Int64}(0)
- Cindex{Int64}(3)
- Cindex{Int64}(3)
- Cindex{Int64}(5)
-julia> idx_jl = unsafe_wrap(Array, reinterpret(Ptr{Cindex{Int}}, pointer(idx_c)), length(idx_c); own = false)
-5-element Vector{Cindex{Int64}}:
- Cindex{Int64}(1)
- Cindex{Int64}(2)
- Cindex{Int64}(3)
- Cindex{Int64}(0)
- Cindex{Int64}(2)
-julia> A = Fiber(Dense(SparseList{Cindex{Int}, Cindex{Int}}(Element{0.0, Float64}(val_c), m, ptr_jl, idx_jl), n))
-Dense [:,1:3]
-├─[:,1]: SparseList (0.0) [1:Cindex{Int64}(3)]
-│ ├─[Cindex{Int64}(1)]: 1.1
-│ ├─[Cindex{Int64}(2)]: 2.2
-│ ├─[Cindex{Int64}(3)]: 3.3
-├─[:,2]: SparseList (0.0) [1:Cindex{Int64}(3)]
-├─[:,3]: SparseList (0.0) [1:Cindex{Int64}(3)]
-│ ├─[Cindex{Int64}(0)]: 4.4
-│ ├─[Cindex{Int64}(2)]: 5.5
-```
-
-We can also convert between representations by by copying to or from `Cindex` fibers.
+```

docs/src/index.md

@@ -5,11 +5,10 @@ CurrentModule = Finch
 # Finch
 
 [Finch](https://github.com/willow-ahrens/Finch.jl) is an adaptable compiler for
-loop nests over structured arrays. Finch can specialize to tensors with runs of
-repeated values, or to tensors which are sparse (mostly zero). Finch supports
-general sparsity as well as many specialized sparsity patterns, like clustered
-nonzeros, diagonals, or triangles.  In addition to zero, Finch supports
-optimizations over arbitrary fill values and operators.
+loop nests over sparse or otherwise structured arrays. Finch supports general
+sparsity as well as many specialized sparsity patterns, like clustered nonzeros,
+diagonals, or triangles.  In addition to zero, Finch supports optimizations over
+arbitrary fill values and operators, even run-length-compression.
 
 At it's heart, Finch is powered by a domain specific language for coiteration,
 breaking structured iterators into units we call Looplets. The Looplets are
@@ -24,6 +23,7 @@ julia> using Pkg; Pkg.add("Finch")
 
 ## Usage:
 
-We're working on adding more documentation, for now take a look at the
-[benchmarks](https://github.com/willow-ahrens/Finch.jl/blob/main/benchmark/benchmarks.jl)
-for a few example algorithms.
+We're working on adding more documentation, for now take a look at the examples
+for [linear
+algebra](https://github.com/willow-ahrens/Finch.jl/blob/main/apps/linalg.jl) or
+[graphs](https://github.com/willow-ahrens/Finch.jl/blob/main/apps/graphs.jl).

docs/src/interop.md

@@ -0,0 +1,74 @@
+# Using Finch with Other Languages
+
+You can use Finch in other languages through our C interface! We also include
+convenience types for converting between 0-indexed and 1-indexed arrays.
+
+## finch.h
+
+Refer to
+[finch.h](https://github.com/willow-ahrens/Finch.jl/blob/main/embed/finch.h) for
+detailed documentation. The public functions include a few shortcuts for
+constructing finch datatypes, as well as convenience functions for calling Julia
+from C. Refer also to the [Julia
+documentation](https://docs.julialang.org/en/v1/manual/embedding/) for more
+general advice. Refer to the tests for a [working
+example](https://github.com/willow-ahrens/Finch.jl/blob/main/test/embed/test_embed_simple.c)
+of embedding in C. Note that calling `finch_init` will call `jl_init`, as well
+as initializing a few function pointers for the interface. Julia cannot see C
+references to Julia objects, so `finch.h` includes a few functions to introduce
+references on the Julia side that mirror C objects.
+
+## 0-Index Compatibility
+
+Julia, Matlab, etc. index arrays [starting at
+1](https://docs.julialang.org/en/v1/devdocs/offset-arrays/). C, python, etc.
+index starting at 0. In a dense array, we can simply subtract one from the
+index, and in fact, this is what Julia will does under the hood when you pass a
+vector [between C to
+Julia](https://docs.julialang.org/en/v1/manual/embedding/#Working-with-Arrays). 
+
+However, for sparse array formats, it's not just a matter of subtracting one
+from the index, as the internal lists of indices, positions, etc all start from
+zero as well. To remedy the situation, Finch defines a handy zero-indexed integer
+type called `CIndex`. The internal representation of `CIndex` is one less than the
+value it represents, and we can use `CIndex` as the index or position type of
+a Finch array to represent arrays in other languages.
+
+For example, if `idx_c`, `ptr_c`, and `val_c` are the internal arrays of a CSC
+matrix in a zero-indexed language, we can represent that matrix as a one-indexed
+Finch array without copying by calling
+```@meta
+DocTestSetup = quote
+    using Finch
+    using Finch: Cindex
+end
+```
+```jldoctest example2
+julia> m = 4; n = 3; ptr_c = [0, 3, 3, 5]; idx_c = [1, 2, 3, 0, 2]; val_c = [1.1, 2.2, 3.3, 4.4, 5.5];
+
+julia> ptr_jl = unsafe_wrap(Array, reinterpret(Ptr{Cindex{Int}}, pointer(ptr_c)), length(ptr_c); own = false)
+4-element Vector{Cindex{Int64}}:
+ Cindex{Int64}(0)
+ Cindex{Int64}(3)
+ Cindex{Int64}(3)
+ Cindex{Int64}(5)
+julia> idx_jl = unsafe_wrap(Array, reinterpret(Ptr{Cindex{Int}}, pointer(idx_c)), length(idx_c); own = false)
+5-element Vector{Cindex{Int64}}:
+ Cindex{Int64}(1)
+ Cindex{Int64}(2)
+ Cindex{Int64}(3)
+ Cindex{Int64}(0)
+ Cindex{Int64}(2)
+julia> A = Fiber(Dense(SparseList{Cindex{Int}, Cindex{Int}}(Element{0.0, Float64}(val_c), m, ptr_jl, idx_jl), n))
+Dense [:,1:3]
+├─[:,1]: SparseList (0.0) [1:Cindex{Int64}(3)]
+│ ├─[Cindex{Int64}(1)]: 1.1
+│ ├─[Cindex{Int64}(2)]: 2.2
+│ ├─[Cindex{Int64}(3)]: 3.3
+├─[:,2]: SparseList (0.0) [1:Cindex{Int64}(3)]
+├─[:,3]: SparseList (0.0) [1:Cindex{Int64}(3)]
+│ ├─[Cindex{Int64}(0)]: 4.4
+│ ├─[Cindex{Int64}(2)]: 5.5
+```
+
+We can also convert between representations by by copying to or from `Cindex` fibers.

src/Finch.jl

@@ -83,28 +83,6 @@ include("modifiers.jl")
 
 export fsparse, fsparse!, fsprand, fspzeros, ffindnz, countstored
 
-module h
-    using Finch
-    function generate_embed_docs()
-        finch_h = read(joinpath(dirname(pathof(Finch)), "../embed/finch.h"), String)
-        blocks = map(m -> m.captures[1], eachmatch(r"\/\*\!(((?!\*\/)(.|\n|\r))*)\*\/", finch_h))
-        map(blocks) do block
-            block = strip(block)
-            lines = collect(eachline(IOBuffer(block)))
-            key = Meta.parse(strip(lines[1]))
-            body = strip(join(lines[2:end], "\n"))
-            @eval begin
-                """
-                    $($body)
-                """
-                $key
-            end
-        end
-    end
-
-    generate_embed_docs()
-end
-
 include("base/abstractarrays.jl")
 include("base/abstractunitranges.jl")
 include("base/broadcast.jl")

src/FinchNotation/syntax.jl

@@ -68,8 +68,9 @@ end
 """
     initwrite(z)(a, b)
 
-`initwrite(z)` is a function which may assert that `a` [isequal](@ref) to `z`, and
-`returns `b`.  By default, `lhs[] = rhs` is equivalent to `lhs[]
+`initwrite(z)` is a function which may assert that `a`
+[`isequal`](https://docs.julialang.org/en/v1/base/base/#Base.isequal) to `z`,
+and `returns `b`.  By default, `lhs[] = rhs` is equivalent to `lhs[]
 <<initwrite(default(lhs))>>= rhs`.
 """
 initwrite(z) = InitWriter{z}()

src/annihilate.jl

@@ -15,8 +15,9 @@ end
     choose(z)(a, b)
 
 `choose(z)` is a function which returns whichever of `a` or `b` is not
-[isequal](@ref) to `z`. If neither are `z`, then return `a`. Useful for getting
-the first nonfill value in a sparse array.
+[isequal](https://docs.julialang.org/en/v1/base/base/#Base.isequal) to `z`. If
+neither are `z`, then return `a`. Useful for getting the first nonfill value in
+a sparse array.
 ```jldoctest setup=:(using Finch)
 julia> a = @fiber(sl(e(0.0)), [0, 1.1, 0, 4.4, 0])
 SparseList (0.0) [1:5]