mcabbott · lazarusA · Aug 5, 2023 · Aug 5, 2023 · Aug 5, 2023 · Aug 5, 2023
diff --git a/.github/workflows/Documenter.yml b/.github/workflows/Documenter.yml
@@ -0,0 +1,35 @@
+name: Documenter
+on:
+  push:
+    branches:
+      - master
+    tags: '*'
+  pull_request:
+
+concurrency:
+  # Skip intermediate builds: always.
+  # Cancel intermediate builds: only if it is a pull request build.
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: ${{ startsWith(github.ref, 'refs/pull/') }}
+jobs:
+  build:
+    permissions:
+      contents: write
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: julia-actions/setup-julia@v1
+      - uses: julia-actions/cache@v1
+        with:
+          cache-registries: "true"
+      - name: Install documentation dependencies
+        run: julia --project=docs -e 'using Pkg; pkg"dev ."; Pkg.instantiate()'
+      - name: Build and deploy
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # For authentication with GitHub Actions token
+          DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }} # For authentication with SSH deploy key
+          JULIA_DEBUG: "Documenter"
+          DATADEPS_ALWAYS_ACCEPT: true
+        run: |
+          julia --code-coverage=user --project=docs/ --color=yes docs/genfiles.jl
+          julia --code-coverage=user --project=docs/ --color=yes docs/make.jl
diff --git a/.gitignore b/.gitignore
@@ -1,2 +1,14 @@
-.DS_Store
 /Manifest.toml
+/.vscode/
+tmp
+generated
+build
+.vscode
+.DS_Store
+
+docs/docs
+docs/site
+docs/build
+docs/var
+deps/build.jl
+Manifest.toml
diff --git a/README.md b/README.md
@@ -1,9 +1,11 @@
 # AxisKeys.jl
 
-[![Docstrings](https://img.shields.io/badge/j_hub-docstrings-blue)](https://juliahub.com/docs/AxisKeys/)
-[![Github CI](https://github.com/mcabbott/AxisKeys.jl/workflows/CI/badge.svg)](https://github.com/mcabbott/AxisKeys.jl/actions?query=workflow%3ACI+branch%3Amaster)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://github.com/mcabbott/AxisKeys.jl/blob/main/LICENSE)
+[![Docs: Latest](https://img.shields.io/badge/Docs-Latest-blue.svg)](https://github.com/mcabbott/AxisKeys.jl/dev)
+[![Build Status](https://github.com/mcabbott/AxisKeys.jl/workflows/CI/badge.svg)](https://github.com/mcabbott/AxisKeys.jl/actions?query=workflow%3ACI+branch%3Amaster)
+[![Downloads](https://shields.io/endpoint?url=https://pkgs.genieframework.com/api/v1/badge/AxisKeys&label=Downloads)](https://pkgs.genieframework.com?packages=AxisKeys)
 
-<!--<img src="docs/readmefigure.png" alt="block picture" width="400" align="right">-->
+<img src="/docs/src/assets/readmefigure.png" align="right" style="padding-left:10px;" width="150"/>
 
 This package defines a thin wrapper which, alongside any array, stores a vector of "keys" 
 for each dimension. This may be useful to store perhaps actual times of measurements, 
@@ -23,7 +25,7 @@ A = KeyedArray(data; channel=[:left, :right], time=range(13, step=2.5, length=10
 ```
 
 <p align="center">
-<img src="docs/readmeterminal.png" alt="terminal pretty printing" width="550" align="center">
+<img src="docs/src/assets/readmeterminal.png" alt="terminal pretty printing" width="550" align="center">
 </p>
 
 The package aims not to be opinionated about what you store in these "key vectors":
@@ -34,144 +36,6 @@ leaving as much functionality as possible to other packages.
 
 See <a href="#elsewhere">§ eleswhere</a> below for other packages doing similar things.
 
-### Selections
-
-Indexing still works directly on the underlying array, 
-and keyword indexing (of a nested pair) works exactly as for a `NamedDimsArray`.
-But in addition, it is possible to pick out elements based on the keys,
-which for clarity I will call lookup. This is written with round brackets:
-
-| Dimension `d` | Indexing: `i ∈ axes(A,d)` | Lookup: `key ∈ axiskeys(A,d)` |
-|--------------------|---------------------|---------------------|
-| by position        | `A[1,2,:]`          | `A(:left, 15.5, :)` |
-| by name            | `A[iter=1]`         | `A(iter=31)`        |
-| by type            |  --                 | `B = A(:left)`      |
-
-When using dimension names, fixing only some of them will return a slice, 
-such as `B = A[channel=1]`.
-You may also give just one key, provided its type matches those of just one dimension,
-such as `B = A(:left)` where the key is a Symbol.
-
-Note that indexing is the primary way to access the data. Lookup calls for example
-`i = findfirst(axiskeys(A,1), :left)` to convert keys to indices, thus will always be slower.
-If you want this to be the primary mode of access, then you may want a dictionary,
-possibly [Dictionaries.jl](https://github.com/andyferris/Dictionaries.jl).
-
-There are also a numer of special selectors, which work like this:
-
-|                 | Indexing         | Lookup                  |         |
-|-----------------|------------------|-------------------------|---------|
-| one nearest     | `B[time = 3]`    | `B(time = Near(17.0))`  | vector  |
-| all in a range  | `B[2:5, :]`      | `B(Interval(14,25), :)` | matrix  |
-| all matching    | `B[3:end, Not(3)]` | `B(>(17), !=(33))`    | matrix  |
-| mixture         | `B[1, Key(33)]`  | `B(Index[1], 33)`       | scalar  |
-| non-scalar      | `B[iter=[1, 3]]` | `B(iter=[31, 33])`      | matrix  |
-
-Here `Interval(13,18)` can also be written `13..18`, it's from [IntervalSets.jl](https://github.com/JuliaMath/IntervalSets.jl). 
-Any functions can be used to select keys, including lambdas: `B(time = t -> 0<t<17)`. 
-You may give just one `::Base.Fix2` function 
-(such as `<=(18)` or `==(20)`) provided its argument type matches the keys of one dimension.
-An interval or a function always selects via `findall`, 
-i.e. it does not drop a dimension, even if there is exactly one match. 
-
-While this table shows lookup selectors inside `B(...)`, they can in fact all be 
-used inside `B[...]`, not just `Key(k)` as shown. They still refer to keys not indices!
-(This will not select dimension based on type, i.e. `A[Key(:left)]` is an error.)
-You may also write `Index[end]` but not `Index[end-1]`.
-
-By default lookup returns a view, while indexing returns a copy unless you add `@views`. 
-This means that you can write into the array with `B(time = <=(18)) .= 0`.
-For scalar output, you cannot of course write `B(13.0, 33) = 0` 
-as this parsed as a function definition, but you can write `B[Key(13.0), Key(33)] = 0`,
-or else `B(13.0, 33, :) .= 0` as a trailing colon makes a zero-dimensional view.
-
-### Construction
-
-```julia
-KeyedArray(rand(Int8, 2,10), ([:a, :b], 10:10:100)) # AbstractArray, Tuple{AbstractVector, ...}
-```
-
-A nested pair of wrappers can be constructed with keywords for names,
-and everything should work the same way in either order:
-
-```julia
-KeyedArray(rand(Int8, 2,10), row=[:a, :b], col=10:10:100)     # KeyedArray(NamedDimsArray(...))
-NamedDimsArray(rand(Int8, 2,10), row=[:a, :b], col=10:10:100) # NamedDimsArray(KeyedArray(...))
-```
-
-Calling `AxisKeys.keyless(A)` removes the `KeyedArray` wrapper, if any, 
-and `NamedDims.unname(A)` similarly removes the names (regardless of which is outermost).
-
-There is another more "casual" constructor, via the function `wrapdims`.
-This does a bit more checking of inputs, and will adjust the length of ranges of keys if it can, 
-and will fix indexing offsets if needed to match the array.
-The resulting order of wrappers is controlled by `AxisKeys.nameouter()=false`.
-
-```julia
-wrapdims(rand(Int8, 10), alpha='a':'z') 
-# Warning: range 'a':1:'z' replaced by 'a':1:'j', to match size(A, 1) == 10
-
-wrapdims(OffsetArray(rand(Int8, 10),-1), iter=10:10:100)
-axiskeys(ans,1) # 10:10:100 with indices 0:9
-```
-
-Finally, `wrapdims` will also convert `AxisArray`s, `NamedArray`s, as well as `NamedTuple`s. 
-
-### Functions
-
-The function `axes(A)` returns (a tuple of vectors of) indices as usual, 
-and `axiskeys(A)` similarly returns (a tuple of vectors of) keys.
-If the array has names, then `dimnames(A)` returns them.
-These functions work like `size(A, d) = size(A, name)` to get just one.
-
-The following things should work:
-
-* Broadcasting `log.(A)` and `map(log, A)`, as well as comprehensions 
-  `[log(x) for x in A]` should all work. 
-
-* Transpose etc, `permutedims`, `mapslices`.
-
-* Concatenation `hcat(B, B .+ 100)` works. 
-  Note that the keys along the glued direction may not be unique afterwards.
-
-* Reductions like `sum(A; dims=:channel)` can use dimension names. 
-  Likewise `prod`, `mean` etc., and `dropdims`.
-
-* Sorting: `sort` and `sortslices` permute keys & data by the array, 
-  while a new function `sortkeys` goes by the keys.
-  `reverse` similarly re-orders keys to match data.
-
-* Some linear algebra functions like `*` and `\` will work. 
-
-* Getproperty returns the key vector, to allow things like
-  `for (i,t) in enumerate(A.time); fun(val = A[i,:], time = t); ...`.
-
-* Vectors support `push!(V, val)`, which will try to extend the key vector. 
-  There is also a method `push!(V, key => val)` which pushes in a new key. 
-
-To allow for this limited mutability, `V.keys isa Ref` for vectors, 
-while `A.keys isa Tuple` for matrices & higher. But `axiskeys(A)` always returns a tuple.
-
-* Named tuples can be converted to and from keyed vectors,
-  with `collect(keys(nt)) == Symbol.(axiskeys(V),1)`
-
-* The [Tables.jl](https://github.com/JuliaData/Tables.jl) interface is supported,
-  with `wrapdims(df, :val, :x, :y)` creating a matrix from 3 columns.
-
-* Some [StatsBase.jl](https://github.com/JuliaStats/StatsBase.jl) and 
-  [CovarianceEstimation.jl](https://github.com/mateuszbaran/CovarianceEstimation.jl) functions 
-  are supported. ([PR#28](https://github.com/mcabbott/AxisKeys.jl/pull/28).)
-
-* [FFTW](https://github.com/JuliaMath/FFTW.jl)`.fft` transforms the keys; 
-  if these are times such as [Unitful](https://github.com/PainterQubits/Unitful.jl)`.s` 
-  then the results are fequency labels. ([PR#15](https://github.com/mcabbott/AxisKeys.jl/pull/15).)
-
-* [LazyStack](https://github.com/mcabbott/LazyStack.jl)`.stack` understands names and keys.
-  Stacks of named tuples like `stack((a=i, b=i^2) for i=1:5)` create a matrix with `[:a, :b]`.
-
-* [NamedPlus](https://github.com/mcabbott/NamedPlus.jl) has a macro which works on comprehensions:
-  `@named [n^pow for n=1:10, pow=0:2:4]` has names and keys.
-
 ### Absent
 
 * There is no automatic alignment of dimensions by name. 
@@ -268,7 +132,6 @@ the keys with the axes in its own `Axis` type. This is returned by `Base.axes(A)
 
 See also [docs/speed.jl](docs/speed.jl) for some checks on this package, 
 and comparisons to other ones.
-And see [docs/repl.jl](docs/repl.jl) for some usage examples, showing pretty printing. 
 
 In 🐍-land:
 

diff --git a/docs/Project.toml b/docs/Project.toml
@@ -0,0 +1,19 @@
+[deps]
+AcceleratedArrays = "44e12807-9a19-5591-91cf-c1b4fb89ce64"
+AxisArrays = "39de3d68-74b9-583c-8d2d-e117c070f3a9"
+AxisKeys = "94b1ba4f-4ee9-5380-92f1-94cde586c3c5"
+BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
+CategoricalArrays = "324d7699-5711-5eae-9e2f-1d82baa6b597"
+DataStructures = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
+DimensionalData = "0703355e-b756-11e9-17c0-8b28908087d0"
+Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
+DocumenterMarkdown = "997ab1e6-3595-5248-9280-8efb232c3433"
+LazyStack = "1fad7336-0346-5a1a-a56f-a06ba010965b"
+Literate = "98b081ad-f1c9-55d3-8b20-4c87d4299306"
+NamedArrays = "86f7a689-2022-50b4-a561-43c23ac3c673"
+NamedDims = "356022a1-0364-5f58-8944-0da4b18d706f"
+OffsetArrays = "6fe1bfb0-de20-5000-8ca7-80f57d26f881"
+Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
+Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
+UniqueVectors = "2fbcfb34-fd0c-5fbb-b5d7-e826d8f5b0a9"
+Unitful = "1986cc42-f94f-5a68-af5c-568840ba703d"
diff --git a/docs/_overrides/partials/source.html b/docs/_overrides/partials/source.html
@@ -0,0 +1,30 @@
+{% import "partials/language.html" as lang with context %}
+<a href="{{ config.repo_url }}" title="{{ lang.t('source.link.title') }}" class="md-source" data-md-component="source">
+  <div class="md-source__icon md-icon">
+    {% set icon = config.theme.icon.repo or "fontawesome/brands/git-alt" %}
+    {% include ".icons/" ~ icon ~ ".svg" %}
+  </div>
+  <div class="md-source__repository">
+    {{ config.repo_name }}
+  </div>
+</a>
+{% if config.theme.twitter_url %}
+<a href="{{ config.theme.twitter_url }}" title="Go to Twitter" class="md-source">
+  <div class="md-source__icon md-icon">
+    {% include ".icons/fontawesome/brands/twitter.svg" %}
+  </div>
+  <div class="md-source__repository">
+    {{ config.theme.twitter_name }}
+  </div>
+</a>
+{% endif %}
+{% if config.theme.sponsor_url %}
+<a href="{{ config.theme.sponsor_url }}" title="Go to Sponsor" class="md-source">
+  <div class="md-source__icon md-icon">
+    {% include ".icons/fontawesome/regular/heart.svg" %}
+  </div>
+  <div class="md-source__repository">
+    {{ config.theme.sponsor_name }}
+  </div>
+</a>
+{% endif %}
diff --git a/docs/genfiles.jl b/docs/genfiles.jl
@@ -0,0 +1,28 @@
+using Documenter, DocumenterMarkdown
+using Literate
+
+get_example_path(p) = joinpath(@__DIR__, ".", "scripts", p)
+OUTPUT = joinpath(@__DIR__, "src", "scripts", "generated")
+
+folders = readdir(joinpath(@__DIR__, ".", "scripts"))
+setdiff!(folders, [".DS_Store"])
+
+function getfiles()
+    srcsfiles = []
+    for f in folders
+        names = readdir(joinpath(@__DIR__, ".", "scripts", f))
+        setdiff!(names, [".DS_Store"])
+        fpaths  = "$(f)/" .* names
+        srcsfiles = vcat(srcsfiles, fpaths...)
+    end
+    return srcsfiles
+end
+
+srcsfiles = getfiles()
+
+for (d, paths) in (("tutorial", srcsfiles),)
+    for p in paths
+    Literate.markdown(get_example_path(p), joinpath(OUTPUT, dirname(p));
+            documenter=true)
+    end
+end
diff --git a/docs/make.jl b/docs/make.jl
@@ -0,0 +1,27 @@
+using Documenter, DocumenterMarkdown
+using AxisKeys, BenchmarkTools
+
+makedocs(
+    modules=[AxisKeys],
+    clean=true,
+    doctest=false,
+    #format   = Documenter.HTML(prettyurls = get(ENV, "CI", nothing) == "true"),
+    sitename="AxisKeys.jl",
+    authors="Michael Abbott et al.",
+    strict=[
+        :doctest,
+        :linkcheck,
+        :parse_error,
+        :example_block,
+        # Other available options are
+        # :autodocs_block, :cross_references, :docs_block, :eval_block, :example_block,
+        # :footnote, :meta_block, :missing_docs, :setup_block
+    ], checkdocs=:all, format=Markdown(), draft=false,
+    build=joinpath(@__DIR__, "docs")
+)
+
+deploydocs(; repo="https://github.com/mcabbott/AxisKeys.jl", push_preview=true,
+    deps=Deps.pip("mkdocs", "pygments", "python-markdown-math", "mkdocs-material",
+        "pymdown-extensions", "mkdocstrings", "mknotebooks",
+        "pytkdocs_tweaks", "mkdocs_include_exclude_files", "jinja2"),
+    make=() -> run(`mkdocs build`), target="site", devbranch="master")