sphinx-external-toc-strict

A sphinx extension that allows the documentation site-map (a.k.a Table of Contents) to be defined external to the documentation files.

In normal Sphinx documentation, the documentation site-map is defined via a bottom-up approach - adding toctree directives within pages of the documentation.

This extension facilitates a top-down approach to defining the site-map structure, within a single YAML file.

It also allows for documents not specified in the ToC to be auto-excluded.

• Python 3.10 through 3.12, and 3.13.0a3 and up.

New in 2.0.x:

intersphinx support; ref > url; Sphinx py310+ drop py39;

New in 1.2.x:

create_site no overwrite and existing files informative message; SiteMap.file_format ignore unknown use cases; branches test Windows and MacOS;

This is a fork

sphinx-external-toc-strict is a fork of sphinx-external-toc

Comparison

Matric	TOC	TOC-Strict
intersphinx support	No	Yes!
yaml package	pyyaml / yaml	strictyaml / ruemel.yaml
.hidden.files.rst	Yes	No
docs theme	sphinx-book-theme	alabaster
markdown support	Yes	Yes
both	No	Yes, root doc must be index.rst
dump yaml	use yaml.dump	[package].parsing_strictyaml.dump_yaml
static type checking	patchy	100%
coverage	patchy	90%+
in-code manual	No	Yes

The core API should be compatible. To avoid confusion, on the command line, rather than sphinx-etoc, use sphinx-etoc-strict

The author of sphinx-external-toc [source ToC] is Chris Sewell

The author of sphinx-external-toc-strict [source ToC-strict] is Dave Faulkmore

User Guide

Sphinx Configuration

Add to your conf.py:

source_suffix = [".md", ".rst"]
extensions = ["sphinx_external_toc_strict", "myst-parser"]
external_toc_path = "_toc.yml"  # optional, default: _toc.yml
external_toc_exclude_missing = True

Or to your pyproject.toml:

[tool.sphinx-pyproject]
source_suffix = [".md", ".rst"]
extensions = [
    "sphinx.ext.autodoc",
    "sphinx.ext.autosectionlabel",
    "sphinx.ext.todo",
    "sphinx.ext.doctest",
    "sphinx_paramlinks",
    "sphinx.ext.intersphinx",
    "sphinx.ext.extlinks",
    "sphinx_external_toc_strict",
    "myst_parser",
]
external_toc_path = "_toc.yml"  # optional, default: _toc.yml
external_toc_exclude_missing = true
myst_enable_extensions = ["colon_fence", "html_image"]

Note the external_toc_path is always read as a Unix path, and can either be specified relative to the source directory (recommended) or as an absolute path.

Basic Structure

A minimal ToC defines the top level root key, for a single root document file:

root: intro

The value of the root key will be a path to a file, in Unix format (folders split by /), relative to the source directory, and can be with or without the file extension.

Note

Configure root file

This root file will be set as the master_doc.

Document files can then have a subtrees key - denoting a list of individual toctrees for that document - and in-turn each subtree should have a entries key - denoting a list of children links, that are one of:

• file: path to a single document file in Unix format, with or without the file extension (as for root)
• glob: path to one or more document files via Unix shell-style wildcards (similar to fnmatch, but single stars don't match slashes.)
• url: path for an external URL (starting e.g. http or https)

Important

Each document file can only occur once in the ToC!

This can proceed recursively to any depth.

root: intro
subtrees:
- entries:
  - file: doc1
    subtrees:
    - entries:
      - file: doc2
        subtrees:
        - entries:
          - file: doc3
  - url: https://example.com
  - glob: subfolder/other*

This is equivalent to having a single toctree directive in intro, containing doc1, and a single toctree directive in doc1, with the glob: flag and containing doc2, https://example.com and subfolder/other*.

As a shorthand, the entries key can be at the same level as the file, which denotes a document with a single subtree.

For example, this file is exactly equivalent to the one above:

root: intro
entries:
- file: doc1
  entries:
  - file: doc2
    entries:
    - file: doc3
- url: https://example.com
- glob: subfolder/other*

File and URL titles

By default, the initial header within a file document will be used as its title in generated Table of Contents. With the title key you can set an alternative title for a document. and also for url:

root: intro
subtrees:
- entries:
  - file: doc1
    title: Document 1 Title
  - url: https://example.com
    title: Example URL Title

External URLs

intersphinx_mapping contains the base url(s). This is found in docs/conf.py.

sphinx.ext.intersphinx inventories contain the std:label entries; the rest of the url.

Placing urls in the _toc.yml is still supported. For those who avoided the learning curve and are not looking to use intersphinx, url: is not going away.

ref: is now preferred over url:. intersphinx is made for managing all the urls in our documentation. Use it!

This is how external urls are stored. For internal docs, use file:.

The title: is optional. If not provided, the title is taken from the inventory entry. In the example, the title would become, The Julia Domain.

Sphinx inventory v2

Sphinx inventory version 2
Project: foo
Version: 2.0
The remainder of this file is compressed with zlib.
The-Julia-Domain std:label -1 write_inventory/#$ The Julia Domain

^^ write this into docs/objects-test.txt

cd docs
sphobjinv co -q zlib objects-test.txt objects.test.inv

_toc.yml

root: intro
subtrees:
- entries:
  - file: doc1
    title: Document 1 Title
  - ref: The-Julia-Domain
    title: btw who is Julia?

Create files: docs/doc1.rst and docs/intro.rst. Empty files ... ok.

conf.py

extensions = [
    "sphinx_external_toc_strict",
    "sphinx.ext.intersphinx",
    "myst-parser",
]
master_doc = intro
source_suffix = [".md", ".rst"]
intersphinx_mapping = {
    "python": (
         "https://docs.python.org/3",
         ("objects-test.inv", "objects-test.txt"),
     ),
 }
 myst_enable_extensions = ["colon_fence", "html_image"]
 external_toc_exclude_missing = true

Makefile not shown. Make that too.

cd docs
touch doc1.rst
touch intro.rst
make html

KNOWN LIMITATIONS

1. Not being able to open an external URL in a new window or tab is a Sphinx limitation. In the TOC, an external URL not opening in a new window or tab is very confusing UX.

2. When there is no inventory entry for a ref:, there is no warning, the link will just not be displayed.

The workflow should be:

1. inventory entry
2. ref: into the _toc.yml

intersphinx-data

ToC tree options

Each subtree can be configured with a number of options (see also sphinx toctree options):

• caption (string): A title for the whole the subtree, e.g. shown above the subtree in ToCs
• hidden (boolean): Whether to show the ToC within (inline of) the document (default False). By default it is appended to the end of the document, but see also the tableofcontents directive for positioning of the ToC.
• maxdepth (integer): A maximum nesting depth to use when showing the ToC within the document (default -1, meaning infinite).
• numbered (boolean or integer): Automatically add numbers to all documents within a subtree (default False). If set to True, all sub-trees will also be numbered based on nesting (e.g. with 1.1 or 1.1.1), or if set to an integer then the numbering will only be applied to that depth.
• reversed (boolean): If True then the entries in the subtree will be listed in reverse order (default False). This can be useful when using glob entries.
• titlesonly (boolean): If True then only the first heading in the document will be shown in the ToC, not other headings of the same level (default False).

These options can be set at the level of the subtree:

root: intro
subtrees:
- caption: Subtree Caption
  hidden: False
  maxdepth: 1
  numbered: True
  reversed: False
  titlesonly: True
  entries:
  - file: doc1
    subtrees:
    - titlesonly: True
      entries:
      - file: doc2

or, if you are using the shorthand for a single subtree, set options under an options key:

root: intro
options:
  caption: Subtree Caption
  hidden: False
  maxdepth: 1
  numbered: True
  reversed: False
  titlesonly: True
entries:
- file: doc1
  options:
    titlesonly: True
  entries:
  - file: doc2

You can also use the top-level defaults key, to set default options for all subtrees:

root: intro
defaults:
  titlesonly: True
options:
  caption: Subtree Caption
  hidden: False
  maxdepth: 1
  numbered: True
  reversed: False
entries:
- file: doc1
  entries:
  - file: doc2

Warning

numbered

numbered should not generally be used as a default, since numbering cannot be changed by nested subtrees, and sphinx will log a warning.

Note

title numbering

By default, title numbering restarts for each subtree. If you want want this numbering to be continuous, check-out the sphinx-multitoc-numbering extension.

Using different key-mappings

For certain use-cases, it is helpful to map the subtrees/entries keys to mirror e.g. an output LaTeX structure.

The format key can be used to provide such mappings (and also initial defaults). Currently available:

• jb-article: - Maps entries -> sections - Sets the default of titlesonly to true
• jb-book: - Maps the top-level subtrees to parts - Maps the top-level entries to chapters - Maps other levels of entries to sections - Sets the default of titlesonly to true

For example:

defaults:
  titlesonly: true
root: index
subtrees:
- entries:
  - file: doc1
    entries:
    - file: doc2

is equivalent to:

format: jb-book
root: index
parts:
- chapters:
  - file: doc1
    sections:
    - file: doc2

Important

key names changes

These change in key names do not change the output site-map structure

Add a ToC to a page's content

By default, the toctree generated per document (one per subtree) are appended to the end of the document and hidden (then, for example, most HTML themes show them in a side-bar).

But if you would like them to be visible at a certain place within the document body, you may do so by using the tableofcontents directive:

ReStructuredText:

.. tableofcontents::

MyST Markdown:

```{tableofcontents}
```

Currently, only one tableofcontents should be used per page (all toctree will be added here), and only if it is a page with child/descendant documents.

Note, this will override the hidden option set for a subtree.

Excluding files not in ToC

By default, Sphinx will build all document files, regardless of whether they are specified in the Table of Contents, if they:

1. Have a file extension relating to a loaded parser (e.g. .rst or .md)
2. Do not match a pattern in exclude_patterns

To automatically add any document files that do not match a file or glob in the ToC to the exclude_patterns list, add to your conf.py:

external_toc_exclude_missing = True

Note that, for performance, files that are in hidden folders (e.g. in .tox or .venv) will not be added to exclude_patterns even if they are not specified in the ToC. You should exclude these folders explicitly.

Important

incompatible with orphan files

This feature is currently incompatible with orphan files.

Command-line

This package comes with the sphinx-etoc-strict command-line program, with some additional tools.

To see all options:

Usage: sphinx-etoc-strict [OPTIONS] COMMAND [ARGS]...

  Command-line for sphinx-external-toc-strict.

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  from-project  Create a ToC file from a project directory.
  migrate    Migrate a ToC from a previous revision.
  parse      Parse a ToC file to a site-map YAML.
  to-project    Create a project directory from a ToC file.

To build a template project from only a ToC file:

Note, you can also add additional files in meta/create_files and append text to the end of files with meta/create_append, e.g.

root: intro
entries:
- glob: doc*
meta:
  create_append:
    intro: |
      This is some
      appended text
  create_files:
  - doc1
  - doc2
  - doc3

To build a ToC file from an existing site:

Some rules used:

• Files/folders will be skipped if they match a pattern added by -s (based on [fnmatch docs] Unix shell-style wildcards)
• Sub-folders with no content files inside will be skipped
• File and folder names will be sorted by natural order
• If there is a file called index (or the name set by -i) in any folder, it will be treated as the index file, otherwise the first file by ordering will be used.

The command can also guess a title for each file, based on its path:

• The folder name is used for index files, otherwise the file name
• Words are split by _
• The first "word" is removed if it is an integer

For example, for a project with files:

index.rst
1_a_title.rst
11_another_title.rst
.hidden_file.rst
.hidden_folder/index.rst
1_a_subfolder/index.rst
2_another_subfolder/index.rst
2_another_subfolder/other.rst
3_subfolder/1_no_index.rst
3_subfolder/2_no_index.rst
14_subfolder/index.rst
14_subfolder/subsubfolder/index.rst
14_subfolder/subsubfolder/other.rst

will create the ToC:

root: index
entries:
- file: 1_a_title
  title: A title
- file: 11_another_title
  title: Another title
- file: 1_a_subfolder/index
  title: A subfolder
- file: 2_another_subfolder/index
  title: Another subfolder
  entries:
  - file: 2_another_subfolder/other
    title: Other
- file: 3_subfolder/1_no_index
  title: No index
  entries:
  - file: 3_subfolder/2_no_index
    title: No index
- file: 14_subfolder/index
  title: Subfolder
  entries:
  - file: 14_subfolder/subsubfolder/index
    title: Subsubfolder
    entries:
    - file: 14_subfolder/subsubfolder/other
      title: Other

Note

hidden files are unsupported

On a filesystem, somewhere within your home directory, hidden files are meant for config files. Documents are not hidden files!

The file stem and file suffix handling has improved dramatically.

But a hidden file, like .hidden_file.rst, and .tar.gz looks similar. Both have no file stem

Either can have markdown support or hidden file support, not both. Fate chose markdown support; that's the way the dice rolled

API

The ToC file is parsed to a SiteMap, which is a MutableMapping subclass, with keys representing docnames mapping to a Document that stores information on the toctrees it should contain:

from sphinx_external_toc.parsing_strict import parse_toc_yaml, dump_yaml

path = "path/to/_toc.yml"
site_map = parse_toc_yaml(path)
dump_yaml(site_map)

Would produce e.g.

root: intro
documents:
  doc1:
    docname: doc1
    subtrees: []
    title: null
  intro:
    docname: intro
    subtrees:
    - caption: Subtree Caption
      numbered: true
      reversed: false
      items:
      - doc1
      titlesonly: true
    title: null
meta: {}

Development Notes

Questions / TODOs:

• Add additional top-level keys, e.g. appendices (see sphinx#2502) and bibliography
• Integrate sphinx-multitoc-numbering into this extension? (or upstream PR)
• document suppressing warnings
• test against orphan file
• sphinx-book-theme#304
• CLI command to generate toc from existing documentation toctrees (and then remove toctree directives)
• test rebuild on toc changes (and document how rebuilds are controlled when toc changes)
• some jupyter-book issues point to potential changes in numbering, based on where the toctree is in the document. So could look into placing it e.g. under the first heading/title