Skip to content

Commit

Permalink
add docgen tools binary
Browse files Browse the repository at this point in the history
This commit adds a `docgen` binary to the tools crate. This tool can
generate a .json file describing the API based on parsing the `cbindgen`
generated `rustls.h` header file using a tree-sitter C grammar.

The produced JSON can in turn be used to generate markdown web
documentation, or any other format required.

Along with generating docs this tool applies some basic policy. In
particular it requires that **all** public API items be documented or it
will produce an error instead of the JSON data. We can extend this in
the future to require (for example) describing arguments and return
values in doc comments.

The generated JSON has the following structure:

```json
{
  "structs": [ .. ],
  "functions": : [ .. ],
  "callbacks": [ .. ],
  "enums": [ .. ],
  "externs": [ .. ],
  "aliases": [ .. ]
}
```

Where each key is a general category of item:

* Structure definitions
* API function prototypes
* Callback typedefs
* Enum typedefs
* Extern typedefs
* Simple type alias typdefs

Within each category items are described like:

```json
{
  "anchor": "rustls-accepted",
  "comment": "A parsed ClientHello produced by a ...",
  "name": "rustls_accepted",
  "text": "```c\nstruct rustls_accepted\n```"
},
```

The anchor field is useful for creating anchor links that identify the
item. The name is the actual name of the item. The comment is the
C block comment content (with the block comment syntax removed, but
other whitespace left as-is). Lastly the text field holds the actual
text that defines the item in rustls.h (without comment) as
a C formatted markdown code block.
  • Loading branch information
cpu committed Dec 19, 2024
1 parent dd6ed2f commit 5b42319
Show file tree
Hide file tree
Showing 5 changed files with 635 additions and 4 deletions.
50 changes: 50 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,7 @@ regex = "1.9.6"
toml = { version = "0.6.0", default-features = false, features = ["parse"] }
hickory-resolver = { version = "=0.25.0-alpha.4", features = ["dns-over-https-rustls", "webpki-roots"] }
tokio = { version = "1.42.0", features = ["io-util", "macros", "net", "rt"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tree-sitter = "0.23" # TODO(@cpu): handle breaking API changes for 0.24
tree-sitter-c = "0.23"
8 changes: 4 additions & 4 deletions tools/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ publish = false
rustls = { workspace = true }
hickory-resolver = { workspace = true }
tokio = { workspace = true }

[[bin]]
name = "ech_fetch"
path = "src/ech_fetch.rs"
serde = { workspace = true }
serde_json = { workspace = true }
tree-sitter = { workspace = true }
tree-sitter-c = { workspace = true }
Loading

0 comments on commit 5b42319

Please sign in to comment.