From ebd0824e1bfca8ae89a05a8168c3aa289502b06a Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Thu, 4 Jul 2024 20:40:17 +0800 Subject: [PATCH 01/15] feat: architecture sections --- docs/book.toml | 4 ++-- docs/src/SUMMARY.md | 4 ++++ docs/src/architecture.md | 28 ++++++++++++++++++++++++++++ 3 files changed, 34 insertions(+), 2 deletions(-) create mode 100644 docs/src/architecture.md diff --git a/docs/book.toml b/docs/book.toml index 0f9409f1..4c99a871 100644 --- a/docs/book.toml +++ b/docs/book.toml @@ -7,8 +7,8 @@ description = "A next-generation terminal multiplexer that records everything yo authors = ["Caleb Foust"] [output.html] -default-theme = "navy" -preferred-dark-theme = "navy" +default-theme = "ayu" +preferred-dark-theme = "ayu" site-url = "/cy/" git-repository-url = "https://github.com/cfoust/cy" additional-css = ["./theme/asciinema-player.css"] diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 1beefee4..7459acd6 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -36,6 +36,10 @@ - [API](./api.md) +# Developer guide + +- [Architecture](./architecture.md) + --- [Acknowledgments](./acknowledgments.md) diff --git a/docs/src/architecture.md b/docs/src/architecture.md new file mode 100644 index 00000000..ed10c82d --- /dev/null +++ b/docs/src/architecture.md @@ -0,0 +1,28 @@ +# Architecture + +This document is intended to be a brief introduction to `cy`'s code structure and its commonly used abstractions. It is useful for anyone interested in contributing to `cy` or any of its constituent libraries, some of which may (eventually) be broken out into separate projects. + +It is safe to assume that the high-level description in this document will remain reliable despite changes in the actual implementation, but if you are ever in doubt: + +1. Read the README for the package you are modifying (typically in `pkg/*`.) +2. Ask for help [in Discord](https://discord.gg/NRQG3wbWGM). +3. Consult the code itself. + +`cy` is written in Go and Janet. I chose Go because I had written other projects with strong concurrency needs and it seemed like a natural fit. Janet is a Lisp-like scripting language that I chose because it sounded like fun. + +## Introduction + +`cy` is a terminal multiplexer, so (as the name implies) most of the complexity comes from doing two things: + +1. **Emulating a terminal**: Just like in `tmux` et al, `cy` works by pretending to be a valid VT100 terminal and attaching to the programs that you run (typically shells). +2. **Multiplexing**: Users expect to be able to switch between the terminals `cy` emulates in order to fulfill the basic requirement of being a terminal multiplexer. + +Terminal emulation, though tedious and error-prone to write yourself, is critical for any terminal multiplexer. Because of the paucity of Go libraries to accomplish this, I had to implement this mostly from scratch in [the emu package](https://github.com/cfoust/cy/tree/main/pkg/emu). + +Multiplexing, of course, is where things get interesting. `cy`'s codebase has a range of different tools for compositing and rendering terminal windows, all of which it does to be able to support an arbitrary number of clients, all of whom may have different screen sizes and need to use `cy` for different things. + +Speaking of clients, just like `tmux`, `cy` uses a server-client model and daemonizes itself on server startup. In simple terms this means that irrespective of where, when, or how you start `cy`, if a `cy` server is running you can connect to it and resume your work exactly as you left it. This is advantageous for those of us who do most of our work on remote machines via `ssh` and is one of the traditional use cases for `tmux`. + +## Screens and streams + +`cy`'s most important abstraction is a [`Screen`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go?plain=1#L42). From 1aee70dda813c1d3c8380e07e05fc2692f6ca8a5 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Thu, 4 Jul 2024 21:05:18 +0800 Subject: [PATCH 02/15] feat: more introduction --- docs/src/architecture.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/src/architecture.md b/docs/src/architecture.md index ed10c82d..f21acfc2 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -12,16 +12,18 @@ It is safe to assume that the high-level description in this document will remai ## Introduction -`cy` is a terminal multiplexer, so (as the name implies) most of the complexity comes from doing two things: +`cy` is a terminal multiplexer. Just like `tmux`, it uses a server-client model and daemonizes itself on server startup. In simple terms this means that irrespective of where, when, or how you start `cy`, if a `cy` server is running you can connect to it and resume your work exactly as you left it. Clients connect to the `cy` server using a WebSocket connection via [Unix domain sockets](https://en.wikipedia.org/wiki/Unix_domain_socket). + +As the name "terminal multiplexer" implies, most of the complexity comes from doing two things: 1. **Emulating a terminal**: Just like in `tmux` et al, `cy` works by pretending to be a valid VT100 terminal and attaching to the programs that you run (typically shells). 2. **Multiplexing**: Users expect to be able to switch between the terminals `cy` emulates in order to fulfill the basic requirement of being a terminal multiplexer. -Terminal emulation, though tedious and error-prone to write yourself, is critical for any terminal multiplexer. Because of the paucity of Go libraries to accomplish this, I had to implement this mostly from scratch in [the emu package](https://github.com/cfoust/cy/tree/main/pkg/emu). +Terminal emulation, though tedious and error-prone to write yourself, is critical for any terminal multiplexer. Because of the paucity of Go libraries that accomplish this, I had to implement this mostly from scratch in [the emu package](https://github.com/cfoust/cy/tree/main/pkg/emu). Multiplexing, of course, is where things get interesting. `cy`'s codebase has a range of different tools for compositing and rendering terminal windows, all of which it does to be able to support an arbitrary number of clients, all of whom may have different screen sizes and need to use `cy` for different things. -Speaking of clients, just like `tmux`, `cy` uses a server-client model and daemonizes itself on server startup. In simple terms this means that irrespective of where, when, or how you start `cy`, if a `cy` server is running you can connect to it and resume your work exactly as you left it. This is advantageous for those of us who do most of our work on remote machines via `ssh` and is one of the traditional use cases for `tmux`. +Given that `cy`'s killer feature is being able to replay your terminal sessions, you would think that it would be a source of significant complexity, but it really isn't: once you have the above, making this functionality is just a matter of recording the timestamps and all data associated with every write to a virtual terminal, then replaying it on demand. Of course, the devil is in the details. ## Screens and streams From cae1447abc588a484bb6d52d117b459698112209 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Fri, 5 Jul 2024 02:44:42 +0800 Subject: [PATCH 03/15] feat: more architecture --- docs/src/architecture.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/docs/src/architecture.md b/docs/src/architecture.md index f21acfc2..aae5a019 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -1,10 +1,10 @@ # Architecture -This document is intended to be a brief introduction to `cy`'s code structure and its commonly used abstractions. It is useful for anyone interested in contributing to `cy` or any of its constituent libraries, some of which may (eventually) be broken out into separate projects. +This document is intended to be a brief introduction to `cy`'s code structure and its commonly used abstractions. The intended audience is anyone interested in contributing to `cy` or any of its constituent libraries, some of which may (eventually) be broken out into separate projects. It is safe to assume that the high-level description in this document will remain reliable despite changes in the actual implementation, but if you are ever in doubt: -1. Read the README for the package you are modifying (typically in `pkg/*`.) +1. Read the README for the package you are modifying. Most packages in `pkg` have their own READMEs (along with some sub-packages.) 2. Ask for help [in Discord](https://discord.gg/NRQG3wbWGM). 3. Consult the code itself. @@ -12,19 +12,28 @@ It is safe to assume that the high-level description in this document will remai ## Introduction -`cy` is a terminal multiplexer. Just like `tmux`, it uses a server-client model and daemonizes itself on server startup. In simple terms this means that irrespective of where, when, or how you start `cy`, if a `cy` server is running you can connect to it and resume your work exactly as you left it. Clients connect to the `cy` server using a WebSocket connection via [Unix domain sockets](https://en.wikipedia.org/wiki/Unix_domain_socket). +`cy` is a [**terminal multiplexer**](https://en.wikipedia.org/wiki/Terminal_multiplexer). Just like `tmux`, it uses a server-client model and daemonizes itself on server startup. In simple terms this means that irrespective of where, when, or how you start `cy`, if a `cy` server is running you can connect to it and resume your work exactly as you left it. Clients connect to the `cy` server using a WebSocket connection via a [Unix domain socket](https://en.wikipedia.org/wiki/Unix_domain_socket). As the name "terminal multiplexer" implies, most of the complexity comes from doing two things: 1. **Emulating a terminal**: Just like in `tmux` et al, `cy` works by pretending to be a valid VT100 terminal and attaching to the programs that you run (typically shells). 2. **Multiplexing**: Users expect to be able to switch between the terminals `cy` emulates in order to fulfill the basic requirement of being a terminal multiplexer. -Terminal emulation, though tedious and error-prone to write yourself, is critical for any terminal multiplexer. Because of the paucity of Go libraries that accomplish this, I had to implement this mostly from scratch in [the emu package](https://github.com/cfoust/cy/tree/main/pkg/emu). +Terminal emulation, though tedious and error-prone to write yourself, is critical for any terminal multiplexer. Because of the paucity of Go libraries that accomplish this, this was implemented mostly from scratch in [the emu package](https://github.com/cfoust/cy/tree/main/pkg/emu). Multiplexing, of course, is where things get interesting. `cy`'s codebase has a range of different tools for compositing and rendering terminal windows, all of which it does to be able to support an arbitrary number of clients, all of whom may have different screen sizes and need to use `cy` for different things. -Given that `cy`'s killer feature is being able to replay your terminal sessions, you would think that it would be a source of significant complexity, but it really isn't: once you have the above, making this functionality is just a matter of recording the timestamps and all data associated with every write to a virtual terminal, then replaying it on demand. Of course, the devil is in the details. +`cy`'s main feature is being able to replay terminal sessions. You would think that it would be a source of significant complexity. But it really isn't: once you have the above, making this functionality is just a matter of recording every write to a virtual terminal, then replaying it on demand. Of course, the devil is in the details. + +## Codebase organization + +`cy`'s code is divided into three directories found at the repository root: + +* `cmd`: Contains the code for all executables (in this case, programs with `main.go` files.) + * `cy`: The main `cy` executable and the code necessary to connect to and create sockets. + * `stories`: +* `pkg` ## Screens and streams -`cy`'s most important abstraction is a [`Screen`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go?plain=1#L42). +The two most important abstractions in `cy`'s codebase are [`Screens`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go?plain=1#L42) and [`Streams`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go#L36), which are defined in the [mux](https://github.com/cfoust/cy/tree/main/pkg/mux) package. From 04b0e4cc0343d66a8fead072df55c00f397cb906 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Fri, 5 Jul 2024 03:30:06 +0800 Subject: [PATCH 04/15] docs: code organization --- docs/src/SUMMARY.md | 2 ++ docs/src/architecture.md | 19 +++++++++++++++---- docs/src/stories.md | 1 + 3 files changed, 18 insertions(+), 4 deletions(-) create mode 100644 docs/src/stories.md diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 7459acd6..514433cc 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -40,6 +40,8 @@ - [Architecture](./architecture.md) +- [Stories](./stories.md) + --- [Acknowledgments](./acknowledgments.md) diff --git a/docs/src/architecture.md b/docs/src/architecture.md index aae5a019..980d0b48 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -29,10 +29,21 @@ Multiplexing, of course, is where things get interesting. `cy`'s codebase has a `cy`'s code is divided into three directories found at the repository root: -* `cmd`: Contains the code for all executables (in this case, programs with `main.go` files.) - * `cy`: The main `cy` executable and the code necessary to connect to and create sockets. - * `stories`: -* `pkg` +- `cmd`: Contains the code for all executables (in this case, programs with `main.go` files.) + - `cy`: The main `cy` executable and the code necessary to connect to and create sockets. + - `stories`: A system for quickly iterating on `cy`'s visual design. Covered in more detail in [a dedicated chapter](./stories.md). + - `perf`: A (seldom-used) program for testing the performance of `cy`'s history search feature. + - `docs`: A simple executable that dumps various information about `cy` to standard out as JSON, such as all of its API functions, built in key bindings, et cetera. This is used in an [mdbook preprocessor](https://rust-lang.github.io/mdBook/format/configuration/preprocessors.html) called [gendoc](https://github.com/cfoust/cy/blob/main/docs/gendoc.py) that generates Markdown content for `cy` on demand. +- `pkg`: Contains a range of different Go packages, all of which might be charitably called libraries. The list below is not intended to be exhaustive, but just highlight several important ones. + - `cy`: The `cy` server, API, default configuration, et cetera. + - `geom`: Simple, high-level geometric primitives (think `Vec2`) used everywhere in the codebase. + - `mux`: A few useful abstractions for multiplexing. + - `janet`: A library for Janet/Go interoperation. + - `emu`: A vt100 terminal emulator. + - `fuzzy`: A [fuzzy finder](./fuzzy-finding.md). + - `replay`: A terminal session player, otherwise known as [replay mode](./replay-mode.md). + - `taro`: A fork of [charmbracelet/bubbletea](https://github.com/charmbracelet/bubbletea) adapted for use in `cy`'s windowing abstraction (described [below](./architecture.md#screens-and-streams).) +- `docs`: Contains all of `cy`'s documentation. `cy` uses [mdbook](https://github.com/rust-lang/mdBook) to build the documentation site. ## Screens and streams diff --git a/docs/src/stories.md b/docs/src/stories.md new file mode 100644 index 00000000..70ac3484 --- /dev/null +++ b/docs/src/stories.md @@ -0,0 +1 @@ +# Stories From 92ae27f8900de93f3346829d96188bb6fac4213d Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Fri, 5 Jul 2024 03:59:12 +0800 Subject: [PATCH 05/15] docs: more --- docs/src/architecture.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/src/architecture.md b/docs/src/architecture.md index 980d0b48..0f350e70 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -48,3 +48,15 @@ Multiplexing, of course, is where things get interesting. `cy`'s codebase has a ## Screens and streams The two most important abstractions in `cy`'s codebase are [`Screens`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go?plain=1#L42) and [`Streams`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go#L36), which are defined in the [mux](https://github.com/cfoust/cy/tree/main/pkg/mux) package. + +A `Stream` is just a resizable (this is important!) stream of bytes that can be read from and written to. As of writing, it looks like this: + +```go +// Note: this was changed from the actual implementation for simplicity. +type Stream interface { + io.ReadWriter + Resize(size Vec2) error +} +``` + +From the perspective of the process you're running, this interface concisely describes the functionality of your terminal emulator (e.g. xterm, kitty.) Typing into your terminal writes to the process; any output it produces is read and interpreted in a predictable, standard way (the VT100 quasi-standard.) Resizing your terminal sends a resize event, `SIGWINCH`, which the process can react to. From 67d6c70989b46ddccd119273324052eccea4b597 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Fri, 5 Jul 2024 06:36:59 +0800 Subject: [PATCH 06/15] feat: streams and screens --- docs/src/architecture.md | 64 ++++++++++++++++++++++++++++++++++++++-- docs/src/replay-mode.md | 2 +- 2 files changed, 62 insertions(+), 4 deletions(-) diff --git a/docs/src/architecture.md b/docs/src/architecture.md index 0f350e70..444bb919 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -8,7 +8,9 @@ It is safe to assume that the high-level description in this document will remai 2. Ask for help [in Discord](https://discord.gg/NRQG3wbWGM). 3. Consult the code itself. -`cy` is written in Go and Janet. I chose Go because I had written other projects with strong concurrency needs and it seemed like a natural fit. Janet is a Lisp-like scripting language that I chose because it sounded like fun. +`cy` is written in Go and Janet. I chose Go because I had written other projects with significant concurrency needs and it seemed like a natural fit. Janet is a Lisp-like scripting language that I chose because it sounded like fun. + +This document assumes basic familiarity with Go. ## Introduction @@ -47,16 +49,72 @@ Multiplexing, of course, is where things get interesting. `cy`'s codebase has a ## Screens and streams -The two most important abstractions in `cy`'s codebase are [`Screens`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go?plain=1#L42) and [`Streams`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go#L36), which are defined in the [mux](https://github.com/cfoust/cy/tree/main/pkg/mux) package. +The two most important abstractions in `cy`'s codebase are [`Screen`s](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go?plain=1#L42) and [`Stream`s](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go#L36), which are defined in the [mux](https://github.com/cfoust/cy/tree/main/pkg/mux) package. + +### Stream A `Stream` is just a resizable (this is important!) stream of bytes that can be read from and written to. As of writing, it looks like this: ```go // Note: this was changed from the actual implementation for simplicity. type Stream interface { - io.ReadWriter + Read(p []byte) (n int, err error) + Write(p []byte) (n int, err error) Resize(size Vec2) error } ``` From the perspective of the process you're running, this interface concisely describes the functionality of your terminal emulator (e.g. xterm, kitty.) Typing into your terminal writes to the process; any output it produces is read and interpreted in a predictable, standard way (the VT100 quasi-standard.) Resizing your terminal sends a resize event, `SIGWINCH`, which the process can react to. + +This is useful because you can represent lots of things as a `Stream`: + +1. [Pseudo-terminals](https://en.wikipedia.org/wiki/Pseudoterminal): By connecting a process to a pseudo-terminal, it behaves as though a user had run it interactively. + - `Write`: Writes are written directly to that process' standard input. + - `Read`: Reads correspond to whatever that process writes to standard output. + - `Resize`: Will set the size of the pseudo-terminal (and thus send `SIGWINCH` to the process). +2. Clients: `cy` clients that connect to the server can be written to and read from. + - `Write`: Writes are interpreted as user input, typically sequences of keys. + - `Read`: Reads consist of the shortest sequence of bytes necessary to update the client's terminal to match `cy`'s understanding of that client's screen. + - `Resize`: Resizing a client indicates to `cy` that it should resize everything on that client's screen and redraw accordingly. + +Streams compose naturally and can form pipes of arbitrary complexity. For example, `cy` records terminal sessions by proxying a `Stream` (sort of like `tee`.) + +However, for a terminal multiplexer this is clearly not enough. A `Stream` is stateless. In other words, there is no way to know what the state of the terminal that is attached to that `Stream`. That's where `Screen`s come in. + +### Screen + +A [`Screen`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go?plain=1#L42) can be thought of, conceptually, as an application to which you can send events (such as user input) and receive any updates it produces (such as changes to the screen's contents). + +The state of a screen (otherwise known as [`tty.State`](https://github.com/cfoust/cy/blob/main/pkg/geom/tty/module.go?plain=1#L9)) is identical to that of a terminal emulator: + +- A fixed, two-dimensional buffer of glyphs +- The state of the cursor including its position and style + +A [**pane**](./groups-and-panes.md#panes), described elsewhere, is a good example of a `Screen`. + +If that all sounds abstract, the interface for `Screen` looks like this: + +```go +type Screen interface { + // State gets the current visual state of the Screen. + State() *tty.State + + // Resize adjusts the screen to fit `size`. + Resize(size Vec2) error + + // Subscribe subscribes to any updates to the screen, which are usually + // caused by changes to the screen's state. + Subscribe(context.Context) *Updater + + // Send sends a message to the Screen. + Send(message interface{}) +} +``` + +`Send` looks scary, but it's used in `cy` mostly for key and mouse events on user input. + +The easiest way to understand this is to think of a `Screen` as something that can render a `Stream` and turn it into something that can be composed with other `Screen`s. In fact, there is a `Screen` that [does just that](https://github.com/cfoust/cy/blob/main/pkg/mux/screen/terminal.go?plain=1#L13). + +`cy`'s [fuzzy finder](./fuzzy-finding.md) and [replay mode](./replay-mode.md) are both just `Screen`s, albeit complicated ones. + +Some `Screen`s just exist to compose other screens in some way, which is the bread and butter of any terminal multiplexer. The simplest example of this is `cy`'s [`Layers`](https://github.com/cfoust/cy/blob/main/pkg/mux/screen/layers.go?plain=1#L22), a `Screen` that lets you render one or more `Screen`s on top of one another, letting the screens underneath show through if any cells of the layer above are transparent. diff --git a/docs/src/replay-mode.md b/docs/src/replay-mode.md index cbf3051c..4a810447 100644 --- a/docs/src/replay-mode.md +++ b/docs/src/replay-mode.md @@ -60,6 +60,6 @@ By default, `cy` records all of the activity that occurs in a terminal session t The directory will be created if it does not exist. -You can access previous sessions through the `cy/open-log` action, which by default can be invoked by searching for `open an existing log file` in the command palette (`ctrl+a` `ctrl+p`). +You can access previous sessions through the {{api action/open-log}} action, which by default can be invoked by searching for `Open a .borg file.` in the command palette (`ctrl+a` `ctrl+p`). You are also free to use the API call {{api replay/open}} to open `.borg` files anywhere on your filesystem. From ada9b14552e1e47e18561abaf94ce0231c9f8986 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Fri, 5 Jul 2024 17:41:46 +0800 Subject: [PATCH 07/15] feat: summary section --- docs/src/architecture.md | 33 ++++++++++++++++++++++++++------- docs/src/default-keys.md | 4 ++-- 2 files changed, 28 insertions(+), 9 deletions(-) diff --git a/docs/src/architecture.md b/docs/src/architecture.md index 444bb919..4401c251 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -53,10 +53,9 @@ The two most important abstractions in `cy`'s codebase are [`Screen`s](https://g ### Stream -A `Stream` is just a resizable (this is important!) stream of bytes that can be read from and written to. As of writing, it looks like this: +A `Stream` is just a resizable (this is important!) bidirectional stream of bytes that can be read from and written to. As of writing, it looks like this: ```go -// Note: this was changed from the actual implementation for simplicity. type Stream interface { Read(p []byte) (n int, err error) Write(p []byte) (n int, err error) @@ -77,7 +76,7 @@ This is useful because you can represent lots of things as a `Stream`: - `Read`: Reads consist of the shortest sequence of bytes necessary to update the client's terminal to match `cy`'s understanding of that client's screen. - `Resize`: Resizing a client indicates to `cy` that it should resize everything on that client's screen and redraw accordingly. -Streams compose naturally and can form pipes of arbitrary complexity. For example, `cy` records terminal sessions by proxying a `Stream` (sort of like `tee`.) +Streams can be composed and form pipes of arbitrary complexity. For example, `cy` records terminal sessions by proxying a `Stream` (sort of like `tee`.) However, for a terminal multiplexer this is clearly not enough. A `Stream` is stateless. In other words, there is no way to know what the state of the terminal that is attached to that `Stream`. That's where `Screen`s come in. @@ -85,9 +84,9 @@ However, for a terminal multiplexer this is clearly not enough. A `Stream` is st A [`Screen`](https://github.com/cfoust/cy/blob/main/pkg/mux/module.go?plain=1#L42) can be thought of, conceptually, as an application to which you can send events (such as user input) and receive any updates it produces (such as changes to the screen's contents). -The state of a screen (otherwise known as [`tty.State`](https://github.com/cfoust/cy/blob/main/pkg/geom/tty/module.go?plain=1#L9)) is identical to that of a terminal emulator: +The state of a screen (represented in the `cy` codebase as a [`tty.State`](https://github.com/cfoust/cy/blob/main/pkg/geom/tty/module.go?plain=1#L9)) is identical to that of a terminal emulator: -- A fixed, two-dimensional buffer of glyphs +- A two-dimensional buffer of Unicode characters - The state of the cursor including its position and style A [**pane**](./groups-and-panes.md#panes), described elsewhere, is a good example of a `Screen`. @@ -111,10 +110,30 @@ type Screen interface { } ``` -`Send` looks scary, but it's used in `cy` mostly for key and mouse events on user input. +`Send` looks scary, but it's used in `cy` mostly for key and mouse events. The easiest way to understand this is to think of a `Screen` as something that can render a `Stream` and turn it into something that can be composed with other `Screen`s. In fact, there is a `Screen` that [does just that](https://github.com/cfoust/cy/blob/main/pkg/mux/screen/terminal.go?plain=1#L13). `cy`'s [fuzzy finder](./fuzzy-finding.md) and [replay mode](./replay-mode.md) are both just `Screen`s, albeit complicated ones. -Some `Screen`s just exist to compose other screens in some way, which is the bread and butter of any terminal multiplexer. The simplest example of this is `cy`'s [`Layers`](https://github.com/cfoust/cy/blob/main/pkg/mux/screen/layers.go?plain=1#L22), a `Screen` that lets you render one or more `Screen`s on top of one another, letting the screens underneath show through if any cells of the layer above are transparent. +Some `Screen`s just exist to compose other screens in some way, which is the bread and butter of any terminal multiplexer. + +The simplest example of this is `cy`'s [`Layers`](https://github.com/cfoust/cy/blob/main/pkg/mux/screen/layers.go?plain=1#L22), a `Screen` that lets you render one or more `Screen`s on top of one another, letting the screens underneath show through if any cells of the layer above are transparent. + +`Layers` is used to place the pane the user is currently interacting with on top of a [frame](./frames.md), such as in the default viewport: + +{{story png placeholder}} + +It is also used for `cy`'s toast messages ({{api cy/toast}}), which are implemented using a noninteractive `Screen` that is layered over the rest of the content on the client's screen. + +### Tying it all together + +To illustrate the difference between `Screen`s and `Streams`, consider the following description of how data flows back and forth from a client to its `Screen`s and back again. + +The flow for client input works like this: + +1. The client presses a key in the terminal where they originally connected to `cy`. The terminal emulator writes the byte sequence for that key to the standard input of the process controlling the terminal, which in this case is `cy` running as a client. + - When `cy` is running in client mode, it represents its connection to the server with a `Stream` ([`ClientIO`](https://github.com/cfoust/cy/blob/main/cmd/cy/client.go?plain=1#L49)), the `Read`, `Write`, and `Resize` methods of which are [connected](https://github.com/cfoust/cy/blob/main/pkg/mux/stream/cli/module.go?plain=1#L20) directly to the standard output, standard input, and `SIGWINCH` events of the controlling terminal. +2. All of the events are sent using the WebSocket protocol via a Unix socket to the `cy` server, which is a separate process. +3. The `cy` server writes the incoming bytes it received from the client to the corresponding [`Client`](https://github.com/cfoust/cy/blob/main/pkg/cy/client.go?plain=1#L34) on the server. A `Client` is just a `Stream`. +4. The `Client` translates the bytes into key and mouse events that are then sent (via `Send`) to the `Screen` the `Client` is attached to. These events usually travel through several different `Screen`s before reaching their destination, but ultimately they are passed into whatever `Screen` the client is currently attached to--whether that be a pane, the fuzzy finder, or replay mode. diff --git a/docs/src/default-keys.md b/docs/src/default-keys.md index de23dd0d..8e651e31 100644 --- a/docs/src/default-keys.md +++ b/docs/src/default-keys.md @@ -1,9 +1,9 @@ # Default key bindings -All of `cy`'s default key bindings use [actions](./keybindings.md#actions) defined in the global scope and therefore are easy to rebind should you so desire. For example, to assign `cy/command-palette` to another key sequence: +All of `cy`'s default key bindings use [actions](./keybindings.md#actions) defined in the global scope and therefore are easy to rebind should you so desire. For example, to assign {{api action/command-palette}} to another key sequence: ```janet -(key/bind :root ["ctrl+b" "p"] cy/command-palette) +(key/bind :root ["ctrl+b" "p"] action/command-palette) ``` ## Global From 046d746cf8e532db778e7f2b48c21c17ecf572e3 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Fri, 5 Jul 2024 17:54:50 +0800 Subject: [PATCH 08/15] feat: finish summary section --- docs/src/architecture.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/src/architecture.md b/docs/src/architecture.md index 4401c251..e399e370 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -137,3 +137,10 @@ The flow for client input works like this: 2. All of the events are sent using the WebSocket protocol via a Unix socket to the `cy` server, which is a separate process. 3. The `cy` server writes the incoming bytes it received from the client to the corresponding [`Client`](https://github.com/cfoust/cy/blob/main/pkg/cy/client.go?plain=1#L34) on the server. A `Client` is just a `Stream`. 4. The `Client` translates the bytes into key and mouse events that are then sent (via `Send`) to the `Screen` the `Client` is attached to. These events usually travel through several different `Screen`s before reaching their destination, but ultimately they are passed into whatever `Screen` the client is currently attached to--whether that be a pane, the fuzzy finder, or replay mode. + +The flow for client output is somewhat simpler: + +1. Whenever the `Screen` the `Client` is attached to changes in some way (in other words, it produces an event that is published to its subscribers via `Subscribe`). +2. The client's [`Renderer`](https://github.com/cfoust/cy/blob/main/pkg/mux/stream/renderer/module.go?plain=1#L24) receives this event and calls `State()` on the client's `Screen`, which produces a `tty.State`. The `Renderer` then calculates the sequence of bytes necessary to transform the actual client's terminal screen to match the `cy` server's state. +3. This byte string is sent via the aforementioned WebSocket connection. +4. It is ultimately `Read` by the user's terminal and written to standard output, thus triggering the visual changes the user expects. From ca326b1f54c62ac508076c1b989c0e85b6d5e92c Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Sat, 6 Jul 2024 11:29:13 +0800 Subject: [PATCH 09/15] feat: refactor gendoc to be more sensible --- docs/gendoc.py | 194 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 127 insertions(+), 67 deletions(-) diff --git a/docs/gendoc.py b/docs/gendoc.py index 19dde06a..262856db 100644 --- a/docs/gendoc.py +++ b/docs/gendoc.py @@ -9,7 +9,16 @@ import argparse import sys from pathlib import Path -from typing import NamedTuple, Optional, Tuple, List, Any, Set +from typing import ( + NamedTuple, + Optional, + Tuple, + Dict, + List, + Any, + Set, + Callable, +) GENDOC_REGEX = re.compile("{{gendoc (.+)}}") KEYS_REGEX = re.compile(r"{{keys (.+)}}") @@ -157,6 +166,111 @@ def render_keys(bindings: List[Binding], args: List[str]) -> str: return output +Transformer = Callable[[str],str] +Replacement = Tuple[int, int, str] + + +def handle_pattern( + pattern: re.Pattern, + handler: Callable[[re.Match], Optional[Replacement]], +) -> Transformer: + """ + Given a regex pattern `pattern` and a function `handler` that turns matches + into in-text replacements, return a Transformer. + """ + + def transform(content: str) -> str: + replace: List[Replacement] = [] + + for match in pattern.finditer(content): + replacement = handler(match) + if not replacement: continue + replace.append(replacement) + + replace = sorted(replace, key=lambda a: a[1]) + + for start, end, text in reversed(replace): + content = content[:start] + text + content[end:] + + return content + + return transform + + +def transform_gendoc( + frames: List[str], + animations: List[str], + symbols: List[Symbol], +) -> Transformer: + def handler(match: re.Match) -> Optional[Replacement]: + command = match.group(1) + if len(command) == 0: + return None + + output = "" + if command == "frames": + output = render_frames(frames) + elif command == "animations": + output = render_animations(animations) + elif command == "api": + output = render_api(symbols) + + return ( + match.start(0), + match.end(0), + output, + ) + + return handle_pattern(GENDOC_REGEX, handler) + + +def transform_keys( + bindings: List[Binding], +) -> Transformer: + def handler(match: re.Match) -> Optional[Replacement]: + args = match.group(1) + if len(args) == 0: + return None + + return ( + match.start(0), + match.end(0), + render_keys( + bindings, + args.split(" "), + ), + ) + + return handle_pattern(KEYS_REGEX, handler) + + +def transform_api( + symbol_lookup: Dict[str, Symbol], +) -> Transformer: + def handler(match: re.Match) -> Optional[Replacement]: + name = match.group(1) + if len(name) == 0: + return None + + if not name in symbol_lookup: + # report_error( + # chapter, + # match.start(0), + # match.end(0), + # f"missing symbol: {name}", + # ) + return None + + symbol = symbol_lookup[name] + + return ( + match.start(0), + match.end(0), + render_symbol_link(symbol), + ) + + return handle_pattern(API_REGEX, handler) + if __name__ == '__main__': args = sys.argv @@ -187,6 +301,16 @@ def render_keys(bindings: List[Binding], args: List[str]) -> str: binding['Function'] = symbol_lookup[func] bindings.append(Binding(**binding)) + transformers: List[Transformer] = [ + transform_gendoc( + api['Frames'], + api['Animations'], + symbols, + ), + transform_keys(bindings), + transform_api(symbol_lookup), + ] + errors: int = 0 def report_error(chapter, start, end, msg): global errors @@ -194,74 +318,10 @@ def report_error(chapter, start, end, msg): print(f"{chapter['name']}:{start}{end}: {msg}", file=sys.stderr) def transform_chapter(chapter) -> None: - replace = [] - content = chapter['content'] - for ref in GENDOC_REGEX.finditer(content): - command = ref.group(1) - if len(command) == 0: - continue - - output = "" - if command == "frames": - output = render_frames(api['Frames']) - elif command == "animations": - output = render_animations(api['Animations']) - elif command == "api": - output = render_api(symbols) - - replace.append( - ( - ref.start(0), - ref.end(0), - output, - ) - ) - - for ref in API_REGEX.finditer(content): - name = ref.group(1) - if len(name) == 0: - continue - - if not name in symbol_lookup: - report_error( - chapter, - ref.start(0), - ref.end(0), - f"missing symbol: {name}", - ) - continue - - symbol = symbol_lookup[name] - - replace.append( - ( - ref.start(0), - ref.end(0), - render_symbol_link(symbol), - ) - ) - - for ref in KEYS_REGEX.finditer(content): - args = ref.group(1) - if len(args) == 0: - continue - - replace.append( - ( - ref.start(0), - ref.end(0), - render_keys( - bindings, - args.split(" "), - ), - ) - ) - - replace = sorted(replace, key=lambda a: a[1]) - for start, end, text in reversed(replace): - content = content[:start] + text + content[end:] + for transform in transformers: + content = transform(content) chapter['content'] = content From e3c91113e4b40e310c3586305e32bbb4080a178f Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Sat, 6 Jul 2024 11:41:22 +0800 Subject: [PATCH 10/15] feat: report errors in gendoc --- docs/gendoc.py | 84 +++++++++++++++++++++++++++++++------------------- 1 file changed, 53 insertions(+), 31 deletions(-) diff --git a/docs/gendoc.py b/docs/gendoc.py index 262856db..5d56fec2 100644 --- a/docs/gendoc.py +++ b/docs/gendoc.py @@ -166,25 +166,33 @@ def render_keys(bindings: List[Binding], args: List[str]) -> str: return output -Transformer = Callable[[str],str] +Error = Tuple[int, str] +Transformer = Callable[[str],Tuple[str, List[Error]]] Replacement = Tuple[int, int, str] def handle_pattern( pattern: re.Pattern, - handler: Callable[[re.Match], Optional[Replacement]], + handler: Callable[ + [re.Match], + Tuple[Optional[Replacement], Optional[Error]], + ], ) -> Transformer: """ Given a regex pattern `pattern` and a function `handler` that turns matches into in-text replacements, return a Transformer. """ - def transform(content: str) -> str: + def transform(content: str) -> Tuple[str, List[Error]]: replace: List[Replacement] = [] + errors: List[Error] = [] for match in pattern.finditer(content): - replacement = handler(match) - if not replacement: continue + replacement, error = handler(match) + if not replacement: + if error: errors.append(error) + continue + replace.append(replacement) replace = sorted(replace, key=lambda a: a[1]) @@ -192,7 +200,7 @@ def transform(content: str) -> str: for start, end, text in reversed(replace): content = content[:start] + text + content[end:] - return content + return content, errors return transform @@ -202,10 +210,13 @@ def transform_gendoc( animations: List[str], symbols: List[Symbol], ) -> Transformer: - def handler(match: re.Match) -> Optional[Replacement]: + def handler(match: re.Match) -> Tuple[ + Optional[Replacement], + Optional[Error], + ]: command = match.group(1) if len(command) == 0: - return None + return None, None output = "" if command == "frames": @@ -219,7 +230,7 @@ def handler(match: re.Match) -> Optional[Replacement]: match.start(0), match.end(0), output, - ) + ), None return handle_pattern(GENDOC_REGEX, handler) @@ -227,10 +238,13 @@ def handler(match: re.Match) -> Optional[Replacement]: def transform_keys( bindings: List[Binding], ) -> Transformer: - def handler(match: re.Match) -> Optional[Replacement]: + def handler(match: re.Match) -> Tuple[ + Optional[Replacement], + Optional[Error], + ]: args = match.group(1) if len(args) == 0: - return None + return None, None return ( match.start(0), @@ -239,7 +253,7 @@ def handler(match: re.Match) -> Optional[Replacement]: bindings, args.split(" "), ), - ) + ), None return handle_pattern(KEYS_REGEX, handler) @@ -247,19 +261,19 @@ def handler(match: re.Match) -> Optional[Replacement]: def transform_api( symbol_lookup: Dict[str, Symbol], ) -> Transformer: - def handler(match: re.Match) -> Optional[Replacement]: + def handler(match: re.Match) -> Tuple[ + Optional[Replacement], + Optional[Error], + ]: name = match.group(1) if len(name) == 0: - return None + return None, None if not name in symbol_lookup: - # report_error( - # chapter, - # match.start(0), - # match.end(0), - # f"missing symbol: {name}", - # ) - return None + return None, ( + match.start(0), + f"missing symbol: {name}", + ) symbol = symbol_lookup[name] @@ -267,7 +281,7 @@ def handler(match: re.Match) -> Optional[Replacement]: match.start(0), match.end(0), render_symbol_link(symbol), - ) + ), None return handle_pattern(API_REGEX, handler) @@ -311,17 +325,25 @@ def handler(match: re.Match) -> Optional[Replacement]: transform_api(symbol_lookup), ] - errors: int = 0 - def report_error(chapter, start, end, msg): - global errors - errors += 1 - print(f"{chapter['name']}:{start}{end}: {msg}", file=sys.stderr) + num_errors: int = 0 def transform_chapter(chapter) -> None: - content = chapter['content'] + global num_errors + content: str = chapter['content'] for transform in transformers: - content = transform(content) + content, errors = transform(content) + + for index, message in errors: + num_errors += 1 + # not accurate since other transformers may have changed this, + # but whatever + line = len(content[:index].split("\n")) + + print( + f"{chapter['name']}:{line}: {message}", + file=sys.stderr, + ) chapter['content'] = content @@ -337,8 +359,8 @@ def transform_chapter(chapter) -> None: transform_chapter(section['Chapter']) - if errors > 0: - print(f"{errors} error(s) while preprocessing") + if num_errors > 0: + print(f"{num_errors} error(s) while preprocessing", file=sys.stderr) exit(1) print(json.dumps(book)) From ad5605fb892242f526c347fb2bd2e47d5197a357 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Sat, 6 Jul 2024 11:59:18 +0800 Subject: [PATCH 11/15] feat: automatic package documentation --- docs/gendoc.py | 54 +++++++++++++++++++++++++++++++++------ docs/src/SUMMARY.md | 4 +++ docs/src/documentation.md | 1 + docs/src/packages.md | 3 +++ 4 files changed, 54 insertions(+), 8 deletions(-) create mode 100644 docs/src/documentation.md create mode 100644 docs/src/packages.md diff --git a/docs/gendoc.py b/docs/gendoc.py index 5d56fec2..abd57a37 100644 --- a/docs/gendoc.py +++ b/docs/gendoc.py @@ -20,10 +20,6 @@ Callable, ) -GENDOC_REGEX = re.compile("{{gendoc (.+)}}") -KEYS_REGEX = re.compile(r"{{keys (.+)}}") -API_REGEX = re.compile(r"{{api ([a-z0-9/-]+)}}") - class Symbol(NamedTuple): Name: str @@ -232,7 +228,7 @@ def handler(match: re.Match) -> Tuple[ output, ), None - return handle_pattern(GENDOC_REGEX, handler) + return handle_pattern(re.compile("{{gendoc (.+)}}"), handler) def transform_keys( @@ -255,7 +251,7 @@ def handler(match: re.Match) -> Tuple[ ), ), None - return handle_pattern(KEYS_REGEX, handler) + return handle_pattern(re.compile(r"{{keys (.+)}}"), handler) def transform_api( @@ -283,7 +279,48 @@ def handler(match: re.Match) -> Tuple[ render_symbol_link(symbol), ), None - return handle_pattern(API_REGEX, handler) + return handle_pattern(re.compile(r"{{api ([a-z0-9/-]+)}}"), handler) + + +def transform_packages() -> Transformer: + pkg_dir = os.path.join(os.path.dirname(__file__), "..", "pkg") + + packages: List[Tuple[str, str]] = [] + + for dir, _, _ in os.walk(pkg_dir): + relative = os.path.relpath(dir, start=pkg_dir) + readme = os.path.join(dir, "README.md") + if not os.path.exists(readme): continue + + with open(readme, 'r') as f: + packages.append(( + relative, + f.read(), + )) + + packages = sorted( + packages, + key=lambda a: a[0], + ) + + docs = "" + + for name, readme in packages: + # Skip the first line, usually # + readme = "\n".join(readme.split("\n")[1:]) + docs += f"""## {name} + +[source](https://github.com/cfoust/cy/tree/main/pkg/{name}) + +{readme}""" + + def handler(match: re.Match) -> Tuple[ + Optional[Replacement], + Optional[Error], + ]: + return (match.start(0), match.end(0), docs,), None + + return handle_pattern(re.compile(r"{{packages}}"), handler) if __name__ == '__main__': @@ -316,12 +353,13 @@ def handler(match: re.Match) -> Tuple[ bindings.append(Binding(**binding)) transformers: List[Transformer] = [ + transform_packages(), + transform_keys(bindings), transform_gendoc( api['Frames'], api['Animations'], symbols, ), - transform_keys(bindings), transform_api(symbol_lookup), ] diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 514433cc..6732428b 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -40,8 +40,12 @@ - [Architecture](./architecture.md) +- [Packages](./packages.md) + - [Stories](./stories.md) +- [Documentation site](./documentation.md) + --- [Acknowledgments](./acknowledgments.md) diff --git a/docs/src/documentation.md b/docs/src/documentation.md new file mode 100644 index 00000000..25f8d456 --- /dev/null +++ b/docs/src/documentation.md @@ -0,0 +1 @@ +# Documentation diff --git a/docs/src/packages.md b/docs/src/packages.md new file mode 100644 index 00000000..957957e8 --- /dev/null +++ b/docs/src/packages.md @@ -0,0 +1,3 @@ +# Packages + +{{packages}} From 885e7b075289cdd40b6b8bc28ef7979473d6e277 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Sat, 6 Jul 2024 12:06:24 +0800 Subject: [PATCH 12/15] feat: build docs on every commit --- .github/workflows/ci.yml | 16 ++++++++++++++++ docs/storybook.py | 4 ++++ 2 files changed, 20 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2d6e3c7f..5cc44dfa 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -22,3 +22,19 @@ jobs: - name: Test run: go test -v ./pkg/... ./cmd/... + + - name: Install latest mdbook + run: | + tag=$(curl 'https://api.github.com/repos/rust-lang/mdbook/releases/latest' | jq -r '.tag_name') + url="https://github.com/rust-lang/mdbook/releases/download/${tag}/mdbook-${tag}-x86_64-unknown-linux-gnu.tar.gz" + mkdir mdbook + curl -sSL $url | tar -xz --directory=./mdbook + echo `pwd`/mdbook >> $GITHUB_PATH + + - name: Build book + run: | + cd docs + # Building all of the assets for the docs site takes a while; we only + # build it on release. This is just to verify that this PR/commit did + # not break any documentation. + CY_SKIP_ASSETS=1 mdbook build diff --git a/docs/storybook.py b/docs/storybook.py index 3da68233..e0c8bcc7 100644 --- a/docs/storybook.py +++ b/docs/storybook.py @@ -96,6 +96,10 @@ def transform_chapter(chapter): Path("./src/images").mkdir(parents=True, exist_ok=True) + if 'CY_SKIP_ASSETS' in os.environ: + print(f"CY_SKIP_ASSETS enabled, not building assets", file=sys.stderr) + jobs = {} + for filename, command in jobs.items(): if os.path.exists(filename): continue From 25dfc576ecfc74cfedd6d6a151380f5980538a93 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Sun, 7 Jul 2024 10:32:17 +0800 Subject: [PATCH 13/15] feat: loads of READMEs --- docs/gendoc.py | 9 ++++++++- docs/src/architecture.md | 2 +- docs/src/packages.md | 2 ++ pkg/anim/README.md | 3 +++ pkg/bind/README.md | 5 +++++ pkg/cy/README.md | 3 +++ pkg/emu/README.md | 11 +++++++---- pkg/events/module.go | 1 + pkg/frames/README.md | 3 +++ pkg/fuzzy/README.md | 3 +++ pkg/fuzzy/fzf/README.md | 2 ++ pkg/geom/README.md | 6 ++++++ pkg/io/README.md | 3 +++ pkg/janet/README.md | 8 ++++++-- pkg/mux/README.md | 3 +++ pkg/params/README.md | 3 +++ pkg/replay/README.md | 3 +++ pkg/sessions/README.md | 3 +++ pkg/sessions/search/README.md | 3 +++ pkg/stories/README.md | 3 +++ pkg/taro/README.md | 6 +++++- pkg/util/dir/README.md | 2 ++ 22 files changed, 78 insertions(+), 9 deletions(-) create mode 100644 pkg/anim/README.md create mode 100644 pkg/bind/README.md create mode 100644 pkg/cy/README.md create mode 100644 pkg/frames/README.md create mode 100644 pkg/fuzzy/README.md create mode 100644 pkg/geom/README.md create mode 100644 pkg/io/README.md create mode 100644 pkg/mux/README.md create mode 100644 pkg/params/README.md create mode 100644 pkg/replay/README.md create mode 100644 pkg/sessions/README.md create mode 100644 pkg/sessions/search/README.md create mode 100644 pkg/stories/README.md diff --git a/docs/gendoc.py b/docs/gendoc.py index abd57a37..8f807a89 100644 --- a/docs/gendoc.py +++ b/docs/gendoc.py @@ -306,8 +306,15 @@ def transform_packages() -> Transformer: docs = "" for name, readme in packages: + lines = readme.split("\n") # Skip the first line, usually # - readme = "\n".join(readme.split("\n")[1:]) + lines = lines[1:] + # Increase header level + lines = list(map( + lambda line: "#" + line if line.startswith("#") else line, + lines, + )) + readme = "\n".join(lines) docs += f"""## {name} [source](https://github.com/cfoust/cy/tree/main/pkg/{name}) diff --git a/docs/src/architecture.md b/docs/src/architecture.md index e399e370..f3f6da47 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -4,7 +4,7 @@ This document is intended to be a brief introduction to `cy`'s code structure an It is safe to assume that the high-level description in this document will remain reliable despite changes in the actual implementation, but if you are ever in doubt: -1. Read the README for the package you are modifying. Most packages in `pkg` have their own READMEs (along with some sub-packages.) +1. Read the README for the package you are modifying. Most [packages](./packages.md) in `pkg` have their own READMEs (along with some sub-packages.) 2. Ask for help [in Discord](https://discord.gg/NRQG3wbWGM). 3. Consult the code itself. diff --git a/docs/src/packages.md b/docs/src/packages.md index 957957e8..fa13a565 100644 --- a/docs/src/packages.md +++ b/docs/src/packages.md @@ -1,3 +1,5 @@ # Packages +This chapter contains an index of all of the Go packages in `cy`'s `pkg` directory, the READMEs of which are consolidated here for your convenience. + {{packages}} diff --git a/pkg/anim/README.md b/pkg/anim/README.md new file mode 100644 index 00000000..769db146 --- /dev/null +++ b/pkg/anim/README.md @@ -0,0 +1,3 @@ +# anim + +Package anim contains a range of terminal animations. These are used on `cy`'s splash screen and in the background while fuzzy finding. diff --git a/pkg/bind/README.md b/pkg/bind/README.md new file mode 100644 index 00000000..999f00ea --- /dev/null +++ b/pkg/bind/README.md @@ -0,0 +1,5 @@ +# bind + +Package bind is a key binding engine. It checks incoming key events against all registered key bindings to determine whether an action should be fired. bind uses a trie data structure, implemented in the bind/trie package, to describe sequences of keys. + +As distinct from a traditional trie, in which nodes have a fixed value, bind's trie also supports regex values. Key events are stringified and then compared against the regex pattern to determine if the state machine should transition to that node. diff --git a/pkg/cy/README.md b/pkg/cy/README.md new file mode 100644 index 00000000..391370c3 --- /dev/null +++ b/pkg/cy/README.md @@ -0,0 +1,3 @@ +# cy + +Package cy contains `cy`'s server and Janet API. diff --git a/pkg/emu/README.md b/pkg/emu/README.md index 8a9fe6b5..d31ec19e 100644 --- a/pkg/emu/README.md +++ b/pkg/emu/README.md @@ -1,6 +1,9 @@ # emu (formerly vt10x) -Package emu is a emu terminal emulation backend, influenced -largely by st, rxvt, xterm, and iTerm as reference. Use it for terminal -muxing, a terminal emulation frontend, or wherever else you need -terminal emulation. +Package emu provides a VT100-compatible terminal emulator. For the most part it attempts to emulate xterm as closely as possible (ie to be used with `TERM=xterm-256color.`) + +emu's basic mode of operation is quite simple: you `Write()` some bytes and it will correctly calculate the state of the virtual terminal which you can happily capture and send elsewhere (with `Terminal.View()`). + +emu's magic, however, comes from `Terminal.Flow()`, which is an API for viewing the terminal's scrollback buffer _with a viewport of arbitrary size_. This is important because `cy`'s core feature is to be able to replay terminal sessions, the lines of which should wrap appropriately to fit your terminal screen. + +This package is a fork of [github.com/hinshun/vt10x](https://github.com/hinshun/vt10x). The original library was rough, incomplete, and had a range of serious bugs that I discovered after integrating it. Because of this, little of the original code remains. diff --git a/pkg/events/module.go b/pkg/events/module.go index 155e3168..2043a28b 100644 --- a/pkg/events/module.go +++ b/pkg/events/module.go @@ -4,4 +4,5 @@ import ( tea "github.com/charmbracelet/bubbletea" ) +// TODO(cfoust): 07/07/24 Why does this exist? type Msg = tea.Msg diff --git a/pkg/frames/README.md b/pkg/frames/README.md new file mode 100644 index 00000000..3f30e51e --- /dev/null +++ b/pkg/frames/README.md @@ -0,0 +1,3 @@ +# frames + +Package frames contains static backgrounds. diff --git a/pkg/fuzzy/README.md b/pkg/fuzzy/README.md new file mode 100644 index 00000000..db95d05e --- /dev/null +++ b/pkg/fuzzy/README.md @@ -0,0 +1,3 @@ +# fuzzy + +Package fuzzy is a fully-featured fuzzy finder a la fzf. In fact, it makes use of fzf's actual agorithm, forked here as the fuzzy/fzf package. diff --git a/pkg/fuzzy/fzf/README.md b/pkg/fuzzy/fzf/README.md index 7a4da2a4..44149008 100644 --- a/pkg/fuzzy/fzf/README.md +++ b/pkg/fuzzy/fzf/README.md @@ -1 +1,3 @@ +# fzf + This is a fork of [fzf's matching algorithm](https://github.com/junegunn/fzf/tree/master/src/algo). diff --git a/pkg/geom/README.md b/pkg/geom/README.md new file mode 100644 index 00000000..5cd559fa --- /dev/null +++ b/pkg/geom/README.md @@ -0,0 +1,6 @@ +# geom + +Package geom provides a range of geometric primitives and convenience methods. Notably, it contains: + +* Data types for representing static bitmaps of terminal data (`image.Image`) and terminal state (`tty.State`). +* `Vec2`, a traditional vector data type but with a terminal flavor: ie it uses `R` and `C` (for rows and columns) instead of `X` and `Y` diff --git a/pkg/io/README.md b/pkg/io/README.md new file mode 100644 index 00000000..594ea4c4 --- /dev/null +++ b/pkg/io/README.md @@ -0,0 +1,3 @@ +# io + +Package io is an assortment of packages with a general theme of IO, including the protocol used by clients to interact with the `cy` server. diff --git a/pkg/janet/README.md b/pkg/janet/README.md index fa044209..12e02687 100644 --- a/pkg/janet/README.md +++ b/pkg/janet/README.md @@ -1,3 +1,7 @@ -# go-janet +# janet -To update the [janet](https://github.com/janet-lang/janet) version, clone it and run `make`, then copy `build/c/janet.c`, `src/include/janet.h`, and `src/conf/janetconf.h` to this directory. +Package janet contains a Janet virtual machine for interoperation between Go and Janet code. Users of its API can register callbacks, define symbols in the Janet environment, execute Janet code, and convert between Go and Janet values. + +## Updating Janet + +To update the [janet](https://github.com/janet-lang/janet) version, clone the janet-lang/janet repository and run `make`, then copy `build/c/janet.c`, `src/include/janet.h`, and `src/conf/janetconf.h` to this directory. diff --git a/pkg/mux/README.md b/pkg/mux/README.md new file mode 100644 index 00000000..5f8dab02 --- /dev/null +++ b/pkg/mux/README.md @@ -0,0 +1,3 @@ +# mux + +Package mux defines `Screen` and `Stream`, two of `cy`'s core abstractions for representing interactive windows and streams of terminal data, respectively. It also contains a wide range of useful `Screen`s and `Stream`s used across `cy`. diff --git a/pkg/params/README.md b/pkg/params/README.md new file mode 100644 index 00000000..8bcd9adb --- /dev/null +++ b/pkg/params/README.md @@ -0,0 +1,3 @@ +# params + +Package params is a thread-safe map data structure used as a key-value store for all nodes in `cy`'s node tree. diff --git a/pkg/replay/README.md b/pkg/replay/README.md new file mode 100644 index 00000000..4448f1f2 --- /dev/null +++ b/pkg/replay/README.md @@ -0,0 +1,3 @@ +# replay + +Package replay is an interface for playing, searching, and copying text from recorded terminal sessions. diff --git a/pkg/sessions/README.md b/pkg/sessions/README.md new file mode 100644 index 00000000..d47fabde --- /dev/null +++ b/pkg/sessions/README.md @@ -0,0 +1,3 @@ +# sessions + +Package sessions contains a data type for recorded terminal sessions and a range of utilities for (de)serializing, searching through, and exporting them. diff --git a/pkg/sessions/search/README.md b/pkg/sessions/search/README.md new file mode 100644 index 00000000..8034322b --- /dev/null +++ b/pkg/sessions/search/README.md @@ -0,0 +1,3 @@ +# search + +Package search is a high-performance search algorithm to find matches for a regex pattern on the terminal screen over the course of a recorded terminal session. This is more complicated than it seems: it must track the exact byte at which a match first appeared and calculate how long that match remained intact on the screen. diff --git a/pkg/stories/README.md b/pkg/stories/README.md new file mode 100644 index 00000000..47842bac --- /dev/null +++ b/pkg/stories/README.md @@ -0,0 +1,3 @@ +# stories + +Package stories is an interface for registering and viewing stories. Stories are predefined configurations of `cy`'s UI components that may also describe a sequence of user inputs that are "played" into the story after it is loaded. This is similar to [Storybook](https://storybook.js.org/). diff --git a/pkg/taro/README.md b/pkg/taro/README.md index 59248d22..11467505 100644 --- a/pkg/taro/README.md +++ b/pkg/taro/README.md @@ -1 +1,5 @@ -Much of this code was forked from [charmbracelet/bubbletea](https://github.com/charmbracelet/bubbletea). I needed to be able to expand on its key/mouse event parsing and build a new `Program`-esque abstraction. +# taro + +Package taro is a high level framework for defining terminal interfaces that obey cy's `Screen` interface. It is a fork of [charmbracelet/bubbletea](https://github.com/charmbracelet/bubbletea) and borrows that library's state machine paradigm, originally inspired by [the Elm framework](https://elm-lang.org/). + +I wanted bubbletea `Program`s to be able to write to arbitrary parts of the screen without futzing with strings. I also needed to improve on bubbletea's key/mouse event parsing (which, at any rate, has since been patched). diff --git a/pkg/util/dir/README.md b/pkg/util/dir/README.md index ef69a1ab..ceff81ce 100644 --- a/pkg/util/dir/README.md +++ b/pkg/util/dir/README.md @@ -1 +1,3 @@ +# dir + Taken from [robertknight/rd](https://github.com/robertknight/rd/tree/master). No license specified. From fb27f7a34b3b3bbbf27224304b32d955c444856e Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Sun, 7 Jul 2024 10:57:17 +0800 Subject: [PATCH 14/15] feat: stories documentation --- cmd/stories/main.go | 29 +++++++++++++++++++++++++++++ docs/src/stories.md | 42 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 71 insertions(+) diff --git a/cmd/stories/main.go b/cmd/stories/main.go index 5c9c6902..af0a5409 100644 --- a/cmd/stories/main.go +++ b/cmd/stories/main.go @@ -81,6 +81,35 @@ func main() { }(animation) } + // A stories story + stories.Register( + "stories", + func(ctx context.Context) (mux.Screen, error) { + return ui.New(ctx, CLI.Prefix) + }, + stories.Config{ + Input: []interface{}{ + stories.Type("ctrl+j"), + stories.Wait(stories.Some), + stories.Type("ctrl+j"), + stories.Wait(stories.Some), + stories.Type("ctrl+k"), + stories.Wait(stories.Some), + stories.Type("ctrl+k"), + stories.Wait(stories.Some), + stories.Type("input"), + stories.Type("ctrl+j"), + stories.Wait(stories.Some), + stories.Type("ctrl+j"), + stories.Wait(stories.Some), + stories.Type("ctrl+k"), + stories.Wait(stories.Some), + stories.Type("ctrl+k"), + stories.Wait(stories.Some), + }, + }, + ) + haveCast := len(CLI.Cast) > 0 if len(CLI.Single) == 0 && haveCast { panic(fmt.Errorf("to use --cast, you must provide a single story")) diff --git a/docs/src/stories.md b/docs/src/stories.md index 70ac3484..210efe16 100644 --- a/docs/src/stories.md +++ b/docs/src/stories.md @@ -1 +1,43 @@ # Stories + +{{story cast stories --width 120 --height 26}} + +> The above is the stories interface. Typing filters the list of stories and you can use up and down to move between them. Stories are not interactive, though this may change. + +`cy`'s user interface is complex and some UI states are tedious to navigate to when you're trying to iterate quickly. To remedy this, the `cy` repository contains a mechanism (uncreatively) called **stories**. + +A **story** is a preconfigured [`Screen`](architecture.html#screen) along with an (optional) sequence of user inputs that will be played back on that screen. Every story also has a unique string name that looks like a path, e.g. `input/find/search`. After defining a story in Go code, you can open a special interface that lets you quickly view that story. + +Stories can be registered by any package in `cy` and can be browsed in a central interface. + +This functionality was inspired by [Storybook](https://storybook.js.org/), a framework used to develop UI components. + +## Viewing stories + +Run the following to open the stories interface: + +```bash +go run ./cmd/stories/main.go +``` + +Press q to quit at any time. + +The stories executable accepts a range of arguments. Provide `--help` to see them all. + +To run only a single story: + +```bash +go run ./cmd/stories/main.go -s input/find/search +``` + +To filter the list of stories with a prefix: + +```bash +go run ./cmd/stories/main.go -p input +``` + +Any stories with names that do not begin with `input` will be filtered out. + +## Registering a new story + +Stories are registered using the [`Register`](https://github.com/cfoust/cy/blob/main/pkg/stories/module.go?plain=1#L74) function in the [stories package](./packages.md#stories). Search the codebase for usage examples. From bdc2dc7ab31c52f167816a198bf5275e7a39af58 Mon Sep 17 00:00:00 2001 From: Caleb Foust Date: Sun, 7 Jul 2024 11:17:30 +0800 Subject: [PATCH 15/15] feat: doc docs --- docs/src/documentation.md | 52 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/docs/src/documentation.md b/docs/src/documentation.md index 25f8d456..843cb899 100644 --- a/docs/src/documentation.md +++ b/docs/src/documentation.md @@ -1 +1,51 @@ -# Documentation +# Documentation site + +`cy`'s documentation site lives in the repository's `docs` directory. It uses [mdbook](https://github.com/rust-lang/mdBook). After installing `mdbook`, you can serve the documentation site by running the following in the `docs` directory: + +```bash +mdbook serve +``` + +## Preprocessors + +`cy`'s documentation makes extensive use of [mdbook preprocessors](https://rust-lang.github.io/mdBook/format/configuration/preprocessors.html) to generate content and assets on the fly with the version of `cy` currently checked out in the repository. + +A preprocessor allows you to define custom transformations of the site's raw Markdown. `cy` uses this for a range of things described below. Using preprocessors, the `cy` documentation site defines a suite of special markup tags used to generate documentation and assets from the `cy` code. + +All of these markup tags are enclosed in double curly brackets (e.g. `{{some-tag}}`) to avoid interfering with Markdown directives. The documentation below omits these double brackets for the sake of implementation simplicity. + +### Stories + +The `story` tag allows you to render [stories](./stories.md) as static PNGs, animated GIFs, or an interactive [asciinema](https://docs.asciinema.org/) player. + +The default filename for generated assets is the hash of the tag's arguments. + +Some examples: + +```bash +# Generate a gif of the splash story and insert it +story gif splash + +# You can also specify a file name +story main.gif splash + +# Insert a png snapshot of the splash story +story png splash + +# Render an asciinema cast of the cy/replay story +story cast cy/replay + +# You can also specify the terminal dimensions of the story, which will +# overwrite the dimensions in the story's configuration +story cast cy/viewport --width 120 --height 26 +``` + +### API symbols + +You can reference symbols in `cy`'s API using the `api` tag, which will link to that symbol's documentation in the API reference. This is also useful because if you reference a symbol that does not exist, the preprocessor reports the error and fails CI. This is an effort to prevent broken links after API changes. + +For example: + +```bash +api input/find +```