Skip to content

Commit

Permalink
Merge pull request #28 from dlants/node
Browse files Browse the repository at this point in the history
Node
  • Loading branch information
dlants authored Jan 11, 2025
2 parents 0dcaf5d + 39175d0 commit e49733f
Show file tree
Hide file tree
Showing 83 changed files with 5,676 additions and 1,173 deletions.
20 changes: 6 additions & 14 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,28 +19,20 @@ jobs:
neovim: true
version: stable

- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version-file: ".bun-version"

- name: Install typescript-language-server
run: bun add -g typescript-language-server typescript
run: npm install -g typescript-language-server typescript

- name: Install dependencies
run: bun install --frozen-lockfile

- name: tsc version
run: bun x tsc --version
run: npm install --frozen-lockfile

- name: Run typecheck
run: bun x tsc --noEmit
run: npx tsc --noEmit

- name: Run eslint
run: bun x eslint .
run: npx eslint .

- name: Run prettier check
run: bun x prettier --check .
run: npx prettier --check .

- name: Run tests
run: bun test
run: npx vitest
43 changes: 20 additions & 23 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
# Contributions

Note! I am probably going to port this from bun to node, see https://github.com/dlants/magenta.nvim/issues/25


## issues and discussions

I'd like to keep the issues down to just known bugs and things that I'm confident about implementing. Support questions and feature requests should go in the discussion board.
Expand Down Expand Up @@ -31,7 +28,7 @@ require("lazy").setup {
"dlants/magenta.nvim",
dev = true,
lazy = false,
build = "bun install --frozen-lockfile",
build = "npm install --frozen-lockfile",
config = function()
require('magenta').setup()
end
Expand All @@ -44,39 +41,39 @@ This will load the plugin from `~/src/magenta.nvim` instead of from git. You can

All significant changes should come with accompanying tests. There should go into `*.spec.ts` files placed adjacent to the code they are testing. So if the "meat" of the functionality you're testing is in `a.ts`, you should put the test for that in `a.spec.ts` in the same directory.

To run tests, use `bun test`. You can also mark certain tests to run via `describe.only` or `it.only` and then run `bun test --only` to just test those (all of the tests in describe.only blocks will be run even if only some of them are annotated with it.only).
To run tests, use `npx vitest`. You can also mark certain tests to run via `describe.only` or `it.only` and then run `npx vitest filter` to just tests marked as only from test files matching `filter`.

Some test make use of the bun test snapshot feature. To update a snapshot, use `bun test --update-snapshots`.
Some test make use of the vitest snapshot feature. To update a snapshot, use `npx vitest -u`.

## how to see logs

You can run `bunx bunvim logs magenta` to see logs generated by the plugin.
You can run `npx tsx node_modules/nvim-node/src/cli/index.ts logs magenta` to see logs generated by the plugin.

You can run `bunx bunvim logs test` to see logs generated when running tests.
You can run `npx tsx node_modules/nvim-node/src/cli/index.ts logs test` to see logs generated when running tests.

These will contain all of the logs sent via `nvim.logger`. It will also log all of the RPC messages sent between neovim and the bun process.
These will contain all of the logs sent via `nvim.logger`. It will also log all of the RPC messages sent between neovim and the node process.

## how to debug

When invoking tests, you can run `bun test --only --inspect-wait`. This will print a url that you can open in your browser to bring up a debug console. This will stop on `debugger` statements encountered in the code, and will allow you to step through the code.
When invoking tests, you can run `npx vitest filter --inspect-wait`. This will print a url that you can open in your browser to bring up a debug console. This will stop on `debugger` statements encountered in the code, and will allow you to step through the code.

# Code orientation

## startup

When neovim starts, the `start` function is run in [init.lua](https://github.com/dlants/magenta.nvim/blob/main/lua/magenta/init.lua). This kicks off the bun process. Neovim creates a socket and passes it to the bun process via the `NVIM` env var.
When neovim starts, the `start` function is run in [init.lua](https://github.com/dlants/magenta.nvim/blob/main/lua/magenta/init.lua). This kicks off the node process. Neovim creates a socket and passes it to the node process via the `NVIM` env var.

The entrypoint for the bun process is [index.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/index.ts). This checks for the presence of the env variable, establishes the bunvim connection, and kicks off the static `Magenta.start` method.
The entrypoint for the node process is [index.ts](https://github.com/dlants/magenta.nvim/blob/main/node/index.ts). This checks for the presence of the env variable, establishes the nvim-node connection, and kicks off the static `Magenta.start` method.

The start function in [magenta.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/magenta.ts) sets up the notification listeners and calls the `require('magenta').bridge` method from `init.lua`. This passes the `channelId` back to the lua side, so that it can finish initializing the magenta lua module, which we can then invoke to communicate back to the plugin.
The start function in [magenta.ts](https://github.com/dlants/magenta.nvim/blob/main/node/magenta.ts) sets up the notification listeners and calls the `require('magenta').bridge` method from `init.lua`. This passes the `channelId` back to the lua side, so that it can finish initializing the magenta lua module, which we can then invoke to communicate back to the plugin.

Most commands are defined in `init.lua`, though some are defined on the bun side, like [sidebar.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/sidebar.ts#L93)
Most commands are defined in `init.lua`, though some are defined on the node side, like [sidebar.ts](https://github.com/dlants/magenta.nvim/blob/main/node/sidebar.ts#L93)

## testing setup

The startup for tests is a little different, handled in [test/preamble.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/test/preamble.ts). Here, the bun process starts first. In every tests, it creates an nvim socket, and then starts nvim with the `--listen` flag to attach to that socket. It then proceeds to init the magenta plugin against that socket, as in the normal startup sequence.
The startup for tests is a little different, handled in [test/preamble.ts](https://github.com/dlants/magenta.nvim/blob/main/node/test/preamble.ts). Here, the node process starts first. In every tests, it creates an nvim socket, and then starts nvim with the `--listen` flag to attach to that socket. It then proceeds to init the magenta plugin against that socket, as in the normal startup sequence.

Each test gets a fresh neovim instance. Convenience methods for interacting with the nvim/plugin setup live in [test/driver.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/test/driver.ts).
Each test gets a fresh neovim instance. Convenience methods for interacting with the nvim/plugin setup live in [test/driver.ts](https://github.com/dlants/magenta.nvim/blob/main/node/test/driver.ts).

## architecture

Expand All @@ -99,13 +96,13 @@ So the general flow is:

The key files are:

- [tea.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/tea/tea.ts) - sets up the render - dispatch - update cycle
- [view.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/tea/view.ts) - implements the VDOM-like declarative rendering template
- [tea.ts](https://github.com/dlants/magenta.nvim/blob/main/node/tea/tea.ts) - sets up the render - dispatch - update cycle
- [view.ts](https://github.com/dlants/magenta.nvim/blob/main/node/tea/view.ts) - implements the VDOM-like declarative rendering template

## code organization

- [magenta.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/magenta.ts) - the entrypoint. Sets up the communication with the neovim process, initializes the app, receives commands from the neovim process.
- [sidebar.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/sidebar.ts) - manages the chat sidebar state. Mostly just for showing/hiding it, managing the keybindings, etc...
- [toolManager.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/tools/toolManager.ts) - manages the tool executions and rendering. Each tool execution has an id, and this contains the state mapping that id to the execution state. Also provides tools for dealing with generic "tools".
- [provider.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/providers/provider.ts) - abstraction around an LLM provider. Creates general ways of declaring tools, messages and other interactions with various providers.
- [chat.ts](https://github.com/dlants/magenta.nvim/blob/main/bun/chat/chat.ts) - the top-level of the TEA app. Contains messages, context, etc... this is what is displayed in the chat display buffer.
- [magenta.ts](https://github.com/dlants/magenta.nvim/blob/main/node/magenta.ts) - the entrypoint. Sets up the communication with the neovim process, initializes the app, receives commands from the neovim process.
- [sidebar.ts](https://github.com/dlants/magenta.nvim/blob/main/node/sidebar.ts) - manages the chat sidebar state. Mostly just for showing/hiding it, managing the keybindings, etc...
- [toolManager.ts](https://github.com/dlants/magenta.nvim/blob/main/node/tools/toolManager.ts) - manages the tool executions and rendering. Each tool execution has an id, and this contains the state mapping that id to the execution state. Also provides tools for dealing with generic "tools".
- [provider.ts](https://github.com/dlants/magenta.nvim/blob/main/node/providers/provider.ts) - abstraction around an LLM provider. Creates general ways of declaring tools, messages and other interactions with various providers.
- [chat.ts](https://github.com/dlants/magenta.nvim/blob/main/node/chat/chat.ts) - the top-level of the TEA app. Contains messages, context, etc... this is what is displayed in the chat display buffer.
23 changes: 13 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,17 @@ Flagship models will continue to get better at tools use, and as this happens, t

# Installation (lazy.nvim)

Install [bun](https://bun.sh/)
Make sure you have [node](https://nodejs.org/en/download) installed, at least `v20`:

```
node --version
```

```lua
{
"dlants/magenta.nvim",
lazy = false, -- you could also bind to <leader>mt
build = "bun install --frozen-lockfile",
build = "npm install --frozen-lockfile",
config = function()
require('magenta').setup()
end
Expand All @@ -33,9 +37,9 @@ The plugin will look for configuration for providers in the following env variab

Global keymaps are set [here](https://github.com/dlants/magenta.nvim/blob/main/lua/magenta/init.lua#L12).

Input and display buffer keymaps are set [here](https://github.com/dlants/magenta.nvim/blob/main/bun/sidebar.ts#L87).
Input and display buffer keymaps are set [here](https://github.com/dlants/magenta.nvim/blob/main/node/sidebar.ts#L87).

Commands are all nested under `:Magenta <cmd>`, and can be found [here](https://github.com/dlants/magenta.nvim/blob/main/bun/magenta.ts#L54).
Commands are all nested under `:Magenta <cmd>`, and can be found [here](https://github.com/dlants/magenta.nvim/blob/main/node/magenta.ts#L54).

TLDR:

Expand All @@ -54,7 +58,7 @@ The display buffer is not modifiable, however you can interact with some parts o

## tools available to the LLM

See the most up-to-date list of implemented tools [here](https://github.com/dlants/magenta.nvim/tree/main/bun/tools).
See the most up-to-date list of implemented tools [here](https://github.com/dlants/magenta.nvim/tree/main/node/tools).

- [x] list a directory (only in cwd, excluding hidden and gitignored files)
- [x] list current buffers (only buffers in cwd, excluding hidden and gitignored files)
Expand All @@ -65,12 +69,11 @@ See the most up-to-date list of implemented tools [here](https://github.com/dlan

# Why it's cool

- It uses [bun](https://bun.sh/) for faster startup, a lower memory footprint, and ease of development with Typescript.
- It uses the new [rpc-pased remote plugin setup](https://github.com/dlants/magenta.nvim/issues/1). This means more flexible plugin development (can easily use both lua and typescript), and no need for `:UpdateRemotePlugins`! (h/t [wallpants](https://github.com/wallpants/bunvim)).
- The state of the plugin is managed via an elm-inspired architecture (The Elm Architecture or [TEA](https://github.com/evancz/elm-architecture-tutorial)) [code](https://github.com/dlants/magenta.nvim/blob/main/bun/tea/tea.ts). I think this makes it fairly easy to understand and lays out a clear pattern for extending the feature set, as well as [eases testing](https://github.com/dlants/magenta.nvim/blob/main/bun/chat/chat.spec.ts). It also unlocks some cool future features (like the ability to persist a structured chat state into a file).
- I spent a considerable amount of time figuring out a full end-to-end testing setup. Combined with typescript's async/await, it makes writing tests fairly easy and readable. The plugin is already fairly well-tested [code](https://github.com/dlants/magenta.nvim/blob/main/bun/magenta.spec.ts#L8).
- In order to use TEA, I had to build a VDOM-like system for rendering text into a buffer. This makes writing view code declarative. [code](https://github.com/dlants/magenta.nvim/blob/main/bun/tea/view.ts#L141) [example defining a tool view](https://github.com/dlants/magenta.nvim/blob/main/bun/tools/getFile.ts#L139)
- we can leverage existing sdks to communicate with LLMs, and async/await to manage side-effect chains, which greatly speeds up development. For example, streaming responses was pretty easy to implement, and I think is typically one of the trickier parts of other LLM plugins. [code](https://github.com/dlants/magenta.nvim/blob/main/bun/anthropic.ts#L49)
- The state of the plugin is managed via an elm-inspired architecture (The Elm Architecture or [TEA](https://github.com/evancz/elm-architecture-tutorial)) [code](https://github.com/dlants/magenta.nvim/blob/main/node/tea/tea.ts). I think this makes it fairly easy to understand and lays out a clear pattern for extending the feature set, as well as [eases testing](https://github.com/dlants/magenta.nvim/blob/main/node/chat/chat.spec.ts). It also unlocks some cool future features (like the ability to persist a structured chat state into a file).
- I spent a considerable amount of time figuring out a full end-to-end testing setup. Combined with typescript's async/await, it makes writing tests fairly easy and readable. The plugin is already fairly well-tested [code](https://github.com/dlants/magenta.nvim/blob/main/node/magenta.spec.ts#L8).
- In order to use TEA, I had to build a VDOM-like system for rendering text into a buffer. This makes writing view code declarative. [code](https://github.com/dlants/magenta.nvim/blob/main/node/tea/view.ts#L141) [example defining a tool view](https://github.com/dlants/magenta.nvim/blob/main/node/tools/getFile.ts#L139)
- we can leverage existing sdks to communicate with LLMs, and async/await to manage side-effect chains, which greatly speeds up development. For example, streaming responses was pretty easy to implement, and I think is typically one of the trickier parts of other LLM plugins. [code](https://github.com/dlants/magenta.nvim/blob/main/node/anthropic.ts#L49)

# How is this different from other coding assistant plugins?

Expand Down
Binary file removed bun.lockb
Binary file not shown.
Loading

0 comments on commit e49733f

Please sign in to comment.