Implement a shell in TypeScript running on Bun

[vjpr]

Summary

I would prefer to configure my shell in TypeScript. A shell needs to start quickly. Bun is the first runtime to make this feasible.

Problem

Shell config files are messy and complex

Every developer has a large and messy .bash_profile, .bashrc, .zshrc, etc. Tons of tools append lines to these files. There is a smattering of alias, export PATH=foo/bar:${PATH}, and other wierd hacks. If you mess up the ordering of PATH modifications, could can break heaps of stuff. There are also lots of flags that can interfere with compilers such as LIBRARY_PATH.

You can also find heaps of StackOverflow questions where the answer is to paste in a bunch of stuff into your shell config, and then a bunch of comments saying that this doesn't work and it actually breaks other things. "Lines of shell script" solutions are not safe - it would be better to have a more abstract api.

Cross-platform

These files are also not cross platform. Ideally you want similar stuff on Windows.

Also, any script you write probably won't run on Windows. TS is a better and safer option.

Shell script sucks

TypeScript is a much better language that ShellScript. Everything is typesafe, and it's syntax is easier and more familiar. I never want to write ShellScript, I am always forced to.

Limited output

The current pty/terminal emulator API is ANSI escape codes. This limits what is possible to do with the output. There are often times it would be better to allow tools to communicate via a richer API. Sometimes I want an html table, or a collapsible element, and links to jump up to errors, etc. We can do much better than ansi hyperlinks.

Plugins and package management

A shell config is a huge dependency graph. A series of config options are needed to support certain tools.

Solution

It would be far better to use an abstracted API to configure your shell.

With TS, it can be typesafe.

An API could also be generalized to work with not only terminal emulators, but also web pages/apps.

The current pty/terminal emulator API is ANSI escape codes. Almost everything would be better as HTML. There are already some abstract CLI formatting libraries such as https://github.com/vadimdemedes/ink and chalk + supports-color.

Plugins would be better done as npm packages.

All config should feed into an intermediary model that can check config issues, and also be easily visualizable.

Similar

There is already hyper which is a terminal emulator written in JS. I'm talking about writing a shell.

Why Bun?

A shell needs to start quickly. Before Bun, there was no TypeScript runtime that me these requirements.

Motivation

I had a config issue where I accidentally messed up the ordering of some config lines and brew reverted to intel arch from apple silicon arch, and everything broke slightly.

I then ran a TS script with Bun, from my shell config file, that would communicate via unix pipes, to eval commands. This allows me to manage all my shell config now in TS, but still use ZSH (which mac uses as its default login shell now).

The next level would be that the shell is actually Bun running a TS script, that can write all its output via websockets or ansi or shared memory in a future Bun desktop runtime.

Approach

a. Shell would be entirely implemented in TS using some fast Bun apis, bun shell.ts
b. Core shell functionality could be written in Zig, runnable like bun shell, and then offer an easy communication channel to run TS code. Currently uses pipes/sockets to do shell <-> TS/Bun is convoluted.
c. Leave zsh as the main shell, and offer an easy way to run shell commands from TS (for reasons listed in line above).

Interesting

Terminal emulator and shell (and shell script) are core to all devs. Would be a good way to grow Bun's appeal to dev crowd.

Related

#688

[vjpr]

Here is an example of my script to allow TS code to communicate with zsh shell.

Note that is long and complex and tackles lots of edge cases.

I used pipes because Bun didn't support sockets yet.

#!/bin/bash

IS_BUN=1

# DEBUG
IS_DEBUG=0

IS_LOG=0


# --------------------

basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
pkgdir="$basedir/.."

# --------------------

export SHELL_HELPER_SCRIPT_START_TIME=$(date +"%Y-%m-%dT%H:%M:%S%z")

dir=/tmp/live-shell-helper
pipeIn=$dir/pipe-in
pipeOut=$dir/pipe-out

# ------------------------------------------------------------------------------

main() {

  # --------------------

  # We must explicitly set `node` because this script runs before `.zshrc` can set it up.
  export PATH="$PATH:/Users/Vaughan/Library/pnpm"
  export PATH="$PATH:/Users/Vaughan/.bun/bin"

  # --------------------

  setup_pipes

  [[ $IS_DEBUG -eq 1 ]] && set -x

  loop

  [[ $IS_DEBUG -eq 1 ]] && set +x

  log "Shell script finished"

}

log() {
  [[ $IS_LOG -eq 1 ]] && echo "[sh]" $@
}

setup_pipes() {

  # Cleanup pipes on exit.

  mkdir -p $dir

  # ---

  # Removes pipes when finished executing script.
  # TODO: Ensure this is when this script ends, not when our terminal closes.

  # We don't wrap in a function because otherwise we would have to move it to the top scope.
  #  trap "rm $pipeIn" EXIT
  #  trap "rm $pipeOut" EXIT

  # NOTE: '-p' checks if its a named pipe.

  #  rm $pipeIn 2>/dev/null
  #  if [[ ! -p $pipeIn ]]; then
  #    mkfifo $pipeIn
  #  fi
  #
  #  rm $pipeOut 2>/dev/null
  #  if [[ ! -p $pipeOut ]]; then
  #    mkfifo $pipeOut
  #  fi

  # ---

  log "made pipes"
  rm $pipeIn $pipeOut
  mkfifo $pipeIn $pipeOut
  #ls $pipeIn $pipeOut

}

# ------------------------------------------------------------------------------

# Run background job silently.
# From (modified): https://superuser.com/a/1334617/23916

# Run the command given by "$@" in the background.
silent_background() {
  if [[ -n $ZSH_VERSION ]]; then

    # zsh:  https://superuser.com/a/1285272/365890

    setopt local_options no_notify no_monitor
    # We'd use &| to background and disown, but incompatible with bash, so:

    # Some output.
    # NOTE: '1>&1' redirects bg proc stdout to our script's stdout.
    "$@" 1>&1 &

    # No output.
    #"$@" &

  elif [[ -n $BASH_VERSION ]]; then

    # bash: https://stackoverflow.com/a/27340076/5353461

    { "$@" 2>&3 & } 3>&2 2>/dev/null

  else

    # Unknownness - just background it
    "$@" &
  fi

  # TODO: Look up what this does.
  disown &>/dev/null # Close STD{OUT,ERR} to prevent whine if job has already completed
}

background() {
  "$@" &
}

# ------------------------------------------------------------------------------

start() {
  if [ $IS_BUN -eq 1 ]; then
    # a. Use bun
    silent_background $pkgdir/bin/index.js
  else
    # b. Use node
    silent_background $pkgdir/bin/index.sh
  fi

  BGPROCPID=$!
  log "coprocess started"
}

# ------------------------------------------------------------------------------

# Re: FIFO re-opening...
# https://stackoverflow.com/questions/33773710/why-read-n-from-fifo-file-will-lost-data-in-shell
# https://stackoverflow.com/questions/25900873/write-and-read-from-a-fifo-from-two-different-script

loop() {

  start

  # ---

  # We shouldn't send a bonjour message because our JS script will be slower to start and
  #   when running Bun, if we don't have a pipe listening yet, we get an 'Interrupted system call' message.

  # NOTE: 'Interrupted system call' error can be caused by opening a FIFO.

  echo "bonjour" >$pipeOut
  log "errorno: $?"

  #echo "bonjour2" >$pipeOut
  #echo "errorno:" $?
  #echo "bonjour3" >$pipeOut
  #echo "errorno:" $?
  #log "Sent bonjours"

  # ---

  # From: https://stackoverflow.com/a/41046119/130910

  while :; do # If we hit end-of-FIFO, then loop around and try to reopen
    while read -r line; do
      log "SH recv:" ${line}

      if [[ "$line" == 'quit' ]]; then
        break 2 # Break out of both loops.
      fi

      {
        # try
        if [ $IS_BUN -eq 1 ]; then
          # NOTE: When reading the 'set -x' logs, it seems to escape everything.
          #   The 'eval' logs show unescaped though, so its seems fine.
          eval $line
        else
          eval $line >$pipeOut
        fi
      } || {
        # catch
        log "Last error code: $?"
      }
    done <$pipeIn # Note that the < "$pipeIn" is on the loop, not on the 'read'...
  done

}

# ------------------------------------------------------------------------------

main "$@"

---

A sockets alternative:

From: https://unix.stackexchange.com/a/336919/24559

#!/bin/bash

coproc nc -l localhost 3000

while read -r cmd; do
  case $cmd in
    d) date ;;
    q) break ;;
    *) echo 'What?'
  esac
done <&"${COPROC[0]}" >&"${COPROC[1]}"

kill "$COPROC_PID"