Local development setup

Development these days uses Go, NVM (to manage your NodeJS install), Yarn 1.x (a NodeJS package manager), and Docker.

On Ubuntu 20.04, these steps should work:

1. Clone the DBHub.io source code repository

$ git clone https://github.com/sqlitebrowser/dbhub.io
$ cd dbhub.io

2. Install NVM

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash

3. Install NodeJS version 20 (not any other version!)

$ nvm install --latest-npm --lts 20
$ nvm alias default 20
$ nvm use 20

Confirm version 20 of NodeJS is active:

$ nvm list

4. Install Go manually

Download the appropriate installation package from the Go download page, then follow the instructions there to install it and verify your installation.

5. Install Yarn 1.x

$ npm install --global [email protected]

6. Install Docker

$ sudo apt install docker.io

7. Build local DBHub.io Docker image

$ yarn docker:build

This will take a few minutes, and likely download a GB or so of various docker image layers.

8. Start the DBHub.io Docker container

$ yarn docker:startlocal
$ yarn docker:recompile

You use yarn commands to control things.

For the full list of the DBHub.io yarn commands, look at the package.json file in the repo. Specifically look at the "script" section:

https://github.com/sqlitebrowser/dbhub.io/blob/ed2459acec21b1c1030e14955ca3c4d04e1518dd/package.json#L11-L28

The command yarn docker:startlocal starts the DBHub.io docker container you just created, with the DBHub.io daemons listening on specific tcp ports:

• DBHub.io web interface daemon listening on https://localhost:9443
• DBHub.io API daemon listening on https://localhost:9444
• DB4S end point daemon listening on https://localhost:9445.

You should be able to connect to either of the first two with a web browser, though you will need to tell it to ignore the invalid https certificate(s) they use. 😉

Install Cypress

$ yarn cypress:install

This downloads the Cypress (test suite) binary for your system, so the other yarn Cypress commands can use it.

The yarn commands

The most useful Yarn commands (from that package.json file) are:

yarn docker:emptydb

Terminates the dbhub daemons running inside the container, then drops the dbhub database and recreates a complete empty version of it. No tables, sequences, indexes, etc. Useful for emptying the database of all contents prior to loading a pgdump into it.

yarn docker:exec

Starts a cli shell (as the root user), logged into the running container.

yarn docker:recompile

Compiles your local DBHub.io source code in the container. eg any changes you've made in your local source code directory will be picked up and compiled into the running container

This command now does restart the running DBHub.io daemons in the container after recompiling, so you don't need to manually restart the daemons afterwards.

yarn docker:restart

Restarts the DBHub.io daemons running inside the local Docker container. Useful if you've changed stuff in the templates. If you've changed the backend Go code though, you'll need to recompile instead.

yarn docker:stop

Stops the local DBHub.io docker container

yarn docker:psql

Connects psql (the PostgreSQL cli) to the database running inside the container

yarn docker:tail

Shows the console output of the DBHub.io daemons in the container. Only works after they've been restarted at least once though (eg after yarn docker:restart)

yarn cypress:install

Downloads and installs the Cypress (test suite) executable

yarn cypress:open

Opens the Cypress GUI. Only works after you've installed Cypress using yarn cypress:install.

yarn cypress:test and yarn cypress:testff

Run the test suite on the DBHub.io daemons in the local Docker container, reporting pass/failure, etc. The test variant uses the bundled Electron browser, whereas the testff variant uses a locally installed Firefox to test with, if it's installed.

yarn mq:bind, yarn mq:conn, yarn mq:cons, yarn mq:ex, yarn mq:q

These display various pieces of information from RabbitMQ server running in the container. So far, only the yarn mq:q one has been all that useful when developing, as it shows the number of messages waiting in each AMQP queue. (Was useful in tracking down a message response bug)

Using JetBrains IDEA as your IDE

JetBrains IDEA allows for remote debugging of Go executables (when configured).

There is an additional yarn command useful here:

yarn docker:debug

Restarts the DBHub.io daemons inside the local docker container, but does so in a way that makes them ready for debugging.

Use this instead of yarn docker:restart when you're using JetBrains IDEA and want to do a debugging session.

Note that when you're using this command, you cannot connect to the DBHub.io daemons in your docker container (eg using a web browser or whatever) until you've first started a debugging session in your IDE.

eg configure Jetbrains IDEA to connect to them, then press the "Debug" button in IDEA

This is because when the DBHub.io daemon start in debug mode in the container, they start in a "paused" state. When your IDE connects to them, it starts them running.

Configuration in the Jetbrains IDEA GUI is super simple. Here's the screenshot from mine:

You only really need to get the host (localhost) and port (2345) right to match what the Docker container is using.

• Port 2345 for the DBHub.io webUI daemon
• Port 2346 for the DBHub.io API daemon
• Port 2347 for the DB4S end point daemon
  • Although the DB4S end point debugger has technically been setup on port 2347, no-one's tried debugging it in the docker container yet. So it may need some additional tweaking to work 😉
• Port 2348 for the DB4S live daemon (first running instance, called "node1")
• Port 2349 for the DB4S live daemon (second running instance, called "node2")