Skip to content

Folder Structure

Jump to bottom
Akshay Katyal edited this page Jul 25, 2023 · 2 revisions

The Framework

I have been building graphql APIs in Golang for a while. Based on what I have learned so far, I have created this framework, focussing on the following principles:

  1. Single Responsibility
  2. Configurability
  3. Testability
  4. Dependency Isolation
  5. Environment Parity

One question I get asked frequently: Why did I build a custom framework when there are so many good ones available in the market? The answer is pretty straightforward, I wanted a super-fast framework with best practices in place and without any bloat. So I started by connecting a few pieces together & slowly we reached a place where I was satisfied with the speed & developer productivity that I was on the lookout for. I have nothing against other open source frameworks out there. At the time when I started writing my service with this framework, none of the frameworks supported all the 5 points mentioned above & the only choice I had was to wire different libraries together & build our own.

The general idea of the framework is to divide each responsibility into go packages and abstract each part of the stack as much as possible. The code structure has evolved over time and will keep evolving as we find more use cases & edge conditions throughout the lifecycle of building out products that users ♥.

To give you a general idea of the framework, the following technologies are used in the stack:

  • Golang (1.17 and above)
  • Beego ORM (Just ORM from the beego framework)
  • Gin (For handling REST requests, if any)
  • PostgreSQL for storing relational data
  • GraphQL for API abstractions to facilitate inter-service & client-server communication

Dependency Injection

The most important idea behind the framework is the concept of dependency injection. If you don’t know about dependency injection, it is a pretty simple concept that says:

ℹ️ Decouple your dependencies

In simple terms, do not import your dependencies directly into the target file. Instead, inject them through an initialization method. To support this idea, (almost) each file is defined in the form of a public interface with a private struct & an initialization method that takes a dependency as the input.

For example, let's assume the following scenario:

  • You have a type called Tyre in the parts package
  • You are creating a Car object which takes the Tyre as a dependency

This is how your code will look in car.go

package vehicle

import "service/parts"

type Car interface {
  Drive() error
}

type car struct {
  tyrePart parts.Tyre
}

func (c *car) Drive() error {
  // drive the car
  err := c.tyrePart.Move(ctx)
  return err
}

func NewCar(tyrePart parts.Tyre) Car {
  return &car{
    tyrePart: tyrePart,
  }
}

What we did here was:

  • Create a Car interface that defines what functions a car can have
  • Define a car struct that has tyre as one of the parts as a property
  • Create an initialization method called NewCar which is nothing but a constructor for the car object and takes an instance of the tyre as a dependency & returns an object of the Car type.

When you want to use drive a car, this is what your code will look like:

package main

import (
  "fmt"

  "service/parts"
  "service/vehicle"
)

func main() {
  // intialize the tyre object
  t := parts.NewTyre()

  // initialize the car
  c := vehicle.NewCar(t)

  // drive
  _ = c.Drive()
}

The biggest advantage of using this method is that we can mock all of our dependencies when we write unit tests. It also decouples one piece of code from another which makes debugging easier.

Now that you understand the idea behind the framework, let's jump into the folder structure in the following sections. We will dive deep into each package in further documents.

main.go

This as you might already know is the starting point of any golang app. The same is the case here. We try to keep main.go as small as possible & use it only to bootstrap dependencies and start the service components.

config

The config package as the name says handles the configuration of the app.

constants

The constants package holds app-wide constants in the package. By default it has 2 files:

  • app.go or common.go: This file is for common constants used in the app all across. If you don’t know where a constant should go, this is probably the best place for it.
  • errors.go: This file is for storing error types sent in the API response. To define an error constant, 2 things are needed:
    • Add an error constant in the const block with a variable name that correctly describes the error message
    • Add an error string for that variable in the ErrorString map. This error string is what will be returned as the error message in the API response

logger

The logger package is a straightforward package that is a wrapper over the https://github.com/sirupsen/logrus package. Nothing fancy here, you can use logger.Log.Info, logger.Log.Error, logger.Log.Warn etc just like you would use any logging library.

instance

The instance package is the singleton package & struct for handling use cases where a single instance should be created, for example, database connections, HTTP clients, queue connections etc. The Init method in the instance.go initializes all the singleton connections using [once.Do](https://golang.org/pkg/sync/#Once.Do).

So, let's say if you want to add support for another datastore in the service, you will need to initialize the connection in the Init method & expose the connection like the DB method present in the file.

One important method in the file is Destroy, which acts like a destructor for the instance. This will be called automatically when the app crashes or shuts down and should be used to close the connections.

utils

The utils package is the generic utility package that you might have seen in other codebases. Yes, we know that golang [doesn’t recommend](https://dave.cheney.net/2019/01/08/avoid-package-names-like-base-util-or-common) using a package called utils and we agree with that but there is just no better way we could think of which can have small utility methods.

ℹ️ Ask yourself this question before adding a new file to this package: Does this connect to any data source or is it going to be a one/two liner? If the answer is one/two-liner, then this is probably the right place for that method

db

The db package handles all db related declarations. The declaration is an important keyword here. We only declare database models & migrations in this package without defining their behavior (read queries). We do that in the repository package, described below.

The db package has 2 subfolders:

  • models: This package contains all the database models in the form of structs. Each model has a separate file with ORM declarations & Serializers/Deserializers
  • migrations: This folder contains all the database migrations for the Postgres database.

More information about this package & usage is here: https://creatorstack.atlassian.net/wiki/spaces/TD/pages/521568021

repository

The repository package handles the how of the models defined in the db package. Each model has a different repository file that defines the functionality of the model.

More information about this package & usage is here: https://creatorstack.atlassian.net/wiki/spaces/TD/pages/521568021

graph

The graph package holds the graphql schema & resolver declarations for the graphql API

More information about this package & usage is here: https://creatorstack.atlassian.net/wiki/spaces/TD/pages/521568036

runner

The runner package is a pretty important package in the framework. It starts all the processes that required to run a service. In 90% of the cases, you won’t have to deal with this package unless you are adding a new type of process, for ex, a new HTTP server, queueing system, etc.

api

The api package contains all the API services & business logic which is used to serve responses to the GraphQL/REST APIs.

More information about this package & usage is here: https://creatorstack.atlassian.net/wiki/spaces/TD/pages/521568036

queue

The queue package contains all the queue subscription & publishing business logic which is used for background processing

More information about this package & usage is here: https://creatorstack.atlassian.net/wiki/spaces/TD/pages/521568043

docker

The docker folder contains the docker configuration for running the service code.

More information about this folder & usage is here: https://creatorstack.atlassian.net/wiki/spaces/TD/pages/521502519

.vscode

The vscode folder is a special folder that contains vscode launch configurations for running & debugging the application code.

Clone this wiki locally