Build System

Pilot Light Build

The pl-build tool is a lightweight utility used to generate batch/bash build scripts.

Information • Documentation • API

Information

Background

The pl-build tool is a part of the larger Pilot Light project. In this larger project, we do not have a "build system" per se. Instead we prefer to write batch/bash scripts that directly call the compiler (in a manner similar to Casey Muratori's Handmade Hero). If this project was an end user product, this would be the end of it. However, this is not the case. It is meant to be easily extended through adding additional extensions and being used as a "pilot light" to start new projects. With this comes a couple issues. Extensions are meant to be cross platform, so users need the ability to easily add new binaries for all target platforms with minimal duplication. Users shouldn't need to be bash or batch scripting experts to build new targets for all platforms and shouldn't need to test the build scripts continuously on each platform.

Another way of putting it, is we want to focus on what matters to build binaries. Ultimately this is just compiler & linker settings. We don't want to think about the differences in bash/batch syntax.

The idea is simple:

flowchart LR
    gen_build.py --> build_win32.bat
    gen_build.py --> build_linux.sh
    gen_build.py --> build_macos.sh

Loading

Features

• entire system can be understood in an hour
• minimizes duplicated information
• generates standalone simple build scripts
• fine-grained control over compilation & linker settings
• supports hot reloading
• easily extended to add new platforms & compilers
• extremely light weight
• no preference on editor/IDE
• doesn't pretend different platforms don't exist

High Level

The idea here is to create a single build script in python that is used to generate build scripts for different platforms/compilers.

Typical build scripts follow a similar pattern:

1. import core & backends
2. specify project
3. specify targets
4. specify configurations
5. specify platforms
6. specify compilers
7. apply settings
8. generate build scripts using backends

In code:

import os
import pl_build.core as pl
import pl_build.some_backend as backend

with pl.project("project name"):

    # add project only settings here
   
    # add profiles here

    # add settings shared by all targets here

    with pl.target("target name", pl.TargetType.EXECUTABLE, False):

        # add settings shared by all this target's configurations here

        with pl.configuration("configuration name"):

            # add settings shared by all this configuration's platforms

            with pl.platform("platform name"):

                # add settings shared by all this platform's compilers

                with pl.compiler("compiler name"):
                    
                    # add settings

working_directory = os.path.dirname(os.path.abspath(__file__)) # where this python file is
backend.generate_build(working_directory + '/' + "build.sh")

It is worth noting there is a hierarchy relationship that is made obvious through the use of python context managers. This is also used to allow inheriting many settings if the settings are common across lower scopes. For example, if a source file main.c is used by all configurations, platforms, and compilers for a target, you can use add_source_files("main.c") once at the target scope.

API

General Settings

The following settings can be set at any scope:

$${\color{rgb(180, 55, 76)}set\_output\_directory(directory)}$$

Sets the output directory of the final build script.

Example:

pl.set_output_directory("../out")

$${\color{rgb(180, 55, 76)}set\_output\_binary(binary)}$$

Sets the name of the output binary (do not include extension).

Example:

pl.set_output_binary("pilot_light")

$${\color{rgb(180, 55, 76)}set\_output\_binary\_extension(extension)}$$

Sets the output binary extension. If this is not set, the backend chooses an appropriate default:

For executables:

• Windows -> exe
• MacOS/Linux-> no extension

For dynamic libraries:

• Windows -> dll
• Linux -> so
• MacOS -> dylib

For static libraries:

• Windows -> lib
• Linux -> a
• MacOS -> a

Example:

pl.set_output_binary_extension("dll")

$${\color{rgb(180, 55, 76)}add\_source\_files(*files)}$$

Adds source files.

Examples:

pl.add_source_files("lib.c")
pl.add_source_files("../extensions/pl_extensions.c", "main.c")

$${\color{rgb(180, 55, 76)}add\_static\_link\_libraries(*libraries)}$$

Adds static link libraries.

The backend will handle appropriate flags for compiler/linker and other things like prepending "lib" for clang.

Example:

pl.add_static_link_libraries("pthread")

$${\color{rgb(180, 55, 76)}add\_dynamic\_link\_libraries(*libraries)}$$

Adds dynamic link libraries. The backend will handle appropriate flags for compiler/linker among other things.

Example:

pl.add_dynamic_link_libraries("pthread")

$${\color{rgb(180, 55, 76)}add\_link\_frameworks(*frameworks)}$$

Adds frameworks for MacOS.

The backend will handle appropriate flags for compiler/linker among other things.

Example:

pl.add_link_frameworks("Metal", "MetalKit", "Cocoa")

$${\color{rgb(180, 55, 76)}add\_definitions(*definitions)}$$

Adds preprocessor definitions. The backend will handle flags.

Example:

pl.add_definitions("PL_METAL_BACKEND")

$${\color{rgb(180, 55, 76)}add\_compiler\_flags(*flags)}$$

Adds compiler flags. The backend does **NOT** handle flags. That is your responsibility.

Examples:

pl.add_compiler_flags("-Wno-deprecated-declarations")
pl.add_compiler_flags("-std=c99", "-fmodules", "-ObjC", "-fPIC")

$${\color{rgb(180, 55, 76)}add\_linker\_flags(*flags)}$$

Adds linker flags. The backend does **NOT** handle flags. That is your responsibility.

Examples:

pl.add_linker_flags("-incremental:no")
pl.add_linker_flags("-ldl", "-lm")
pl.add_linker_flags("-Wl,-rpath,/usr/local/lib")

$${\color{rgb(180, 55, 76)}add\_include\_directories(*directories)}$$

Adds include directories. The backend will handle flags.

Example:

pl.add_include_directories("../examples", "../src")

$${\color{rgb(180, 55, 76)}add\_link\_directories(*directories)}$$

Adds link directories. The backend will handle flags.

Example:

pl.add_link_directories("../out")

Project Scope Only

The following settings can only be set at the project scope:

$${\color{rgb(180, 55, 76)}set\_hot\_reload\_target(target)}$$

When specified, only targets that are specified as hot reloadable will rebuild when the hot reload target is still running.

Example:

pl.set_hot_reload_target("../out/pilot_light_test")

$${\color{rgb(180, 55, 76)}add\_profile(...)}$$

Filter Arguments:

• compiler_filter -> list of compilers or None for all compilers
• platform_filter -> list of platforms or None for all platforms
• configuration_filter -> list of configurations or None for all configurations
• target_filter -> list of targets or None for all targets
• target_type_filter -> list of target types or None for all target types

Setting Arguments:

• definitions -> list of definitions
• include_directories -> list of include directories
• link_directories -> list of link directories
• static_link_libraries -> list of static link libraries
• dynamic_link_libraries -> list of dynamic link libraries
• link_frameworks -> list of link frameworks
• source_files -> list of source files
• compiler_flags -> list of compiler flags
• linker_flags -> list of linker flags
• output_directory -> output directory
• output_binary_extension -> output binary extension

This function adds settings if the current compiler/platform/configuration/target/target type either matches on of the filters or if the filter in that category is set to None.

Examples

# add compiler flags to anything that is using both "msvc" compiler
# and "debug" configuration
pl.add_profile(compiler_filter=["msvc"],
                    configuration_filter=["debug"],
                    compiler_flags=["-Od", "-MDd", "-Zi"])

# add definitions to anything that is either "msvc" or "gcc" compilers and "debug"
# or "release" configuration
pl.add_profile(configuration_filter=["debug", "release"], compiler_filter=["gcc", "msvc"],
                    definitions=["PL_VULKAN_BACKEND"])

Compiler Scope Only

The following settings can only be set at the compiler scope:

$${\color{rgb(180, 55, 76)}set\_pre\_target\_build\_step(code)}$$

Inserts code before the current build step.

$${\color{rgb(180, 55, 76)}set\_post\_target\_build\_step(code)}$$

Inserts code after the current build step.

Documentation

Projects

Projects are the root scope used to contain all targets of the build.

Targets

Targets represent an actual binary produced by the build. The context manager has 2 arguments for specifying the target type and whether or not the target is hot reloadable.

The following types of targets are:

• TargetType.EXECUTABLE (default)
• TargetType.STATIC_LIBRARY
• TargetType.DYNAMIC_LIBRARY

If the target is specified as hot reloadable, it will rebuild even if the hot reload target is still running. If the target is not hot reloadable, it will only rebuild if the hot reload target is not running.

Configurations

Configurations are used to specify different settings based on a configuration name. For example, configurations can be used to specify a release or debug build.

Configurations are controlled in the final build script like so:

./build_linux.sh # uses first config found by default
./build_linux.sh -c debug # uses 'debug' config

Note

The first configuration encountered will be the default configuration.

Platforms

Platforms are used to specify different settings & compilers based on a platform. This is used by the backends to generate the correct scripts. The current backends support the following platform types:

• "Windows"
• "Linux"
• "Darwin"

Compilers

Compilers are used to specify different settings based on a compiler. This is used by the backends to generate the correct scripts. The current backends support the following compiler types:

• "msvc"
• "gcc"
• "clang"

Note

The current backends each only support a single backend (Windows supports msvc, MacOS supports clang, and Linux supports gcc). We will add further support to the backends soon.

Scopes

The context managers for projects, targets, configurations, and platforms form a hierarchy of scopes. Settings applied at a higher scope are inherited by lower scopes. Taking advantage of this feature reduces code duplication significantly and is encouraged. For example if a specific target has the source file main.c shared across all configurations, platforms, and compilers, you can just use add_source_files("main.c") once at the target scope instead of at each compiler scope (the lowest scope).

Profiles

In the same spirit of Scopes, Profiles aim to reduce code duplication. Instead of being scope based, Profiles are filter based. You can apply settings across a project to anything that matches the filters. You can filter based on compiler, platform, configuration, target, and target type.

For example, if you wanted to apply a certain set of compiler and linker flags to anything using the "gcc" compiler and "debug" configuration you could do that:

pl.add_profile(configuration_filter=["debug"], compiler_filter=["gcc"],
                    compiler_flags=["-std=gnu11", "-fPIC"],
                    linker_flags=["-ldl", "-lm"])

Note

Setting a filter to None ignores that filter

Backends

Under construction.