The primary units this build system deals with are targets: the user requests the system to build (or install) a target, targets depend on other targets, etc. Targets typically reflect the units a software developer thinks in: libraries, binaries, etc. The definition of a target only describes the information directly belonging to the target, e.g., its source, private and public header files, and its direct dependencies. Any other information needed to build a target (like the public header files of an indirect dependency) are inferred by the build tool. In this way, the build description can be kept maintainable
A built target consists of files logically belonging together (like the actual library file and its public headers) as well as information on how to use the target (linking arguments, transitive header files, etc). For a consumer of a target, the definition of this collection of files as well as the additionally provided information is what defines the target as a dependency, respectively of where the target is coming from (i.e., targets coinciding here are indistinguishable for other targets).
Of course, to actually build a single target from its dependencies, many invocations of the compiler or other tools are necessary (so called "actions"); the build tool translates these high level description into the individual actions necessary and only re-executes those where inputs have changed.
This translation of high-level concepts into individual actions is not hard coded into the tool. It is provided by the user as "rules" and forms additional input to the build. To avoid duplicate work, rules are typically maintained centrally for a project or an organization.
The code base is usually split into many directories, each containing source files belonging together. To allow the definition of targets where their code is, the targets are structured in a similar way. For each directory, there can be a targets files. Directories for which such a targets file exists are called "modules". Each file belongs to the module that is closest when searching upwards in the directory tree. The targets file of a module defines the targets formed from the source files belonging to this module.
Larger projects are often split into "repositories". For this build tool, a repository is a logical unit. Often those coincide with the repositories in the sense of version control. This, however, does not have to be the case. Also, from one directory in the file system many repositories can be formed that might differ in the rules used, targets defined, or binding of their dependencies.
A peculiarity of this build system is the complete separation between physical and logical paths. Targets have their own view of the world, i.e., they can place their artifacts at any logical path they like, and this is how they look to other targets. It is up to the consuming targets what they do with artifacts of the targets they depend on; in particular, they are not obliged to leave them at the logical location their dependency put them.
When such a collection of artifacts at logical locations (often referred to as the "stage") is realized on the file system (when installing a target, or as inputs to actions), the paths are interpreted as paths relative to the respective root (installation or action directory).
This separation is what allows flexible combination of targets from various sources without leaking repository names or different file arrangement if a target is in the "main" repository.
A repository uses a (logical) directory for several purposes: to obtain source files, to read definitions of targets, to read rules, and to read expressions that can be used by rules. While all those directories can (and often are) be the same, this does not have to be the case. For each of those purposes, a different logical directory (also called "root") can be used. In this way, one can, e.g., add target definitions to a source tree originally written for a different build tool without modifying the original source tree.
Those roots are usually defined in a repository configuration. For the
"main" repository, i.e., the repository from which the target to be
built is requested, the roots can also be overwritten at the command
line. Roots can be defined as paths in the file system, but also as
git tree identifiers (together with the location of some repository
containing that tree). The latter definition is preferable for rules and
dependencies, as it allows high-level caching of targets. It also
motivates the need of adding target definitions without changing the
root itself.
The same flexibility as for the roots is also present for the names of
the files defining targets, rules, and expressions. While the default
names TARGETS, RULES, and EXPRESSIONS are often used, other file
names can be specified for those as well, either in the repository
configuration or (for the main repository) on the command line.
The final piece of data needed to describe a repository is the binding of the open repository names that are used to refer to other repositories. More details can be found in the documentation on multi-repository builds.
In description files, targets, rules, and expressions are referred to by name. As the context always fixes if a name for a target, rule, or expression is expected, they use the same naming scheme.
- A single string refers to the target with this name in the same module.
- A pair
[module, name]refers to the targetnamein the modulemoduleof the same repository. There are no module names with a distinguished meaning. The naming scheme is unambiguous, as all other names given by lists have length at least 3. - A list
["./", relative-module-path, name]refers to a target with the given name in the module that has the specified path relative to the current module (in the current repository). - A list
["@", repository, module, name]refers to the target with the specified name in the specified module of the specified repository.
Additionally, there are special targets that can also be referred to in target files.
- An explicit reference of a source-file target in the same module,
specified as
["FILE", null, name]. The explicitnullat the second position (where normally the module would be) is necessary to ensure the name has length more than 2 to distinguish it from a reference to the module"FILE". - An explicit reference of a non-upwards symlink target in the same module,
specified as
["SYMLINK", null, name]. The explicitnullat the second position is required for the same reason as in the explicit file reference. It is the user's responsibility to ensure the symlink pointed to is non-upwards. - A reference to an collection, given by a shell pattern, of explicit
source files in the top-level directory of the same module,
specified as
["GLOB", null, pattern]. The explicitnullat second position is required for the same reason as in the explicit file reference. - A reference to a tree target in the same module, specified as
["TREE", null, name]. The explicitnullat second position is required for the same reason as in the explicit file reference.
Analyzing a target results in 3 pieces of data.
-
The "artifacts" are a staged collection of artifacts. Typically, these are what is normally considered the main reason to build a target, e.g., the actual library file in case of a library.
-
The "runfiles" are another staged collection of artifacts. Typically, these are files that directly belong to the target and are somehow needed to use the target. For example, in case of a library that would be the public header files of the library itself.
-
A "provides" map with additional information the target wants to provide to its consumers. The data contained in that map can also contain additional artifacts. Typically, this the remaining information needed to use the target in a build.
In case of a library, that typically would include any other libraries this library transitively depends upon (a stage), the correct linking order (a list of strings), and the public headers of the transitive dependencies (another stage).
A target is completely determined by these 3 pieces of data. A consumer of the target will have no other information available. Hence it is crucial, that everything (apart from artifacts and runfiles) needed to build against that target is contained in the provides map.
When the installation of a target is requested on the command line, artifacts and runfiles are installed; in case of staging conflicts, artifacts take precedence.
If a target is not found in the targets file, it is implicitly treated as a source file. Both, explicit and implicit source files look the same. The artifacts stage has a single entry: the path is the relative path of the file to the module root and the value the file artifact located at the specified location. The runfiles are the same as the artifacts and the provides map is empty.
To ensure self-containedness and location-independence, only non-upwards symlinks are expected and accepted. The symlinks must not however be necessarily resolvable, i.e., dangling symlinks are accepted.
An explicit (non-upwards) symlink target is similar to an explicit file target, except that at the specified location there has to be a non-upwards symlink rather than a file and the corresponding symlink artifact is taken instead of a file artifact.
A collection of files given by a shell pattern has, both as artifacts and runfiles, the (necessarily disjoint) union of the artifact maps of the (zero or more) source targets that match the pattern. Only files in the top-level directory of the given modules are considered for matches. The provides map is empty.
A tree describes a directory. Internally, however, it is a single opaque artifact. Consuming targets cannot look into the internal structure of that tree. Only when realized in the file system (when installation is requested or as part of the input to an action), the directory structure is visible again.
An explicit tree target is similar to an explicit file target, except that at the specified location there has to be a directory rather than a file and the tree artifact corresponding to that directory is taken instead of a file artifact.