ARCHITECTURE.md

Architecture

• Updated: December 2021

This document describes the repository organization and the kustomize build process. It's meant to lower the barrier to learning and contributing to the code base.

If not kept up to date, it will just be a historical snapshot.

Repository layout

directory	purpose
api	The api module, holding high level kustomize code, suitable for import by other programs.
cmd	Various Go programs aiding repo management. See also hack. As an outlier, includes the special cmd/config module.
docs	Old home of documentation; contains pointers to new homes: human-edited docs, generated docs and rendered docs.
examples	Full kustomization examples that run as pre-merge tests.
functions	Examples of plugins in KRM function form. TODO(3922): Move under plugin.
hack	Various shell scripts to help with code management.
kustomize	The kustomize module holds the main.go for kustomize.
kyaml	The kyaml module, holding Kubernetes-specific YAML editing packages used by the api module. Wraps go-yaml v3.
plugin	Examples of Kustomize plugins.
releasing	Instructions for releasing the various modules.
site	Old generated documentation, kept to provide redirection links to the new docs.

Modules

The Go modules in the kustomize repository are semantically versioned.

kustomize

Depends on api, cmd/config, kyaml

The kustomize module contains the main.go for kustomize, buildable with

(cd kustomize; go install .)

Below this are packages containing cobra commands implementing build, edit, fix, etc., packages linked together by main.go.

These command packages are intentionally public, semantically versioned, and can be used in other programs. Specifically, the kustomize build command appears in kubectl as kubectl kustomize.

The code in the build package is dominated by flag validation, with minimal business logic. The critical lines are something like

# Make a kustomizer.
k := krusty.MakeKustomizer(
  HonorKustomizeFlags(krusty.MakeDefaultOptions()),
)

# Run the kustomizer, sending location of kustomization.yaml
m := k.Run(fSys, "/path/to/dir")

# Write the result as YAML.
writer.Write(m.AsYaml())

The krusty package is in the api module.

api

Depends on kyaml and code generated from builtin plugin modules

The api module is used by CLI programs like kustomize and kubectl to read and honor kustomization.yaml files and all that implies.

The main public packages in the api module are

package
filters	Implementations of kyaml/kio.Filter used by kustomize to transform Kubernetes objects.
konfig	Configuration methods and constants in the kustomize API.
krusty	Primary API entry point. Holds the kustomizer and hundreds of tests for it.
loader	Loads kustomization files and the files they refer to, enforcing security rules.
resmap	The primary internal data structure over which the kustomizer and filters work.
types	The Kustomization object and ancillary structs.

cmd/config

Depends on kyaml

This module contains cobra commands and kyaml-based functionality to provide unix-like file manipulation commands to kustomize like grep and tree. These commands may be included in any program that manipulates k8s YAML (e.g. kustomize).

kyaml

Has no in-repo dependence

The kyaml module is a kubernetes-focussed enhancement of [go-yaml].

The YAML manipulation performed by a kustomize is based on these libraries.

These libraries evolve independently of kustomize, and other programs depend on them.

The key public packages in the kyaml module include

package
errors	Wrapper for the go-errors/errors lib
filesys	A kustomize-specific file system abstraction, to ease writing tests
fn/framework	An SDK for writing KRM Functions in Go
fn/runtime	Implements the runtime for KRM Function extensions
kio	Libraries for reading and writing collections of Kubernetes resources as RNodes
openapi	Loads and accesses openapi schemas for schema-aware resource manipultaion
resid	Representations to aid in unique identification of Kubernetes resources
yaml	A Kubernetes-focused wrapper of [go-yaml], notably including the RNode object

---

How kustomize build works

The command kustomize build accepts a single string argument, which must resolve to a directory, possibly in a git repository, called the kustomization root.

This directory must contain a file called kustomization.yaml, with YAML that marshals into a single instance of a Kustomization object.

For the remainder of this document, the word kustomization refers to either of these things.

This kustomization is the access point to a directed, acyclic graph of Kubernetes objects, including other kustomizations, to include in a build.

Execution of build starts and ends in the api module, frequently dipping into the kyaml module for lower level YAML manipulation.

The build flow

• Validate command lines arguments and flags.

• Make a Kustomizer as a function of those arguments.

• Call Run on the kustomizer, passing it the path to the kustomization.

  Run returns an instance of ResMap, the api package's representation of a set of kubernetes Resource objects.

  This structure offers resource lookup methods (map behavior), but also retains the resources in the order they were specified in kustomization files (list behavior).

  Post-run, the objects are fully hydrated, per the instructions in the kustomization.

• Marshal the objects as YAML to a file or stdout.

The Run function

• Create various objects

  • A ResMap factory.

    Makes ResMaps from byte streams, other ResMaps, etc.

  • A file loader.Loader.

    It's fed an appropriate set of restrictions, and the path to the kustomization.

  • A plugin loader.

    It finds plugins (transformers, generators or validators) and prepares them for running.

  • A KustTarget encapsulating all of the above.

    A KustTarget contains one Kustomization and represents everything that kustomization can reach. This will include other KustTarget instances, each having a smaller purview than the one referencing it.

• Call KustTarget.Load to load its kustomization.

  This step deals with deprecations and field changes.

• Load openapi data specified by the kustomization.

  This is needed to recognize k8s kinds and their special properties, e.g. which kinds are cluster-scoped, which kinds refer to others, etc.

• Call KustTarget.makeCustomizedResmap to create the ResMap result.

  This visits everything referenced by the kustomization, performing all generation, transformation and validation.

• Finish the Run with

  • Optional reordering of objects in ResMap, overriding the FIFO rule.

  • Optional addition of kustomize build annotations to the resources. E.g. from which repo and file the resource was read, the fact that kustomize touched the resource, etc. These kustomize-specific annotations are intended for server-side data analytics, file structure traceability and reconstruction, etc.

The makeCustomizedResmap function

This function starts the process of object transformation, as well as accumulation of recursively referenced data.

• Call ra := KustTarget.AccumulateTarget.

  The result, ra, is a resource accumulator that contains everything referred to by the current kustomization, now fully hydrated.

• Uniquify names of generated objects by appending content hashes.

  This cannot be done until the objects are complete.

• Fix all name references (given that names may have changed).

  E.g. if a ConfigMaps was given a generated name, all objects that refer to that ConfigMap must be given its name.

• Resolve vars, replacing them with whatever they refer to (a legacy feature).

The AccumulateTarget function

• Call AccumulateResources over the resources field (this can recurse).
• Call AccumulateComponents over the components field (this can recurse),
• Load legacy (pre-plugin) global kustomize configuration,
• Load legacy (pre-openapi) Custom Resource Definition data.
• In the context of the data loaded above, run the kustomization's
  • generators,
  • transformers,
  • and validators.
• Accumulate vars (make note of them for later replacement).

AccumulateResources and component accumulation

• If the path is a file:

  • Accumulate the objects in the file (treating them as opaque kubernetes objects).

• If the path is a directory:

  • Create a new KustTarget referring to that directory's kustomization.
  • Call subRa := KustTarget.AccumulateTarget.
  • Call ra.MergeAccumulator(subRa) This completes a recursion.

• If the path is a git URL:

  • Clone the repository to a temporary directory.
  • Process the path optionally specified in the URL as a path in the clone.
  • If no path specified, work from the repository root.

That's as deep as this discussion will go.

The deeper this document goes into the details, the faster it will get out of date.