Generate-DockerImageVariants

Easily generate a repository populated with Docker image variants.

Agenda

It is very common to need to have multiple variants of a docker image. This module solves this problem by creating a simple templating framework to generate a full repository:

Dockerfile build contexts for variants
README.md containing the variants' image tags
Optional: Continuous integration (CI) configuration files to build and push variants' images to a docker registry

Here are some real repositories generated by using this module:

Install

Open powershell or pwsh and type:

Install-Module -Name Generate-DockerImageVariants -Repository PSGallery -Scope CurrentUser -Verbose

If prompted to trust the repository, hit Y and enter.

Usage

Initialize your repository

mkdir -p /path/to/my-project
cd /path/to/my-project

Initialize the ./generate folder

Import-Module Generate-DockerImageVariants
Generate-DockerImageVariants . -Init

Definition and template files are generated in the ./generate folder.

├── generate
│   ├── definitions
│   │   ├── FILES.ps1
│   │   └── VARIANTS.ps1
│   ├── functions
│   │   └── Download-Binary.ps1
│   └── templates
│       ├── Dockerfile.ps1
│       └── README.md.ps1

Edit the definitions and template files in ./generate (as needed)

Generate the variants:

Generate-DockerImageVariants .

Build contexts of variants are generated in ./variants.

The repository tree now looks like:

├── README.md
├── generate
│   ├── definitions
│   │   ├── FILES.ps1
│   │   └── VARIANTS.ps1
│   ├── functions
│   │   └── Download-Binary.ps1
│   └── templates
│       ├── Dockerfile.ps1
│       └── README.md.ps1
└── variants
    └── curl
        └── Dockerfile

Directory structure

A single folder named ./generate at the base of the repository will hold all the definition and template files:

├── generate
│   ├── definitions             # Definitions folder
│   │   ├── FILES.ps1           # An optional definition file for generation of repository files
│   │   └── VARIANTS.ps1        # Variant definition file for generation of variant build context
│   ├── functions               # Functions folder (optional). Useful for Dockerfile templating
│   │   └── Some-Function.ps1   # A function to use in your templates
│   │   └── Another-Function.ps1   # A function to use in your templates
│   └── templates               # Templates folder
│       ├── Dockerfile.ps1      # Dockerfile template (shared among variants across all distros)
│       └── README.md.ps1       # README.md template

Generation of a variant

At minimum, the ./generate/definitions/VARIANTS.ps1 definition file should contain the $VARIANTS definition like this:

$VARIANTS = @(
    @{
        tag = 'sometag'
        buildContextFiles = @{
            templates = @{
                'Dockerfile' = @{
                    common = $true
                    passes = @(
                        @{
                            variables = @{}
                        }
                    )
                }
            }
        }
    }
)

The FILES.ps1 definition file is optional, but if used, at minimum it should look like:

$FILES = @(
    # Paths are relative to the base of the project
    'README.md'
)

Upon generation, the sometag variant's build context is generated in ./variants/sometag with a file Dockerfile, as well as a file ./README.md, both relative to the base of the project.

├── README.md
└── variants
|   └── sometag
|       └── Dockerfile

See this basic example.

Using custom functions

To add a custom function, add a .ps1 file in the ./generate/functions. They will be dot-sourced and available in your templates. For instance, to add two functions Some-Function and Another-Function:

├── generate
│   ├── functions               # Functions is optional. Useful for Dockerfile templating
│   │   └── Some-Function.ps1   # A function to use in your templates
│   │   └── Another-Function.ps1   # A function to use in your templates

Then simply call them in your templates.

See this basic example with function called Download-Binary and Download-Binary2.

Generation of multiple variants

Suppose we want to generate two variants tagged curl and git:

$VARIANTS = @(
    @{
        tag = 'curl'
    }
    @{
        tag = 'git'
    }
)

# This is a optional variable that sets a common buildContextFiles for all variants
# Individual variant buildContextsFiles overrides this
$VARIANTS_SHARED = @{
    buildContextFiles = @{
        templates = @{
            'Dockerfile' = @{
                common = $true
                passes = @(
                    @{
                        variables = @{}
                    }
                )
            }
        }
    }
}

Upon generation, the curl and git variants' build contexts are generated:

└── variants
|   └── curl
|       └── Dockerfile
|   └── git
|       └── Dockerfile

See this example.

Generation of a variant through file copying

Populating a build context might not always involve processing templates. Sometimes we simply want to copy a file into the build context.

To copy a file, simply use the property copies in buildContextFiles:

$VARIANTS = @(
    @{
        tag = 'curl'
        buildContextFiles = @{
            copies = @(
                '/app'
            )
        }
    }
)

This will recursively copy all descending files/folders of the ./app folder located relative to the base of the parent repository into the to-be-generated curl variant's build directory ./variants/curl as ./variants/curl/app.

See this example.

Generation of a variant through multiple template passes

$VARIANTS = @(
    @{
        tag = 'curl'
        buildContextFiles = @{
            templates = @{
                'Dockerfile' = @{
                    common = $false
                    passes = @(
                        # The first pass will generate 'Dockerfile'
                        @{
                            variables = @{
                                'foo' = 'bar'
                            }
                        }
                        # The second pass will generate 'Dockerfile.dev'
                        @{
                            variables = @{
                                'foo2' = 'bar2'
                            }
                            generatedFileNameOverride = 'Dockerfile.dev'
                        }
                    )
                }
            }
        }
    }
)

During generation, in addition to the $VARIANT object, a $PASS_VARIABLES hashtable will be in the scope of the processed Dockerfile.ps1 template. In the first pass, the value of $PASS_VARIABLES['foo'] will be bar, and the file Dockerfile will be generated in the variant's build context. In the second pass, the value of $PASS_VARIABLES['foo2'] will be bar2, and the file Dockerfile.dev will be generated in the same build context.

See this example for using multiple passes with variables.

Generation of a single variant's build context file(s) using Component-chaining

When a variant's tag contains words delimited by -, it is known as Component-chaining. The final generated file will be a concatanation of the product of processing the template of each component specified in this chain.

For instance, suppose you want a variant Dockerfile that installs curl and git:

$VARIANTS = @(
    @{
        tag = 'curl-git'
        buildContextFiles = @{
            templates = @{
                'Dockerfile' = @{
                    common = $true
                    passes = @(
                        @{
                            variables = @{}
                        }
                    )
                }
            }
        }
    }
)

The template pass to generate the variant's build context Dockerfile proceeds as such:

The template ./generate/templates/Dockerfile.ps1 is processed
The build context: ./variants/curl-git is generated with the file ./variants/curl-git/Dockerfile

See this example.

Generation of multiple variants' build context file(s) using Component-chaining

$VARIANTS = @(
    @{
        tag = 'curl-git'
    }
    @{
        tag = 'curl'
    }
    @{
        tag = 'git'
    }
)

# This is a special optional variable that sets a common buildContextFiles definition for all variants
$VARIANTS_SHARED = @{
    buildContextFiles = @{
        templates = @{
            'Dockerfile' = @{
                common = $false
                passes = @(
                    @{
                        variables = @{}
                    }
                )
            }
        }
    }
}

Upon generation, three variants build contexts for variants curl-git, curl, and git are generated:

└── variants
|   └── curl-git
|       └── Dockerfile
|   └── curl
|       └── Dockerfile
|   └── git
|       └── Dockerfile

See this example.

To specify that only certain components be processed, independent of the tag property, ensure to define the components property. See these examples:

/docs/examples/basic-custom-components example.
/docs/examples/basic-custom-components-distro example.

Note if using distro:

If distro is non-empty, and if the variant's tag consist of a word that matches the variant's distro, distro will not be among the components.** For instance, in the above example, if the tag is curl-git-alpine and the distro is alpine, there will still only be two components curl and git. alpine will not be considered a component.

Optional: Generate other repository files

To generate files other than variant build contexts in ./variants, define them in FILES.ps1.

Suppose we want to generate README.md:

$FILES = @(
    'README.md'
)

Then, create their templates in the ./generate/templates directory:

├── generate
│   └── templates
│       └── README.md.ps1       # README.md template

Upon generation, README.md is now generated:

└── README.md

The variables $VARIANTS will be available during the processing of the template files.

See this example.

Appendix

Variant object properties

A $VARIANT definition will contain these properties.

The buildContextFiles property of the $VARIANT object can be used to customize the template processing:

common - (Optional, defaults to $false) Specifies whether this file is shared by all distros. If value is $true, template has to be present in ./generate/templates/<file>.ps1. If value is $false, and if a variant distro is defined, template has to be present in ./generate/templates/<file>/<distro>/, or if a variant distro is omitted, template has to be present in ./generate/templates/<file>/<file>.ps1.
includeHeader - (Optional, defaults to $false) Specifies to process a template <file>.header.ps1. Template path determined by common
includeFooter - (Optional, defaults to $false) Specifies to process a template <file>.footer.ps1. Template path determined by common
passes - (Mandatory) An array of pass definitions that the template will undergo. Each pass will generate a single file.

Each template pass processes a template file <file>.ps1 template and generates a single file named <file> in the variant's build context.

A pass can be configured with the variables and generatedFileNameOverride properties.

$VARIANT = @{
    # Specifies the docker image tag
    # When the tag contains words delimited by '-', it known as component-chaining.
    tag = 'some-cool-tag'

    # Specifies a distro (optional). If you dont define a distro, templates will be sourced from ./generate/templates/<file> folder
    # In contrast, if a distro is specified, templates will be sourced from ./generate/templates/<file>/<distro> folder
    distro = 'somedistro'

    # Specifies that this variant should be tagged ':latest'. This property will be useful in generation of content in README.md or ci files. Automatically populated as $false if unspecified
    tag_as_latest = $false

    # Automatically populated
    tag_without_distro = 'somecomponent1-somecomponent2'

    # Specifies an list of components, independent of the `tag` property
    # If unspecified, this is automatically populated based on the components in the tag
    # If specified, the components override those specified in the tag
    components = @(
        'somecomponent1'
        'somecomponent2'
    )

    # Automatically populated
    build_dir_rel = './variants/distro/<tag_without_distro>'

    # Automatically populated
    build_dir = '/full/path/to/variants/distro/<tag_without_distro>'

    # Build context template definition
    buildContextFiles = @{
        templates = @{
            'Dockerfile' = @{
                # Specifies whether the template is common (shared) across distros
                common = $false

                # Specifies whether the template <file>.header.ps1 will be processed. Useful for Dockerfiles
                includeHeader = $true

                # Specifies whether the template <file>.footer.ps1 will be processed. Useful for Dockerfiles
                includeFooter = $true

                # Specifies a list of passes the template will be undergo, where each pass generates a file
                passes = @(
                    # This first pass generates the file called 'Dockerfile'
                    @{
                        # These variables will be available in $PASS_VARIABLES hashtable when this template is processed
                        variables = @{
                            foo = 'bar'
                        }

                        # If this is uncommented, the pass will generate the file named 'Dockerfile.dev' instead
                        # generatedFileNameOverride = 'Dockerfile.dev'
                    }
                )
            }
        }
        # Specifies the paths, relative to the root of the repository, to recursively copy into each variant's build context
        copies = @(
            '/app'
        )
    }
}

Debugging

Use the -Verbose switch. This gives a detail trace of:

Validation of $VARIANTS definition
Validation of $FILES definition
Template files or to-be-copied repository files for the build context generation
Template files for repository files generation

This is particularly useful when the module is throwing errors about definitions, or about missing template or to-be-copied files.

Validation of `$VARIANTS` and `$FILES` definitions

For instance, if a variant was defined with an incorrect type (expected to be hashtable):

$VARIANTS = @(
    1
    @{
        tag = 'curl'
        distro = 'alpine'
    }
)

Example of a validation trace:

PS > cd /path/to/my-repo
PS > Generate-DockerImageVariants . -Verbose
VERBOSE: Validating $VARIANTS definition
VERBOSE: Validating TargetObject '1 System.Collections.Hashtable System.Collections.Hashtable System.Collections.Hashtable System.Collections.Hashtable System.Collections.Hashtable System.Collections.Hashtable' of type 'Object[]' and basetype 'array'       against Prototype 'System.Collections.Hashtable' of type 'Object[]' and basetype 'array'
VERBOSE:        Validating TargetObject '1' of type 'Int32' and basetype 'System.ValueType'             against Prototype 'System.Collections.Hashtable' of type 'Hashtable' and basetype 'System.Object'
WARNING: Failed with errors. Exception: Type System.Int32 is invalid! It should be of type 'System.Collections.Hashtable'.

This demonstrates that a variant definition has to be of type hashtable. The value 1 is of type int32, and hence is invalid.

Validation of template files or to-be-copied files

Example of a validation trace:

PS > cd /path/to/my-repo
PS > Generate-DockerImageVariants . -Init
PS > Generate-DockerImageVariants . -Verbose
VERBOSE: Validating $VARIANTS definition
...
VERBOSE: Validating $FILES definition
...
Generating build context of variant 'curl': /path/to/my-repo/variants/curl
VERBOSE: Generating build context file: /path/to/my-repo/variants/curl/Dockerfile
VERBOSE: Processing template file: /path/to/my-repo/generate/templates/Dockerfile.ps1
Generating build context of variant 'curl-git': /path/to/my-repo/variants/curl-git
VERBOSE: Generating build context file: /path/to/my-repo/variants/curl-git/Dockerfile
VERBOSE: Processing template file: /path/to/my-repo/generate/templates/Dockerfile.ps1
Generating build context of variant 'my-cool-variant': /path/to/my-repo/variants/my-cool-variant
VERBOSE: Generating build context file: /path/to/my-repo/variants/my-cool-variant/Dockerfile
VERBOSE: Processing template file: /path/to/my-repo/generate/templates/Dockerfile.ps1
Generating repository file: /path/to/my-repo/README.md
VERBOSE: Processing template file: /path/to/my-repo/generate/templates/README.md.ps1

Name		Name	Last commit message	Last commit date
Latest commit History 237 Commits
.github		.github
.vscode		.vscode
build		build
docs/examples		docs/examples
src/Generate-DockerImageVariants		src/Generate-DockerImageVariants
test		test
.gitignore		.gitignore
.gitmodules		.gitmodules
LICENSE		LICENSE
README.md		README.md

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Generate-DockerImageVariants

Agenda

Install

Usage

Directory structure

Generation of a variant

Using custom functions

Generation of multiple variants

Generation of a variant through file copying

Generation of a variant through multiple template passes

Generation of a single variant's build context file(s) using Component-chaining

Generation of multiple variants' build context file(s) using Component-chaining

Optional: Generate other repository files

Appendix

Variant object properties

Debugging

Validation of `$VARIANTS` and `$FILES` definitions

Validation of template files or to-be-copied files

About

Releases 17

Packages

Languages

License

theohbrothers/Generate-DockerImageVariants

Folders and files

Latest commit

History

Repository files navigation

Generate-DockerImageVariants

Agenda

Install

Usage

Directory structure

Generation of a variant

Using custom functions

Generation of multiple variants

Generation of a variant through file copying

Generation of a variant through multiple template passes

Generation of a single variant's build context file(s) using Component-chaining

Generation of multiple variants' build context file(s) using Component-chaining

Optional: Generate other repository files

Appendix

Variant object properties

Debugging

Validation of $VARIANTS and $FILES definitions

Validation of template files or to-be-copied files

About

Topics

Resources

License

Stars

Watchers

Forks

Releases 17

Packages 0

Languages

Validation of `$VARIANTS` and `$FILES` definitions

Packages