Variables for template repositories

[mwarkentin]

Migrating from isaacs/github#1716

Now that github has template repositories, an obvious next step is to enable "variables" to make it easy to stamp out customized repositories based on a set of metadata passed in to the template.

See https://cookiecutter.readthedocs.io for an example of this in a command line tool.

Found someone doing something similar with Github Actions: https://github.com/rochacbruno/python-project-template/blob/main/.github/workflows/rename_project.yml

[simonw]

I would love this. I wrote about a workaround for running cookiecutter against a fresh copied template repository here, inspired by Bruno's rename_project.yml https://simonwillison.net/2021/Aug/28/dynamic-github-repository-templates/

[rochacbruno]

This feature would be really awesome

I created 2 templates

https://github.com/rochacbruno/python-project-template
and
https://github.com/rochacbruno/flask-project-template

The only values I can currently read via a hacky workaround is the username, repo name, repo description.

If I had more variables I would have created different options for the main template:

template_model=[flask,django,fastapi,click,...]

I suggest GitHub to allow template creators to place a .github/template.yml to define inputs using the same structure we have for actions.

Once the user clicks on "use this template" github redirects to the actions page with the workflow input form to be filled, then once completed it runs the workflow defined in the same template.yml file.

[spenserblack]

In addition to inputs, I think .github/template.yml should allow a user to define the brackets that are used around template variables.

Parsing in variables in places that look like {{ this }} might work most of the time, but it becomes a problem if your template has templates. For example, a template Django project could have an HTML template that looks like this:

<p>Hello, {{ name }}!</p>

And you would not want that {{ name }} to be replaced when you generate a new template from the repository.

So I'd suggest a .github/template.yml file that looks like this (since I'm already writing out an example template.yml, I figured I might as well write out example inputs:, too):

inputs:
  - name: project_name
    label: Project Name
    type: text
  - name: indentation_style
    label: Preferred indentation style
    default: spaces
    type: select
    choices:
      - tabs
      - spaces

delimiters: ["${", "}$"]  # default: ["{{", "}}"]

Edit: And, of course, .github/template.yml should always be exempt from any modifications caused by variable interpolation.

[george-gca]

The same happens for jekyll, which uses the same for variables.

[ericbn]

And with the possibly of having global templates in the organization/.github repo, it would be really helpful to have template variables containing values for the current repository in the same organization where the template is being used.

E.g. I'd love to have something like this in my bug_report.md template:

- [ ] I've checked the [existing issues](https://github.com/organization/{{repository.name}}/issues) and I'm not duplicating a report.

EDIT: This is a different scenario. With global templates in organization/.github, you don't even need to duplicate the templates coming from a template repository when creating a new repository. This is how it looks like: https://github.com/zimfw/.github/tree/master/.github

[wyhinton]

Template variables would be a huge help for our application, Cupboard.

Ideally, we'd be able to use template variables in combination with actions to enable the following:

1. User hits "Use This Template"
2. User is provided with prompts or an email notification to specify environment variables, like REACT_APP_SHEET_URL to specify a google sheet to use as a backend. If the user does not provide, then a default value is provided.
3. An action is triggered which enables GitHub pages for the repo in the gh-pages branch, and then builds and deploys to https://{repo.username}.github.io/cuboard/
4. The README.md is updated with badges for these actions.
   <img src = "https://github.com/{repo.username}/{repo.name}/actions/workflows/ci.js.yml/badge.svg">

Template variables would dramatically boost the capabilities of the templating feature.

[hughetop]

I'm honestly completely surprised this isn't already a feature. I'm having a hard time seeing the point of templates if it doesn't incorporate some basic string replacements.

[verdantburrito]

Agreed. The utility of GitHub template repositories is pretty low, without support for variables & using them to transform the new repository you're creating.

[jezen]

To be fair, and to the credit of GitHub's staff, the feature as it is already saves me time. Otherwise I have to spend time taking an existing repository and stripping away the things that I don't need.

[tiarebalbi]

This is kind of a pre-requirement to support templates. Would make a such different

[loreanvictor]

I was in need of this (making multiple repos from a template), so I built a simple version that'll run a nice CLI tool to fill in the blanks and prepare the template. It can also work with GitHub Actions if all the variables you need to fill-in the template are present in Action Contexts (but you can do the same with cookiecutter or any custom script of your own, I feel this might be just a bit simpler).

[john-michael-murphy]

This would be a hugely helpful feature.

[saravsak]

This would be extremely helpful to enforce consistency across all repositories within our org.

[morgzz]

+1 Yes please

[simkimsia]

Hi everyone, my name is Kimsia and I'm an independent web dev.

My day job is that I build Purchase Order processing web app for businesses as a service. My customers include a Fortune 500 company.

I have a web app that can help with creating a GitHub repo from a template repository and allows variables. It requires you to login using your GitHub oAuth and allow the public_repo scope.

Anyone interested to do a beta test, can ping me here.

Update: I am done with an alpha version. See the demo videos at this comment

It targets cookiecutter as the template syntax to build for. If interest is high, can consider other template syntax as well. Cookiecutter is the only one I am familiar with.

Right now, my perfectionist streak is trying to figure out 2 other things before I make it more public:

1. I am trying to figure out the best way in terms of permissions (wouldn't want to ask for more than needed) and GitHub docs aren't the best at explaining this, but I did ask in the discussions. Hopefully, I get back some reply.
2. I am trying to figure out a way to auto convert the user config in cookiecutter yaml file into a form for users to fill up before triggering the repo creation

Limitations when public

1. can only work with GitHub
2. you need to grant the public_repo scope when install with GitHub OAuth
3. can only work with Cookiecutter syntax
4. the repo holding the template must be public
5. can only generate public repos as output for now

Two nice things over GitHub Template Repo

1. You can use variables
2. The target template to generate from can be in a subfolder of a specific branch of the repo. It need not be the whole repo.

Again, anyone interested, can ping me here.

Thank you

[hughetop]

I think a lot of people end up doing something similar. I wrote a basic one as a command-line utility to create rez packages, which requires an existing repository. So it has a few extra steps, but it's better than nothing.
To clarify, I create an "empty" repository with just the .gitignore and README, clone that, cd into to the repo directory then run the utility. It sets up 95% of the rest of the repo based on the command-line arguments. Then I do a commit (which could probably be automated).

[simkimsia]

Hey Topher, thank you for reply.

Do you have a tutorial or example? I want to see how other people do it.

I noticed many people's similar solution is to build a command line utility like you said. Maybe a web app is a bad idea. What do you think?

[hughetop]

I can't share the tool itself but I can give a description of how it works. This is pretty specific for python packages using rez as the package manager. The command-line tool will ask the user for the package name, author (defaulting to the current username), description, and year (defaulting to the current year, used for licensing boilerplates). It then copies a directory structure, first doing string replacements on the file names (so far only the python package directory needs this), then iterating over every file and doing string replacements on the file contents. What I end up with is something like this:

example-package/
    .gitignore
    README.md
    package.py
    bin/
    python/
        example_package/
            __init__.py

The majority of string replacements happen in package.py, which in this example might look like:

name = 'example_package'
version = '0.1.0'
authors = ['hughetop']
description = 'This is a package template example'
requires = ['python-3']  # python-3 is always added as a requirement

def commands():
    env.PYTHONPATH.append('{root}/python')
    # env.PATH.prepend('{root}/bin')  # uncomment if you add any executables to bin

A note about python string replace: the template had to be env.PYTHONPATH.append('{{root}}/python') so that it didn't think root was a key to replace.

Edit: to answer your question, I think a web app could work, but it was just easier for me to set up a command-line tool.

[simkimsia]

Thank you for the detailed breakdown, Topher. It made me think.

I think a web app could work, but it was just easier for me to set up a command-line tool.

Thank you for this.. I also just got a response from the other discussion thread. I guess I just ask for the user's public_repo scope via oAuth and focus on getting the cookiecutter user config and reflect it back as a form.

[simkimsia]

Hi everyone, my name is Kimsia and previously I wrote a comment 2 weeks ago here mentioning that I was halfway with a web version that helps people create a repo from a template with variables.

Today, I am done and I have 4 loom videos explaining how it all works.

I'm calling it GreenDeploy.

• Conceptual Overview of How GreenDeploy works
• How to Install and Permissions Overview
• UseCase 1: Direct from the Template Repo + Your GitHub Username
• UseCase 2: From Specific Branch & Path + Your GitHub Org

The next thing I am going to do is go through each comment here in this history.

If the previous comment is a potential use case, I will use it to demonstrate how GreenDeploy solves it.

If it's a similar solution, then I will do a compare and contrast video. Any questions you can reach me here or via the crisp messenger on the main website at GreenDeploy.io

Thank you.

[ImLunaHey]

Here's how I'm doing it https://github.com/ImLunaHey/ts-template

This replaces the vars in the template files and then does a force push over the repo.

See https://github.com/ImLunaHey/ts-template-demo for an example of it.

[ImLunaHey]

These are the files responsible for the templating.

https://github.com/ImLunaHey/ts-template/blob/main/.github/workflows/templating.yml
https://github.com/ImLunaHey/ts-template/blob/main/.github/workflows/templating.mjs

Unfortunately github doesn't let github actions replace the files in workflows so I've had to add a check before each of them runs so they only run at the correct time.

[ImLunaHey]

If you wanted to customise this then you'd setup something similar with a intermediate step that would wait for a setup.json. You would click "Use this template", push a setup.json, the setup action would trigger and run the templating.

[ImLunaHey]

1. Modify the .mjs file to take the setup.json and spread it with the process.env that's already being passed in.

2. Make the templating only run if the setup.json file exists by adding another [-d] check in the templating.yml.

3. Profit?

[krisalay]

@ImLunaHey You can update the workflow files under .github directory, if you use PAT with repo and workflow scope enabled.

[boemekeld]

Guys, I develop a serverless platform called http://bohr.io (Vercel competitor) that has exactly this feature. At bohr.io it is possible to create templates with environment variables (stored on the platform) that will be replicated when someone uses the template.

[ShadiestGoat]

Just wanted to drop a +1 on this! I hope something comes out with relevance to this...

[IanKnighton]

I just spent a bunch of time trying to build a more streamlined process for our repos and Kubernetes deployments only to find out that this ability doesn't exist.

So you end up with a case where Terraform is using the same remote state file.

Kind of kill the whole plan.

[julijaciric]

+1

[johnnycreightonppb]

+1

[kahirokunn]

+1

[ryandutton]

+1

[kyoh86]

@linrongbin16 @julijaciric @johnnycreightonppb @kahirokunn @megumish @ryandutton @LegendaryB
Just vote in reaction to the first post, PLEASE.
We subscribe this discussion in the hope that some information will be updated, but it is very annoying to be notified of comments that are just +1.

[laith989]

+1

[krisalay]

The template-transformer-and-cleanup GitHub Action is precisely what we need. This action automatically replaces predefined keywords with the new repository's name whenever a repository is created from a template. Moreover, it offers the flexibility to replace any keywords with our custom terms, making it an adaptable solution for various template customization needs.

[FH-Inway]

It is making a commit: https://github.com/thynclabs/template-transformer-and-cleanup/blob/6b17259c95a8bed9a2acf449cd3924c9bdb0f0b5/run.sh#L57

[george-gca]

As I understood this action NEEDS a personal token right? So, the person that will use the template needs to create this token before creating a new repository from the template right?

[krisalay]

@wowkin2 After the repository is created from the template, the default workflow defined in the default-workflow.yml file within the template project will be executed. This workflow will update the keywords and make the initial commit to the repository. Upon completion, the workflow will automatically delete itself.

[krisalay]

Yes, @george-gca , a personal access token with the repo and workflow scope has to be created and added as an organization secret.

[nevillco]

This looks great, but it seems like it only replaces 1 keyword with 1 new value? Do other folks do multiple replacements?

[nichoth]

btw

Have made a template with variables via user-land modules: template-web-component

This is Handlebars + node.

[cyberpwnn]

+1

(people asking for reactions are just trying to ignore it easier, how do you filter by reactions again?)

[george-gca]

Idk if GitHub already have something publicly available to track requested features, but FontAwesome have a leaderboard with the requested icons. These statistics are from GitHub issues, and comments and reactions are used to measure the most requested ones. If something like this would be implemented and considered mostly reactions, the issue about this keeping come back would be alleviated.

[1amkaizen]

It's great to see discussions around enhancing GitHub template repositories with the ability to utilize "variables"! This would indeed simplify the process of creating customized repositories by allowing users to inject metadata dynamically.

Why "Variables" in Template Repositories?

The ability to use variables would empower developers to quickly scaffold new projects tailored to specific needs, making it easier to maintain consistency across multiple repositories. With metadata passed into the template, users could customize project names, descriptions, authors, and other parameters right at the outset, streamlining the initialization process.

Existing Solutions

As you pointed out, tools like Cookiecutter already provide a robust solution for templating projects via the command line, which showcases how this functionality can significantly improve developer workflows.

Additionally, the GitHub Actions example by @rochacbruno demonstrates how to automate aspects of this process using GitHub Actions. This approach not only enhances flexibility but also integrates seamlessly into CI/CD pipelines, promoting best practices right from project creation.

Conclusion

Introducing variables in GitHub template repositories would be a logical next step in enhancing the platform's usability. It could significantly reduce setup time and improve the developer experience by making project initialization more intuitive and customizable. I look forward to seeing potential updates in this direction!

[dgw]

Stop it.