Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Templating Update #358

Open
wants to merge 23 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
77c6544
Refactor, add CustomTemplateValues support
dahendel Jun 20, 2024
1c473cc
Refactor to use renderGoTemplating function
dahendel Jun 20, 2024
84ff1c2
Add new template-based detokenization logic
dahendel Jul 5, 2024
a478882
``` Refactor detokenize logic; remove redundant code ```
dahendel Jul 5, 2024
9f2e7ce
Remove unused toTemplateVariable function
dahendel Jul 5, 2024
3c5bd70
go fmt
dahendel Jul 9, 2024
96bef4f
Adding some code comments
dahendel Jul 9, 2024
98eb9b8
complete test suite and fixes
dahendel Jul 10, 2024
d0faba8
updated delims to not conflict with Terraform <<EOT, more tests
dahendel Jul 10, 2024
60e8ebd
refactor detokenize to handle gitops and metaphor
dahendel Jul 11, 2024
a6eb775
add optional cleanup step based on K1_TEST_CLEANUP
dahendel Jul 11, 2024
0a5e2c6
Merge branch 'main' into main
dahendel Jul 12, 2024
b1040e0
add toJson func for templating
dahendel Jul 16, 2024
c886ff8
chore: bump gitops template version (#366)
CristhianF7 Jul 15, 2024
20fd38a
[skip ci] [CI SKIP] setting kubefirst-api production to chart version…
Jul 15, 2024
a228198
fix : timeout issue for argocd and vault for public clouds (#363)
jokestax Jul 15, 2024
10455f9
go.mod dep update
dahendel Jul 16, 2024
d75f6fb
fmt
dahendel Jul 16, 2024
548959a
Merge remote-tracking branch 'upstream/main'
dahendel Jul 16, 2024
ef3bd8a
update doc comment on return statement ToTemplateVars
dahendel Jul 25, 2024
ffe250f
addressing pr comments
dahendel Jul 25, 2024
5a69273
Update repo URL and add directory existence check
dahendel Jul 25, 2024
6a02964
correct a DetokenizeGitGitops Call from services
dahendel Jul 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ require (
cloud.google.com/go/container v1.24.0
cloud.google.com/go/secretmanager v1.10.0
cloud.google.com/go/storage v1.29.0
github.com/Masterminds/sprig/v3 v3.2.2
github.com/argoproj/argo-cd/v2 v2.6.7
github.com/argoproj/gitops-engine v0.7.3
github.com/atotto/clipboard v0.1.4
Expand Down Expand Up @@ -36,6 +37,7 @@ require (
github.com/otiai10/copy v1.7.0
github.com/rs/zerolog v1.29.1
github.com/sirupsen/logrus v1.9.0
github.com/stretchr/testify v1.8.4
github.com/swaggo/files v1.0.0
github.com/swaggo/gin-swagger v1.5.3
github.com/swaggo/swag v1.16.1
Expand All @@ -52,9 +54,15 @@ require (
)

require (
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/aws/aws-sdk-go-v2/service/ec2 v1.91.0 // indirect
github.com/go-resty/resty/v2 v2.11.0 // indirect
github.com/huandu/xstrings v1.3.1 // indirect
github.com/mitchellh/copystructure v1.0.0 // indirect
github.com/mitchellh/reflectwalk v1.0.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/segmentio/analytics-go v3.1.0+incompatible // indirect
github.com/shopspring/decimal v1.2.0 // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
)

Expand Down
17 changes: 17 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,13 @@ github.com/KyleBanks/depth v1.2.1 h1:5h8fQADFrWtarTdtDudMmGsC7GPbOAu6RVB3ffsVFHc
github.com/KyleBanks/depth v1.2.1/go.mod h1:jzSb9d0L43HxTQfT+oSA1EEp2q+ne2uh6XgeJcm8brE=
github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd h1:sjQovDkwrZp8u+gxLtPgKGjk5hCxuy2hrRejBTA9xFU=
github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd/go.mod h1:64YHyfSL2R96J44Nlwm39UHepQbyR5q10x7iYa1ks2E=
github.com/Masterminds/goutils v1.1.1 h1:5nUrii3FMTL5diU80unEVvNevw1nH4+ZV4DSLVJLSYI=
github.com/Masterminds/goutils v1.1.1/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU=
github.com/Masterminds/semver/v3 v3.1.1/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs=
github.com/Masterminds/semver/v3 v3.2.0 h1:3MEsd0SM6jqZojhjLWWeBY+Kcjy9i6MQAeY7YgDP83g=
github.com/Masterminds/semver/v3 v3.2.0/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
github.com/Masterminds/sprig/v3 v3.2.2 h1:17jRggJu518dr3QaafizSXOjKYp94wKfABxUmyxvxX8=
github.com/Masterminds/sprig/v3 v3.2.2/go.mod h1:UoaO7Yp8KlPnJIYWTFkMaqPUYKTfGFPhxNuwnnxkKlk=
github.com/Microsoft/go-winio v0.4.15/go.mod h1:tTuCMEN+UleMWgg9dVx4Hu52b1bJo+59jBh3ajtinzw=
github.com/Microsoft/go-winio v0.4.17/go.mod h1:JPGBdM1cNvN/6ISo+n8V5iA4v8pBzdOpzfwIujj1a84=
github.com/Microsoft/go-winio v0.5.2 h1:a9IhgEQBCUEk6QCdml9CiJGhAws+YwffDHEMp1VMrpA=
Expand Down Expand Up @@ -660,11 +665,14 @@ github.com/hashicorp/vault/api v1.9.0/go.mod h1:lloELQP4EyhjnCQhF8agKvWIVTmxbpEJ
github.com/heketi/heketi v10.3.0+incompatible/go.mod h1:bB9ly3RchcQqsQ9CpyaQwvva7RS5ytVoSoholZQON6o=
github.com/heketi/tests v0.0.0-20151005000721-f3775cbcefd6/go.mod h1:xGMAM8JLi7UkZt1i4FQeQy0R2T8GLUwQhOP5M1gBhy4=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/huandu/xstrings v1.3.1 h1:4jgBlKK6tLKFvO8u5pmYjG91cqytmDCDvGh7ECVFfFs=
github.com/huandu/xstrings v1.3.1/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
github.com/hudl/fargo v1.3.0/go.mod h1:y3CKSmjA+wD2gak7sUSXTAoopbhU08POFhmITJgmKTg=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.11/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA=
github.com/imdario/mergo v0.3.13 h1:lFzP57bqS/wsqKssCGmtLAb8A0wKjLGrve2q3PPVcBk=
github.com/imdario/mergo v0.3.13/go.mod h1:4lJ1jqUDcsbIECGy0RUJAXNIhg+6ocWgb1ALK2O4oXg=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
Expand Down Expand Up @@ -810,6 +818,8 @@ github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo
github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM=
github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible/go.mod h1:8AuVvqP/mXw1px98n46wfvcGfQ4ci2FwoAjKYxuo3Z4=
github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
github.com/mitchellh/copystructure v1.0.0 h1:Laisrj+bAB6b/yJwB5Bt3ITZhGJdqmxquMKeZ+mmkFQ=
github.com/mitchellh/copystructure v1.0.0/go.mod h1:SNtv71yrdKgLRyLFxmLdkAbkKEFWgYaq1OVrnRcwhnw=
github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
Expand All @@ -824,6 +834,8 @@ github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh
github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/mitchellh/reflectwalk v1.0.0 h1:9D+8oIskB4VJBN5SFlmc27fSlIBZaov1Wpk/IfikLNY=
github.com/mitchellh/reflectwalk v1.0.0/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/mmcloughlin/avo v0.5.0/go.mod h1:ChHFdoV7ql95Wi7vuq2YT1bwCJqiWdZrQ1im3VujLYM=
github.com/moby/ipvs v1.0.1/go.mod h1:2pngiyseZbIKXNv7hsKj3O9UEz30c53MT9005gt2hxQ=
github.com/moby/spdystream v0.2.0 h1:cjW1zVyyoiM0T7b6UoySUFqzXMoqRckQtXwGPiBhOM8=
Expand Down Expand Up @@ -1050,6 +1062,8 @@ github.com/segmentio/backo-go v1.0.1/go.mod h1:9/Rh6yILuLysoQnZ2oNooD2g7aBnvM7r/
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/shopspring/decimal v1.2.0 h1:abSATXmQEYyShuxI4/vyW3tV1MrKAJzCZ/0zLUXYbsQ=
github.com/shopspring/decimal v1.2.0/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFRcu2hWCYk4o=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
Expand All @@ -1074,6 +1088,7 @@ github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z
github.com/spf13/afero v1.9.3 h1:41FoI0fD7OR7mGcKE/aOiLkGreyf8ifIOQmJANWogMk=
github.com/spf13/afero v1.9.3/go.mod h1:iUV7ddyEEZPO5gA3zD4fJt6iStLlL+Lg4m2cihcDf8Y=
github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cast v1.5.0 h1:rj3WzYc11XZaIZMPKmwP96zkFEnnAmV8s6XbB2aY32w=
github.com/spf13/cast v1.5.0/go.mod h1:SpXXQ5YoyJw6s3/6cMTQuxvgRl3PCJiyaX9p6b155UU=
github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
Expand Down Expand Up @@ -1116,6 +1131,7 @@ github.com/stretchr/testify v1.7.4/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/subosito/gotenv v1.4.2 h1:X1TuBLAMDFbaTAChgCBLu3DU3UPyELpnF2jjJ2cz/S8=
github.com/subosito/gotenv v1.4.2/go.mod h1:ayKnFf/c6rvx/2iiLrJUk1e6plDbT3edrFNGqEflhK0=
Expand Down Expand Up @@ -1249,6 +1265,7 @@ golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8U
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200414173820-0848c9571904/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
Expand Down
4 changes: 2 additions & 2 deletions pkg/providerConfigs/adjustDriver.go
Original file line number Diff line number Diff line change
Expand Up @@ -985,7 +985,7 @@ func PrepareGitRepositories(

// DETOKENIZE
//* detokenize the gitops repo
DetokenizeGitGitops(gitopsDir, gitopsTokens, gitProtocol, useCloudflareOriginIssuer)
err = DetokenizeGitGitops(gitopsDir, gitopsTokens, gitProtocol, useCloudflareOriginIssuer)
if err != nil {
return err
}
Expand All @@ -999,7 +999,7 @@ func PrepareGitRepositories(

// DETOKENIZE
//* detokenize the metaphor repo
DetokenizeGitMetaphor(metaphorDir, metaphorTokens)
err = DetokenizeGitMetaphor(metaphorDir, metaphorTokens)
if err != nil {
return err
}
Expand Down
62 changes: 27 additions & 35 deletions pkg/providerConfigs/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,8 @@ package providerConfigs

import (
"fmt"
"os"

"github.com/rs/zerolog/log"
"os"
)

type ProviderConfig struct {
Expand All @@ -27,35 +26,32 @@ type ProviderConfig struct {
VultrToken string
CloudflareAPIToken string
CloudflareOriginCaIssuerAPIToken string

GithubToken string
GitlabToken string

ArgoWorkflowsDir string
DestinationGitopsRepoHttpsURL string
DestinationGitopsRepoGitURL string
DestinationGitopsRepoURL string
DestinationMetaphorRepoHttpsURL string
DestinationMetaphorRepoGitURL string
DestinationMetaphorRepoURL string
GitopsDir string
GitProvider string
GitProtocol string
K1Dir string
Kubeconfig string
KubectlClient string
KubefirstBotSSHPrivateKey string
KubefirstConfig string
LogsDir string
MetaphorDir string
RegistryAppName string
RegistryYaml string
SSLBackupDir string
TerraformClient string
ToolsDir string

GitopsDirectoryValues *GitopsDirectoryValues
MetaphorDirectoryValues *MetaphorTokenValues
GithubToken string
GitlabToken string
ArgoWorkflowsDir string
DestinationGitopsRepoHttpsURL string
DestinationGitopsRepoGitURL string
DestinationGitopsRepoURL string
DestinationMetaphorRepoHttpsURL string
DestinationMetaphorRepoGitURL string
DestinationMetaphorRepoURL string
GitopsDir string
GitProvider string
GitProtocol string
K1Dir string
Kubeconfig string
KubectlClient string
KubefirstBotSSHPrivateKey string
KubefirstConfig string
LogsDir string
MetaphorDir string
RegistryAppName string
RegistryYaml string
SSLBackupDir string
TerraformClient string
ToolsDir string
GitopsDirectoryValues *GitopsDirectoryValues
MetaphorDirectoryValues *MetaphorTokenValues
}

// GetConfig - load default values from kubefirst installer
Expand All @@ -69,12 +65,10 @@ func GetConfig(
cloudflareOriginCaIssuerAPIToken string,
) *ProviderConfig {
config := ProviderConfig{}

homeDir, err := os.UserHomeDir()
if err != nil {
log.Fatal().Msgf("something went wrong getting home path: %s", err)
}

// cGitHost describes which git host to use depending on gitProvider
var cGitHost string
switch gitProvider {
Expand All @@ -83,7 +77,6 @@ func GetConfig(
case "gitlab":
cGitHost = GitlabHost
}

config.DestinationGitopsRepoURL = fmt.Sprintf("https://%s/%s/gitops.git", cGitHost, gitOwner)
config.DestinationGitopsRepoGitURL = fmt.Sprintf("git@%s:%s/gitops.git", cGitHost, gitOwner)
config.DestinationMetaphorRepoURL = fmt.Sprintf("https://%s/%s/metaphor.git", cGitHost, gitOwner)
Expand All @@ -105,6 +98,5 @@ func GetConfig(
config.SSLBackupDir = fmt.Sprintf("%s/.k1/%s/ssl/%s", homeDir, clusterName, domainName)
config.TerraformClient = fmt.Sprintf("%s/.k1/%s/tools/terraform", homeDir, clusterName)
config.ToolsDir = fmt.Sprintf("%s/.k1/%s/tools", homeDir, clusterName)

return &config
}
4 changes: 4 additions & 0 deletions pkg/providerConfigs/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,8 @@ const (

ArgocdPortForwardURL = runtimepkg.ArgocdPortForwardURL
VaultPortForwardURL = runtimepkg.VaultPortForwardURL

TokenRegexPattern = "<([A-Z_0-9]+)>"
leftDelimiter = "[["
rightDelimiter = "]]"
)
Loading