Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update common #10

Merged
merged 1,794 commits into from
Sep 5, 2024
Merged

Update common #10

merged 1,794 commits into from
Sep 5, 2024

Conversation

mbaldessari
Copy link
Contributor

  • Initial commit
  • Automate the creation of common manifests
  • Repair templating
  • test: try populating applications at the submodule level
  • Ensure we look from the top namespace for options
  • Avoid use of uniq and rely on the user
  • ApplicationSets don't support random fields, use Helm ranges instead
  • Consume options from the global namespace
  • Create subscriptions with the correct helm variable
  • Test: Drop the ACM CRD to see if we can use sync-waves
  • Use the repo and revision from argo, and pass to per application charts
  • Enable self healing
  • Use and pass on the global blueprint values
  • Escape the range scope
  • Escape the range scope
  • Turn off argo validation for when apps include a subscription and its config
  • Differentiate between hub/datacenter and edge applications
  • Ignore loggingCA changes
  • Avoid using the site location in values files
  • Consume and pass on a valuesFile directory to allow multiple levels of configuration
  • Remove invalid parameter reference
  • Not too many valueFiles, argo borks on anything thats missing
  • Disabiguate argocd application names
  • Add the ACM application
  • Removed old ACM policies and subscriptions, add back the multiclusterhub
  • bootstrap creates the namespaces
  • Try pre-creating the multiclusterhub
  • Fix the acm application
  • Tell argo to ignore missing acm CRDs that appear once the multiclusterhub is defined
  • Allow for multiple site types driven by the shared values file
  • Support a per-site valuesFile
  • Placement rules are defined by the operator
  • Not too many valueFiles, argo borks on anything thats missing
  • Disabiguate argocd application names and enable acm
  • .argocd-source.yaml is not useful because it needs to be in the chart directory
  • Initial commit
  • Clean up and minimalize helm template variables
  • Simplify the argo application, and see if .argocd-source.yaml gets applied
  • Drop the instance label so argo doesn't try to manage itself
  • Pass the repo and revision to the bootstrap helm chart
  • Fix typo and enable self healing
  • Argo seems to be ignoring .argocd-sources.yaml, try a different approach for now
  • Accept a full URL to the values file and pass to child apps
  • Use a valuesFile directory to allow multiple levels of configuration
  • Not too many valueFiles, argo borks on anything thats missing
  • Add symlink for development
  • Adjust paths to the new structure
  • See if self healing is what prevents subscriptions without a CSV
  • Use the default namespace for the gitops placement rule
  • Fix symbolic link
  • Create a operator group per namespace to make OLM happy
  • Provide more subscription defaults
  • Review and simplify argo sync options
  • Ignore objects derived from our ACM policies
  • See if we can set SkipDryRunOnMissingResource att the application level
  • Revert "See if we can set SkipDryRunOnMissingResource att the application level"
  • Add more subscription defaults
  • Support to kustomize targets
  • Support applications using plugins
  • Cannot assume the subscription name and csv prefix match
  • Allow custom ignoreDifferences values
  • Helper for deploying the repo accurately
  • Include the site name in the initial app
  • Simple site to test argo plugins
  • Pass site value files when using the helm-with-kustomize plugin
  • Use the app name when rendering the helm template
  • Add missing file
  • kustomize debugging
  • The helm-with-kustomize example is useful to keep around
  • Additional help targets
  • Support loading just the site application on install
  • Make it easy to drive the bootstrap option from the helper
  • Make it easy to kustomize the secrets location from the helper
  • Limit helm's template name to under 53 chars
  • Create the application name ordered by significance
  • Site name is included in the site's .Release.Name
  • Try to stay under helm's name template max of 53 chars
  • Fix the application role binding to refer to the actual policy
  • Use the managed site name for the application and values file
  • Enforce quoting of helmOverrides for acm policy applications
  • Provide a default path to the site chart
  • Drop references to manuela in common
  • Allow namespaces without operatorgroups for charts that already include them
  • Add argocd secret handling
  • Add argosecret target
  • Remove unneeded util dir
  • Explain how to use common makefile
  • Allow namespaces without operatorgroups for charts that already include them
  • Add template toplevel makefile
  • Allow templates to know which namespace they're being deployed into
  • Refresh the kustomize-renderer example
  • Refresh the sample datacenter from manuela
  • Don't assume the install symlink is present
  • Attempt new template format
  • Ensure password has length before exiting wait loop
  • Replace makefile template and make embedded shell wait for password to have length as well as 0 exit
  • Add then
  • Make script explain what it's doing
  • make output clearer
  • Revert "Attempt new template format"
  • Try putting applications into a dedicated argo instance
  • All namespaces are created by the site chart
  • Fix default applicaiton namespace handling
  • Make the pattern name availble to argo applications
  • Give preference to whatever was specified in the secrets file
  • Strip off any auth tokens when calculating the target repo
  • Ensure there are no spaces in the namespace
  • Fix namespace for applications
  • Fix namespace for application acm policies
  • Include the pattern name to ensure uniqueness
  • Update repo name
  • Try putting applications into a dedicated argo instance
  • All namespaces are created by the site chart
  • Fix default applicaiton namespace handling
  • Make the pattern name availble to argo applications
  • Give preference to whatever was specified in the secrets file
  • Strip off any auth tokens when calculating the target repo
  • Ensure there are no spaces in the namespace
  • Fix namespace for applications
  • Fix namespace for application acm policies
  • Include the pattern name to ensure uniqueness
  • Fix the destination namespace for datacenter manifests
  • Try a simpler argo name
  • Use a shorter namespace
  • acm: Fix the target namespace for the site application policy
  • Fix application namespace
  • The application name is already unique to the namespace
  • Restore gitops to the name of the argocd CR
  • Match the service accounts to the argocd name
  • Document what argocd values need to be kept in sync
  • Updated note regarding argo name
  • Change the default 'name' label for namespaces
  • Update common Makefile to have more parameters for secret making
  • Re-factor install to not require .ONESHELL features as Mac doesn't support them out of the box
  • Update Makefile doc and SECRET_NAME parameter
  • Don't hardcode SECRET_NAME here
  • Move script to scripts dir and call it from there
  • New s3 secrets file for central-s3 support
  • Took care of merge conflicts with s3-secret.yaml
  • Adding functionalist to have a list of namespaces for a particular subscription
  • Enhance compatibility with older versions of git
  • Trim the example datacenter site
  • Support real helm charts too
  • Adds the if control to force booleans to be string type for argo on helm/vault overrides
  • Improved secrets handling in pipelines (Update common #10)
  • Add note regarding tekton annotation
  • Add note regarding tekon annotation
  • Ensure updated secret template variables are defined
  • Missing template variable
  • Update values.yaml
  • Avoid assumptions about the site being called datacenter leaking into patterns
  • Add missing template variables
  • Standardize on Values.secrets for usernames as well as passwords
  • Sync the example global values file with the i-e pattern
  • Sync the plugin example application with the i-e pattern
  • Ensure helm options stay in sync and add a simple test
  • Make the test more stable and add missing values
  • Extend the unit tests to the acm and site charts
  • Ensure the global.repoURL variable is set consistently in tests
  • Add some elements to .gitignore
  • Fix whitespace in repoURL value in a POSIX-friendly way
  • Remove manuela-specific elements and secrets
  • Modify tests to match removal of secrets and namespace
  • Remove cruft from makefile
  • Loosen regex to extract target repo
  • Add structure for vault
  • Remove vault subdir to prep for alternate inclusion
  • Squashed 'vault/' content from commit 9fa25e9
  • Change site to clustername to allow for multiple clusters in a config group
  • Remove staging and adjust tests to reflect that
  • Update examples for recent cleanups
  • Support ocp authentication for namespaced argos
  • Update examples for recent cleanups
  • Support ocp authentication for namespaced argos
  • Make sure that argo can manage cluster-wide resources from the relevant namespaces
  • Add some vault utilities and add a gitignore entry
  • Add Makefile target to init and unseal vault
  • Add an unseal target and provide it a default in the script
  • Initial import of chart from 9348383 of https://github.com/external-secrets/external-secrets
  • Remove vault and external secrets - we can install from helm directly
  • Protect ourselves from calling vault-init twice
  • Add script for copying vault-token
  • Add Hub cluster domain as a global variable based on ingress config
  • Add code to extract root token
  • Add function to wrap an exec including the token
  • Add pki init to vault init makefile target
  • Expand the PKI domain (knock off the first two domains for the hub cluster, e.g. apps and the next one up to allow the PKI to be bigger
  • Correct pki role and domain creation
  • Add more functions for secrets management
  • pki init is done in vault_init, no need to have a separate makefile task
  • Fix the name of the function to initialize the kubernetes backend
  • Add --interactive to the oc vault exec calls
  • Add a policy init function to setup initial policy for the vault
  • Add variable qualification to prevent helm template errors
  • Add vault-route to workaround hard-coding of passthrough mode in vault helm chart 0.18.0
  • Correct route resource, remove namespace and spell variable correctly
  • Fix TTL lease typo in vault-init
  • Remove extra duplicate subcription YAML and force quoting in application install for consistency
  • Add local domain to ACM policy
  • Propogate localdomain setting to non-privileged argo
  • Fix some tests
  • Fix remaining tests
  • Remove manuela tag from clustergroup chart
  • Add extra framework options to level with clustergroup implementation
  • Remove vault-route application
  • Remove vault-route application
  • Supply hubClusterDomain for localHubCluster where we don't have ACM to populate the lookup
  • Don't conditionalize lookups when we know we need them
  • Remove bashism in vault_exec
  • Add namespace support to the regional gitops installations
  • Add code to validate origin for install/upgrade
  • Add better domain alternation logic and Makefile validation
  • Stop using echo when returning a string in a function
  • Add support for pushing the kube-root-ca.crt from the HUB to the managed clusters
  • Fix the TARGET_REPO calculation
  • Fix common/ make test
  • Remove clusterselector for cases where we want the vault-ca installed on the hub cluster as well
  • Fix common/ make test
  • Replicate Ensure the argo server and applications are created in the same names… multicloud-gitops#36
  • Remove policy to deploy vault CA as unnecessary
  • Changes to vault-utils to support vault/external-secrets combo
  • Rename the namespace and serviceaccounts to the name of the new golang-based external secrets operator
  • Add golang-external-secrets chart
  • Add script to push vault secrets
  • Fix test error in clustergroup example
  • Fix acm naked example test
  • Fix tests/clustergroup-normal.expected.yaml test
  • Add initial checking github action on every pull/push
  • Add a helmlint target that runs helm lint over the charts
  • Run make helmlint on every push/pull request
  • Add golang-external-secrets to the charts being tested
  • Add right params to helmlint
  • Move make_common_subtree.sh to common from multicloud-gitops
  • Find out charts dynamically
  • Fix examples values to use imageregistry
  • Add naked and normal expected.yaml files for examples-kustomize-renderer chart
  • Add proper naked+normal golang-secrets expected yaml files
  • Add external-secrets dependency chart to golang-external-secrets
  • Make sure we fail if helm template errors out
  • Add global hubClusterDomain to golang-external-secrets values
  • Remove Makefile.toplevel
  • Update secret.sh from Industrial Edge
  • Remove old vault-token.sh script
  • Make the default policy for hub/ secrets more restrictive*
  • Remove init target
  • Add load-secrets target
  • Switch to mustonlyhave complianceType in templates pushed out by ACM
  • Add a make help target to list all existing targets
  • Do not bail out when KUBECONFIG is unset
  • Add a kubeval makefile target
  • Add helm kubeval task to linter
  • Allow to override the remote name
  • Update external-secrets to 0.4.4
  • Move to kubeconform
  • Bring in some change that was missed
  • Allow skipped resource types to be customized
  • Make sure we only bail out when both KUBECONFIG and ~/.kube/config are not set
  • Stop bailing out if the vault file exists
  • Add .bak file to .gitignore
  • Remove the bootstrap concept from common/
  • Remove argosecret target from common/
  • Improve error message when values-secret.yaml is not formatted as expected
  • ServiceAccount hub-gitops-argocd-dex-server needs more permissions
  • Add links for clustergroup argo(s)
  • Fix tests
  • Remove wrong line in examples/values-secret.yaml
  • Quote key and value when uploading a secret
  • Fix quoting in case of multiple secrets per-key
  • Remove useless exclude: kube- from ACM policies*
  • Add a small precheck to make sure python kubernetes is importable
  • Convert the test example to non-anonymous lists
  • Support easily disabling subscriptions
  • Fix up helm tests
  • Change all ACM policies' severity definition
  • Improve installation resilience for vault
  • Add initial support for imperative jobs
  • Add some helm tests for imperative jobs
  • Change image and add some additional comments
  • Remove SECRETS from the initial helm chart args passing
  • Update external-secrets to 0.5.6
  • Move to eso-0.5.7
  • Add super linter
  • Switch kubeconform to use OCP 4.10 + ACM 2.5 + ESO 0.5.x
  • Switch to external-secrets.io/v1beta1 for the clusterstore
  • Add support for .disabled in imperative jobs
  • Fix verbosity job attribute
  • Add support for tags in imperative jobs
  • Add super-linter.log to .gitignore
  • Add support for extravars in imperative jobs
  • Move vault push secrets task into a separate role
  • Shuffle files around to keep the ansible tree structure cleaner
  • Remove files from their old place
  • Initial work on vault_init
  • Avoid using all these playbooks
  • Always pass absolute paths
  • Implement most of vault_init and vault_unseal
  • Complete vault_secrets_init
  • Do not print unseal keys in log
  • Added vault_pki_internal command
  • Silence localhost warning
  • Move to tags + cleanups
  • Fix up metadata
  • Clean up README a bit
  • Drop dependencies for now
  • Set debug to default(False) inside the play directly since we dropped the var
  • Error out nicely when secrets key is missing
  • Wait for vault NS and vault pod
  • Make secrets parsing a little more robust
  • Wait for vault ns+pod using until/retries
  • Make parsing of value-secret.yaml more robust
  • Do not use readlink vault-init/vault-seal
  • Add ansible linting job
  • Address all ansible-lint warnings
  • More linting fixes
  • Retry the Init vault operator task
  • Correct the debug statement in vault_delete
  • Various fixes
  • Spell out which tags in the makefile, make things clearer and code shorter
  • Fix pipefail and indent
  • Use FQCN everywhere
  • Fix boolean comparison
  • Keep default vault output file consistent: common/vault.init
  • Ops forgot a merging tag leftover
  • Do not bail out when KUBECONFIG is unset
  • Rename output_file to common/pattern-vault.init
  • Make the default policy for hub/ secrets more restrictive*
  • Stop bailing out if the vault file exists
  • Improve error message
  • Backport quote key and value when uploading a secret
  • Backport more fixes to ansible
  • Backport more fixes to ansible
  • Update ansible-lint to latest version v.6.2.1
  • Add ansible-lint target
  • Fix some ansible linting errors
  • Fix some more ansible linting errors
  • Use path and not args and using ansible-lint
  • Use ansible-lint from a container
  • Only rename file when it exists
  • Fix unseal target
  • Use the simpler realpath filter to get absolute path locally
  • Be more idempotent and fix the vault config policies
  • Break up too long line to fix linting
  • Use the role to push secrets
  • Remove unused scripts/ansible-push-vault-secrets.sh
  • Split off all pre check tasks into a single file
  • Remove spurious debug task
  • Increase wait for vault status and bail out if k8s does not return stdout at all
  • Add short algorithm exaplanation
  • Make things a lot more idempotent and add file_unseal var
  • Forgot to commit the pre_check.yaml file
  • Consolidate vault status checking in a single file
  • Remove too many blank lines and please linter
  • Fix fact parsing from json
  • On my slow local cluster vault pod takes longer to come up
  • Make sure we call bash when setting up the kubernetes backend
  • Make sure we bail out in case of error
  • Fix last escaping bug
  • When using a secret store unseal material in a field called vault_data_json
  • Cleanup meta/main.yml
  • Cleanup README.md
  • Use consistent output_file name. Move to a more portable absolute path function
  • Fix some markdown to please the linters
  • Remove all oc calls and allow for running from inside a cluster
  • Fix up PKI creation and add not about it not being currently used
  • Fix delete to use native ansible
  • Move local roles in front of the path
  • Improve validate-origin
  • Update argo crd
  • In push_secrets wait more if the vault is unsealed
  • Remove spurious double empty-line
  • WIP Add job to unseal the vault from the cluster
  • Add ignoreMissingValueFiles: true on helm charts
  • Add some templates to reduce code duplication
  • Add done container as a named template
  • Switch do a common 'done' container
  • Explicitely enable clusterGroup.unsealVaultInsideCluster to test unseal templates
  • Rename unsealVaultInsideCluster to insecureUnsealVaultInsideCluster
  • This step is not really needed when running make vault-init + load-secrets as everything is sequential It is needed when the vault is unsealed/configured inside the cluster and load-secrets gets run while the cronjob configures the vault. I.e. it might be half configured and return errors below:
  • Add finalizers to all applications
  • Move to import_tasks for vault_init/main.yml
  • Rename old install machinery New install based on operator Move CRD to correct location Add ignoreMissingValueFiles: true on helm charts
  • Remove Pattern from kubeconform skiplist
  • Let's switch the legacy-install back to install
  • Unify install/deploy and upgrade targets
  • Cleanup configManagementPlugins section in argocd.yaml
  • Add explicit legacy-deploy legacy-upgrade targets
  • Drop valuesDirectoryURL entirely
  • Remove spurious older tests
  • Move operator installation in patterns-operator-system namespace
  • Update tls route config for centralized cert management
  • Adding support for .spec.config.env to subscription.yaml template. While testing deploying Industrial Edge on a FIPS mode the AMQ Streams operator was not deploying due to the FIPS mode. There was a need to pass an environment variable to the operator so the way we do that is by defining the ENV in the manifest.
  • Revert "Move operator installation in patterns-operator-system namespace"
  • WIP add validate-prereq target
  • Extend values-secret parsing to include files section
  • Add config file for ansible-lint
  • Comment to allow parsing of config
  • Fix issues pointed out in review
  • Change file to content to remove ambiguity
  • Add support for extraValueFiles section for applications
  • Also fix edge cases with secrets loading
  • Remove stray comment marker
  • Add fileParameters
  • Add applicationsets
  • Add overrides
  • Argo gets angry if it things valuesfiles is a string
  • Add applicationset functionality to clustergroup
  • Remove now redundany applicationset separate template file
  • Remove spurious YAML ---
  • destinationURL -> destinationServer + whitespace fix
  • Add global pattern variables
  • Support k8s 1.24
  • Split long lines for easier readability
  • Add a DISABLE_LINTERS variable to super-linter target
  • Change conditionals on imperative framework to allow unseal or job or both
  • Upgrade external secrets to 0.5.9
  • Allow ha vault deployments
  • Add mechanism to create both content and b64content versions for files keys
  • Skip QA rule because yes, the line must be long
  • Upgrade external secrets to 0.5.9
  • Update noqa and remove trailing space
  • Put noqa on the right line
  • Remove whitespace at the end of the line
  • Fix lint warnings and avoid using json_path
  • Use Apache 2.0 License
  • Silence Get Pods loop a bit
  • Print the name of the pod when getting pods
  • Call expanduser on file to match stat behavior
  • Set retries for joining raft cluster
  • Remove CSV after uninstalling
  • Standardize sync-policy handling and deprecate variable that does not vary
  • Draft of a cluster pool chart based on https://github.com/one-touch-provisioning/otp-gitops-clusters
  • Clean up and simplify external secrets
  • Disable the cluster pool by default
  • Assume multiple entries in the aws vault key
  • Defaults need double quotes
  • Use the current namespace
  • vault keys cannot start with a .
  • Consistently name the platform secret
  • Create cluster claims for managed clusters if a pool is defined
  • Obain the specific key from the vault secret
  • Try to obtain the correct pull secret
  • Create infrastructure credientials
  • Put pools and clusters into a set
  • Correctly apply acm labels for the infra secret
  • Avoid the term 'master'
  • Drop stray characters
  • Additional cleanups
  • It is only important to create the pool last
  • Move the cluster provisioning into the acm chart
  • Switch to file based secrets where appropriate
  • File based pullSecret. Fix aws keys in infra secret
  • Split out provisioning secrets into a new template for legibility
  • Allow the pool to have spare members
  • Disambiguate cluster pools and names by group
  • Add support for provisioning azure clusters
  • Update tests - we define a ManagedClusterSet for every cluster 'group'
  • Separate out the platform specific secrets
  • Include the private ssh key
  • Rename manifest for clarity
  • Add the missing azure credientials
  • ACM provisioning test cases and fixes
  • Avoid using LegacyClusterSetLabel for determining cluster sets
  • Fix tests
  • Revert "Avoid using LegacyClusterSetLabel for determining cluster sets"
  • Integrate the ability to specify IIBs as part of the pattern
  • Add hashicorp-vault chart in common/
  • Only create managedclustersets if there are clusterpools
  • Add operatorchannel parameter
  • Add mechanism to use pattern variables as other overrides
  • Denote new fields as 'extra'
  • Move CSV definition into the uninstall target
  • Allow configuring clusters with ExternalSecrets
  • Allow defining the application destination to be a different cluster
  • Add vault consolelink and fix helmlint
  • Unset KUBECONFIG and specify a non existing kubeconfig file when running tests
  • Disable ansible linting in the super-linter
  • Update vault helm chart to 0.22.0
  • Switch unsealing in the cluster from 9mins to 5mins
  • Add pattern-util wrapper script
  • Remove -x
  • Implement a per-subscription csv field
  • Reduce the complexity of secrets needed to drive the argo push model
  • Fix external secret expansion
  • Fix tests
  • Revert parts of 8e33295 (Move the cluster provisioning into the acm chart)
  • Support argo based managed clusters
  • Clearly distinguish between the types of managed clusters being created
  • Drive external argo cluster secrets from managedClusterGroups too
  • Update examples file
  • Mixing spoke applications with the hub is an anti-pattern
  • Update tests
  • Remove non-existant ClusterPool field
  • Ensure secret manifests are not munged together
  • Fix incorrect capitalization in metadata
  • Drop the additional global variable as it's not required
  • Ensure argo secret data is quoted, and in the same namespace as the app that consumes it
  • Reorganise the clusterGroup chart
  • Update acm tests for new cluster pool examples
  • Allow parts of clusterGroup to be disabled for the remote argo usecase
  • Always deliver the argo plumbing to the hub cluster
  • Break global.localClusterDomain for argo seeds in order to make progress
  • Force the correct cluster Group name
  • Fix passing of the target cluster to argo seeds
  • Namespaces were not being disabled as expected
  • Disambiguate the config map when multiple clustergroups are present on the same cluster
  • Restrict imperative elements to clustergroup 'core' mode
  • Create the external secret as part of the argo seed
  • Put collect remote seeds into a group Argo project
  • Only create the argocd super role once
  • Only create external secrets for remote argo clusters
  • Fix 'enabled' mode in clusterGroup
  • Ensure targetRevision is always set in clusterGroup
  • Argo drops prune=false from applications
  • Create the default group namespace for remote seeds on the remote cluster
  • Do not define another argo for remote seeds
  • Remote argo apps need to be processed by the hub's cluster instance
  • Update tests
  • Ensure remote seed elements on the hub end up in the openshift-gitops namesapce
  • Update tests
  • Fix acm tests
  • Require a domain for argo seeds so we can set global.localClusterDomain
  • Update for subscription.yaml template in common/clustergroup
  • Create an example blank chart (#143)
  • Avoid race condition before ESO operator is deployed
  • Fix external secrets template
  • Explicitely ignore ssh pubkeyacceptedalgorithms config option
  • Bind mount the ssh-agent socket if it is set
  • Consistently define a new variable for the non-apps cluster domain
  • kubeserver is now calculated from the managed cluster group for remote argo clusters
  • Ensure app projects land in the namespaced argo instance
  • Update tests
  • Settle on 'hosted argo site' terminology
  • Avoid magic naming schemes and require explicit vault keys
  • Drop outdated comment
  • Drop readlink -f usage
  • Fix CI
  • Allow for a sensible default when constructing the argo cluster secret
  • Replace 'cat' with the simpler 'print' helm function
  • Start a changes file
  • Sanely handle ACM clusters without an explicit label
  • Track expected differences bewteen naked and normal test
  • Group tests by chart for clarity, rename outputs for when searching templates
  • Tweak output
  • Update changes with implications for pattern authors
  • Interactively prompt to update expected test outputs
  • Consistent use of stderr
  • Allow an optional TARGET_SITE variable to drive main.clusterGroupName
  • Automatically add new tests to git
  • Better test cleanup
  • Fix common/ CI
  • Add all missing tests to git
  • Rename tests back to .yaml
  • Add new ignores for python module
  • Add python module implementing the push-secret functionality
  • Switch push_secrets.yaml to the python module implementation
  • Add more detail to the values-secret.yaml example
  • Fix up tests after the secrets example change
  • Tests cleanups
  • Some more linting fixes
  • Some more linting fixes pt2
  • Pass black --skip-string-normalization over the python plugin
  • Pass black --skip-string-normalization over ansible unit test
  • Switched to full black formatting support
  • Add some noqa: comments to ignore flake8
  • Add .gitleaks.toml
  • Also ignore python modules in gitleaks as they contain secrets examples in the comments
  • Move .gitleaks.toml under .github/linters
  • Add a CHANGES.md entry for the push_secrets rewrite
  • Drop -f example/values-secrets.yaml from make test
  • Drop GIT_SSH_COMMAND from the wrapper
  • Return {} when parsing an empty yaml file
  • Make sure to skip loops that have None in their range
  • Make sure to expand user path when checking for file existence
  • Add a number of testcases that cover corner cases in the values files
  • Fix up flake8 and black warnings
  • Fix ansible-lint
  • Cleanup tests and bail out properly when files/secrets: are lists
  • Reformat isinstance list fail message
  • Added callbacks to ansible.cfg
  • Add a pattern_dir variable to the playbook
  • Add a check a check_missing_secrets parameter to vault_load_secrets
  • Add a number of unit tests to verify the new check_missing_secrets functionality
  • Add all the code to actually verify secrets against values-secret-template.yaml but keep it disabled
  • Fix a couple of linting issues
  • Tweak the SSH paths so ssh-agent works again
  • Pass -e KUBECONFIG only if it is set
  • Get the correct pattern name when the path has a folder
  • Implement global.clusterVersion
  • Tweak the /values-ocpversion calculation in the acm policy
  • Be more precise with the Changes.md
  • Use a simpler expression in ACM templates
  • Add a note about argo hub and spoke and clusterversion having to be the same in that case
  • Add possibility to retry commands in run_command()
  • Symlink .github/linters/.gitleaks.toml to .gitleaks.toml
  • Updated ansible.cfg with non-deprecated feature.
  • Skip git ssh checks when running inside a container
  • Lower case cluster names in values-example.yaml
  • Switch to a more recent setup-helm action
  • Update ESO to 0.6.0
  • Use UBI based images for ESO
  • Use no_log: true in any sensitive place
  • Disable VALIDATE_DOCKERFILE_HADOLINT VALIDATE_TEKTON by default in common/Makefile
  • Remove .Release.name from namespace labels
  • Upgrade to vault-0.22.1 helm chart
  • Clean up how we patch the subchart for vault
  • Remove old patch
  • Write down some new features in Changes.md
  • Upgrade ESO to 0.6.1
  • Make sure we can parse FQDNs after a secrets tag in values-secrets.yaml
  • Reorganize tests in preparation for a V2 secrets format
  • Add ansible unittest
  • Use proper 3.9 version that exists in ubuntu
  • Install pytest via pip
  • Add ansible in the pip modules
  • Add support for creating temporary files and referncing them in tests
  • Revert to a simpler way to test file upload
  • Add tearDown() in unit tests
  • Add ansible/tests/unit/v1/values-secret-good.yaml
  • Fix super linter warnings
  • Add support for /values--.yaml
  • Fix gitleaks file after last unit test change
  • Allow multiple test inputs
  • Initial v2 plumbing
  • Add module_utils path for shared common code
  • Move v1 secret functions in module_utils
  • Split out common function in separate file
  • Move the secret format implementation in classes
  • A bunch of cleanups of unit-testing. Still not fully passing
  • Have super-linter pass
  • Fix patching the right function
  • Start unit testing V2 as well
  • Add version check in v1 implementation
  • Force get_version() to return a string for the time being
  • Pass proper parameters to V2 class
  • Initial start of vault policies implementation
  • More validation and more V2 testing
  • Make sure we check if the vaultPolicy exists
  • Make error message clearer
  • Add a lot more tests and start implementing the files: section
  • Fix black errors
  • Cleanups and also call check_missing_secrets only on v1.0
  • Fix black errors
  • Add a test to cover for empty vaultPrefixes
  • Add support and tests for the password generation via policy
  • Actually use the proper oc exec call to generate secrets
  • Implement file uploading as base64 attribute as well
  • Move more logic into _get_file_path
  • Large rewrite to unify fields and files and add initial base64 support and tests
  • Add base64 support for secrets as well and fix up tests
  • POC mostly working at this point
  • For the time being always stay on getpass.getpass() for input
  • Remove unused modules
  • Add checks for duplicate secret names and field names
  • Switch to using AnsibleModule.run_command
  • Make the secret prompting a little prettier
  • Implement looking for ~/values-secret-.yaml first
  • Test base64 secret
  • Silence yaml indentation linting errors and more
  • Added values-secret examples
  • Drop callbacks_enabled
  • Add another relevant change in Changes.md
  • Error out nicely if yaml.safe_load() cannot parse the file
  • Add support for ansible-vault
  • Clean up comments in test-cases files and implement backingStore
  • Use sensible prompt strings
  • Be clearer in the text about ansible-vault
  • Move checks that are specific to the version inside classes
  • Add a test for erroring out when values_secret_template is not defined and check_missing_secrets is set to true
  • Add a default validatedPatternDefaultPolicy password policy
  • Forgot to add new test for default vp-policy
  • Make sure we re-attempt all vault commands in case of failure
  • Add some more documentation
  • Add initial v2 schema json file
  • Rename the description field to prompt as that is what it is really used for
  • value and path can be 'null' when onMissingValue is set to 'prompt'
  • Make version a string all the time
  • Require some more fields
  • Add some more examples to the schema file
  • Test json schema CI job
  • Fix small ansible lint warning
  • Also try {{ pattern_dir }}/values-secret.yaml.template if all others failed
  • Fix Changes.md with a small nit
  • Prompt ansible-vault password via ansible.builtin.pause
  • Make the tasks text less intimidating
  • Add json schema for the v1 secret format
  • Allow picking a custom values-secret.yaml file via the VALUES_SECRET env var
  • Add filter_plugins path in common
  • Add override attribute support
  • Move to draft-07 of the json schema for V2
  • Do not raise exception if secret's existence check fails
  • Upgrade vault-helm to v0.23.0
  • Filter out patches to test/
  • Add patch to be able to use vault-ssl as a default
  • Switch to vault-ssl as a default
  • Remove stable default from channels
  • Split secret counter in two variables
  • Validation Schema for common/clustergroup
  • - Fixed issues with schema validation in values.schema.json - Modified clustergroup/values.yaml and examples/values-example.yaml to have same sections and values. - Updated tests for tests/clustergroup-naked.expected.yaml and tests/clustergroup-normal.expected.yaml
  • Fixed issues with kubeconform in clustergroup/values.yaml. Reran make test to generate proper files
  • - Updated schema to include all elements in subscriptions - Updated tests associated with clustergroup - Updated values-example.yaml
  • applications and subscriptions can be obj and array
  • restore values.yaml file to a saner commented default
  • Ran values.schema.json through jq to fix indentation issues
  • Fix up tests
  • Move to draft 0.7 of the schema spec
  • Add operatorgroupExcludes which is in examples/industrial-edge-factory.yaml
  • namespace is not mandatory in an application
  • Fix up some more tests
  • add clusterSelector in managedClusterGroups
  • Allow clusterRoleYaml to be an array and timeout to be a string
  • Add image, tags, extravars to imperative job object
  • Add descriptions for SecretStore properties
  • move GlobalGit and GitSecret after Main
  • Forgot ,
  • Move testing targets after the installation ones
  • Move some of the oc calls into the legacy target
  • Simplify getting the remote git URL associated to the target origin
  • Drop legacy-install from common
  • Move comments in the right place
  • Clean up PHONY targets in Makefile
  • Move all testing-related variables closer to the tests target
  • Make sure we error out after the legacy removal
  • Fix up the main section a bit
  • Make subscriptions not mandatory
  • Add description to main.git fields
  • Drop useCSV from global
  • Add description for main.gitops.channel
  • Add some descriptions
  • Mark useCSV as deprecated
  • - Added descriptions for namespaces, subscriptions, managedClusterGroup, applications, and projects
  • - Fixed issue with description definition
  • drop insecureUnsealVaultInsideCluster entirely
  • - Removed insecure boolean from schema file and from values-example.yaml
  • Re-ran make tests
  • Drop any remains of output_file
  • Updated python-version for json schema
  • Update python version for ansible-unittest
  • Add filter to parse ACM secrets
  • Update ESO to 0.7.0
  • Only set vaultkeys fact when the vault is sealed
  • Default vaultPrefixes to be ["hub"]
  • Fix two V2 secret format corner-case bugs
  • Add SkipDryRunOnMissingResource=true on ClusterPools, ClusterClaims and ManagedClusterSets
  • Add mention of VALUES_SECRET env variable
  • Add a get_ini_value() commodity function
  • Fix up ini function and add tests
  • Add ini_file support
  • Relax json schema and drop onMissingValue as required
  • Fix up json schema errors
  • Fix base64 file upload
  • Fix schema json when using HostedArgo
  • Simplify hosted argo secret paths
  • Add "vault_path" to the parse_acm_secrets filter
  • Fix up some pylint warnings
  • Set additionalProperties to true for top-level objects
  • Fix some validation schema warnings
  • Introduce a validate-schema Makefile target
  • Do not run validate-schema by default
  • helmOverride.value can be both a string and a boolean
  • Make sure our readOnly attributes are proper booleans
  • Drop managedClusterGroups as a required property
  • Drop 'pattern' as a required property
  • Run check-json schema for all yaml files under examples/
  • Allow subscriptions to be null
  • Add prereq check to the operator-deploy
  • Vault delete task is unused, remove it
  • Drop pki init as it is unused
  • Drop push_secrets.yaml playbook as it is unused
  • Make sure we error out with a proper message if python-kubernetes is missing
  • Make prerequisited output checking more consistent
  • Prettify the output for checking prerequisites
  • Extend framework to allow for ESO on ACM-imported clusters
  • Add a global policy in the vault configuration
  • Make sure we use specialized vars for spokes
  • Switch to quay.io/hybridcloudpatterns/utility-container
  • Upgrade ansible-lint action
  • Address a bunch of ansible-lint warnings
  • Upgrade ESO to v0.7.2
  • Add acm policy to push HUB's CA to managed clusters
  • Point golang-external-secrets to the secret injected by ACM
  • Drop imperative job to fetch ACM HUB CA as we use ACM to push that out to clusters
  • Fix up tests
  • Loosen grep regex to catch compound/alias Makefile targets
  • Add pattern-home mount to allow resolution of ~ in files
  • Better help output
  • Stop outputting help for deprecated commands
  • Allow additional properties for GlobalGit
  • Print which files we are using to parse values secrets
  • Drop SSH_SOCK_MOUNT from pattern-util.sh
  • Add /values-.yaml and /values--.yaml support
  • Bail out on unsupported podman versions
  • Use more recent python versions to unreak CI
  • Escape all the variables
  • Add index image support to the schema
  • Consume versioned index images, and provide a default name
  • Allow index images to be a map to avoid accidental overrides
  • Support passing index images as extra parameters
  • Update index image example
  • Add SkipDryRunOnMissingResource=true to all PlacementRules
  • Do not quote KUBECONF_ENV
  • Add support for ~/.config/hybrid-cloud-patterns folder to store secrets
  • Add a comment about removing older search paths
  • Add more logic to install target
  • Fix .git missing warnings when running super-linter locally
  • Switch to ansible-automation-platform-23/ee-supported-rhel8:latest image by default
  • Switch to ESO 0.8.2
  • Updated namespaces template to include labels and annotations functionality
  • Added schema validation to support additional formal for labels and annotations
  • Updated the values-example.yaml to include new format for namespaces
  • Updated Changes.md to include new namespaces functionality.
  • Updating CI tests
  • Fixed Markdown errors
  • Drop dollar sign from default password policy
  • Make sure clusters always has an empty default
  • Do not validate certs when we use letsencrypt on the endpoints
  • Add an experimental letsencypt chart
  • Do not run kubeconform on the certificate stuff just yet
  • Update the gitops patterns CRD
  • Switch to gitops-1.8 channel in common
  • Add a change entry for gitops-1.8
  • Do not apply the ocp-gitops-policy on the hub via ACM
  • Do not hardcode the gitops version that gets installed on spokes
  • Fix up kustomize example
  • Upgrade vault-helm to v0.24.0
  • Add a hello-world ansible playbook example
  • Inject ANSIBLE_CONFIG in make ansible-lint
  • Use new ansible-lint action
  • Fix some ansible-lint warnings
  • Fix up python versions
  • Skip cannot find role error
  • Added health check for pvc resource in argocd.yaml
  • adding tests
  • Update super-linter image to latest
  • Update super-linter image to latest
  • Update CI workflows
  • updated template with why implemented comment
  • Add dependabot settings for github actions
  • adding tests
  • - Added functionality to support the following format for labels and annotations: labels: openshift.io/node-selector: "" annotations: openshift.io/cluster-monitoring: "true"
  • Fixed CI Issues
  • Applying @claudiol recommendation
  • make test
  • Avoid exited containers proliferation
  • Handling of pre-release builds is too complex for a helm chart
  • Fixing issues with operator groups
  • Adding CI test
  • Updated operator group template
  • Updating CI issues
  • Removed duplicate code for operatorgroup by using multiple conditions
  • Allow overriding the pattern's name
  • Add precise instruction to upgrade the vault subchart
  • Upgrade vault-helm to v0.24.1
  • Add an item to README.md
  • Fix up common/ tests
  • Fix super linter
  • Set gitOpsSpec.operatorSource
  • Introduce EXTRA_HELM_OPTS
  • Disable var-naming[no-role-prefix] in ansible lint
  • Add new ansible role to deal with IIBs
  • Simplify load-iib target
  • Add templates folder
  • Fix a couple of linting warnings
  • Fix some super-linter complaints
  • Skip the iib-ci playbook
  • Drop var-naming[no-role-prefix] linter
  • Allow for multiple images when calling load-iib
  • Add help for load-iib
  • Output index_image in make
  • Output index_image in make (2)
  • Set facts later in the playbook not in defaults/
  • Fix how we export vars in make load-iib
  • Fix how we export vars in make load-iib (2)
  • Use machineCount to register the number of nodes that need to be ready
  • Add helpful debug messages
  • Add | on shell now that we call pipefail
  • Test dropping nevercontact source
  • Skip insecure tls when logging in
  • Also allow gchr.io
  • Revert "Test dropping nevercontact source"
  • Fix typo
  • Clarify instructions in the README file
  • Automate the channel example
  • Find out KUBEADMINAPI programmatically
  • Use command instead of shell
  • Do not grep for operator bundle unless it is the gitops operator
  • Also whitelist ghcr.io
  • Fetch the operator bundle itself in a more robust way
  • Add more mirrors
  • Some more work to support MCE
  • Cleanup spacing
  • Fix super-linter
  • Move task in right folder
  • Drop last mention of operator instead of item
  • Improve the grepping for the operator bundle
  • Drop display_skipped_hosts
  • Be more specific about the steps in the README
  • Upgrade ESO to v0.8.2
  • Update README.md
  • Update tests after eso 0.8.2 upgrade
  • Move to new spec format for dex/sso
  • Disable ArgoCD from kubeconform
  • Add a short line about username/token for the iib role on OCP <= 4.12
  • Drop https:// from podman login
  • Set the mce-subscription-spec annotation
  • Fix typo in README for iib
  • Simplify the README a bit
  • Add support for extraParams being passed down to all applications
  • Add a lookup playbook to figure out IIB numbers
  • Allow overriding channel and source when installing the patterns-operator
  • Fix small typo in iib instructions
  • Drop a redirect and up retries when pushing the IIB to the internal registry
  • Update ESO to v0.8.3
  • WIP add presync for eso that waits for vault to be up
  • Add tests
  • Fix image and comment
  • Adding rbac to support the vault sa checking on the vault-0 pod status.
  • Make Test
  • Revert "Make Test"
  • Revert "Adding rbac to support the vault sa checking on the vault-0 pod status."
  • Revert "Fix image and comment"
  • Revert "Add tests"
  • Revert "WIP add presync for eso that waits for vault to be up"
  • Increase the default retry limit when syncing
  • Add Changes.md entry
  • Split off global helm variables to a helper definition
  • Switch ApplicationSets to use the newly-introduced helpers
  • Split off valueFiles to _helpers.tbl
  • Switch applicationsets to use the new helper
  • Drop some older comments
  • Tweak the load secret debug message to be clearer
  • Check if the KUBECONFIG file is pointing outside of the HOME folder
  • Include an example SNO cluster pool in the tests
  • Enforce lowercase names for cluster claims
  • Avoid mixing yaml and json in the OCP install-config
  • Update provisioning tests
  • Sanely handle cluster pools with no clusters (yet)
  • Clustergroup Chart.yaml name change
  • Fix the clusterPoolName in clusterClaims
  • Add some comments to make if/else and loops clearer
  • Add some more comments in applications.yaml
  • Add a default for options applicationRetryLimit
  • Split out values files to a helper for the acm chart
  • Fix up tests
  • Fix sa/namespace mixup in vault_spokes_init
  • Update local patch
  • Update ESO to 0.8.5
  • Tweak ESO UBI images
  • Upgrade vault-helm to v0.25.0
  • Error out from load-iib when INDEX_IMAGES is undefined
  • Add docker.io to the whitelisted registries when loading an IIB
  • Fix fetching the bundle name when loading an IIB
  • Add retries when getting related images
  • Add workflow to split helm charts into their own repo
  • Small test for the workflow towards single chart repos
  • Small test for the workflow towards single chart repos (part 2)
  • Simplify split workflow
  • Small test for the workflow towards single chart repos (part 3)
  • Tiny change to trigger split workflow
  • Add initial helm releasing workflow for acm chart
  • Add helm repo updating workflow in the per-chart workflows folder
  • Fix up CI superlinter on github actions
  • Fix tests and make .disabled explicit
  • Make sure we run the split workflow only when the changes land in validatedpatterns/common
  • Update tests. We get an extra (non-impacting whitespace) with the new code
  • re-add logic for extravaluefiles
  • Add more tests for variable definedness/truth
  • Switch helm to v3.12.3 in CI
  • Unroll global.extraValueFiles in application-policies directly due to namespacing in the _helper.tpl
  • Re-add code to operator-install to understand global.extraValueFiles
  • Make sure to add dollar sign
  • Add initial multi-source support
  • Add changelog entry and add an explicit property entry to the schema
  • Correct ifs and ranges in pattern, add comments
  • Also quote name and value values
  • Introduce an argo-healthcheck make target
  • Drop vault.ui.serviceType: "LoadBalancer"
  • Release clustergroup chart version 0.0.2
  • Update crd in common
  • Add support for passing EXTRA_HELM_OPTS
  • Disable kubeconform for the time being
  • Add support for deploying multi source via CLI
  • Upgrade ESO to v0.9.4
  • Release 0.0.2 golang-external-secrets
  • Adding label validatedpatterns.io/pattern to all applications.
  • CI test updates
  • Simplify the passing of KUBECONFIG
  • Update CRD for the operator
  • Expose UUID
  • Move to newly released checkout action version
  • Update URLs to new github org
  • Add ~/.config/validated-patterns in the secret search path
  • Add support for ~/.config/validated-patterns/pattern-uuid
  • Simplify the code around UUID variable definition
  • Rework installation target
  • Simplify the loop
  • Introduce a validate-cluster target in the install target
  • Increase the wait for the internal registry
  • Add a note about SNOs and internal registries
  • Move from resourceCustomization to resourceHealthcheck
  • Fix up common/ tests
  • Upgrade to ESO 0.9.5
  • Release 0.0.3 golang-external-secrets
  • Release 0.0.3 clustergroup
  • Allow custom templating in .extraValueFiles
  • Support pattern-wide templated value files
  • Update tests
  • Pass in platform and ocp version as charts would expect
  • Add test for value file template expansion
  • Drop the Template.{Name,BasePath} hack due to problems with the imperative configmap
  • Fix up tests after last PR
  • Release clustergroup v0.0.4
  • Add --pull=newer when running the container
  • Allow imperative to be nil
  • Adding key to exclude target namespace in operatorgroup
  • Added target namespace logic to namespace map case
  • Changed description in schema
  • Added example to operatorgroupExcludeTargetNS
  • Fixing CI tests
  • Push localClusterName to remote clusters too
  • Preview a chart based on the current k8s cluster
  • Handle explcit value files
  • Update CRD for the operator
  • Add a README containing the CRD update instructions
  • Add ability to read overrides
  • Clean up tests after 7cda9c4
  • Add preview-all and remove some spurious stdout output
  • All prototype preview-all and silence some output
  • - Removed new key operatorgroupExludeTargetNS - Added key to namespace map entry excludeOperatorGroupTargetNS.
  • Updates to CI
  • Update CRD from the operator
  • Make .plugin handling consistent
  • Preseed the patterns-operator-config configmap
  • Small IIB cleanups
  • Add small curl example for IIB
  • Adding option to include/exclude targetNamespaces in OperatorGroup
  • Updated CI tests
  • - Fix: bug in task TASK [iib_ci : Mirror all the needed images]
  • - Updated the mirrordest_tag to use the sha256 of the image instead of the IIB number.
  • Restored mirror template to original implementation
  • - Updated structure for supporting OperatorGroup's per suggestion of decoupling operatorGroup and targetNamespaces. Example: - exclude-targetns: operatorGroup: true targetNamespaces: - Continues to support operatorgroupExcludes - Updated CI tests
  • Update logic to fix multiple targetNamespaces
  • Fix ci issues
  • Upgraded ESO to v0.9.8
  • Upgrade vault-helm to v0.26.1
  • Parametrize ESO caProvider fields
  • Simplify target namespace logic
  • Avoid nonhubCluster + hubCluster naming for ESO
  • Update for new configmanagement plugin feature
  • Remove obsolete comment and update tests
  • Update schema
  • Require plugin.yaml
  • Add tmpdir to sidecar mounts
  • True up to test code
  • Use nindent as appropriate
  • Remove stray files
  • Plugin config is plugin.yaml
  • Remove now-obsolete kustomize-renderer example
  • Allow pluginArgs to be set and add schema
  • Remove redundancy
  • Revert "Remove now-obsolete kustomize-renderer example"
  • Remove legacy configManagementPlugins support
  • Add configManagementPlugins to tests for industrial edge
  • Clustergroup 0.0.5
  • Small whitespace test
  • Stop referencing remote actions via @main. Use a specific commit
  • Updated ESO to v0.9.9
  • Updated vault-helm to v0.27.0
  • Prevent ArgoCD from writing ESO CRs to clusters that need full support
  • Fix whitespaces
  • Release clustergroup v0.8.0
  • Document preview limitations
  • Add support for private repos
  • Amend tests
  • Check for rc attribute to exist
  • Upgrade default imperative image
  • Release clustergroup v0.8.1
  • Update pattern operator CRD
  • Update CRD from the operator
  • Bump actions/setup-python from 4 to 5
  • Release clustergroup v0.8.2
  • Update CRD from the operator
  • Small clarification in IIB
  • Switch imageDigestMirrors to AllowContactingSource
  • Upgrade ESO to v0.9.10
  • Add initial support for deploying private repos via CLI directly
  • Add support for private repo deployments via CLI
  • Fix placement of tokenSecret material
  • Fix placement of tokenSecret material in the right section
  • Upgrade ESO to v0.9.11
  • Make the container to run with the UID and GID of the user running pattern.sh
  • Error out nicely if podman is not present
  • Account for podman versions older than 4.3.0
  • Switch default gitops channel to gitops-1.11
  • Set the user's passwd entry inside the container
  • Upgrade helm to v3.13.2
  • Drop old patch around null subkeys
  • Do check for remote existance all the time
  • Run validate-prereq only when not in a container
  • Bump dorny/paths-filter from 2 to 3
  • Add support for parsing secrets into intermediate structure and creating k8s secret objects
  • Upgrade ESO to v0.9.12
  • Update vault image to 1.15.5-ubi
  • Use gitops-1.11 in acm as well
  • Small gitops channel cleanups
  • Upgrade namespaced argocd version to v1beta1
  • Stop using OpenShiftControllerManager lookups
  • Bump azure/setup-helm from 3 to 4
  • Upgrade ESO to v0.9.13
  • Upgrade vault to 1.15.6
  • Bind mount /etc/pki in the wrapper
  • Properly error out in preview-all when we cannot connect to the cluster
  • Only include values files if they do exist in preview.sh
  • Do not error out in preview when kustomize: true
  • Also pass EXTRA_PLAYBOOK_OPTS environment setting
  • Use oc kustomize
  • Print out application being parsed for easier debugging
  • Add .global.secretStore.backend in the clustergroup schema
  • Fix check for secretStore.backend
  • Introduce a generic parameter override via the EXTRA_ARGS env variable
  • Allow overriding the oc lookups in preview.sh
  • Always include values-global.yaml and values-.yaml
  • Add help and message clarifying that preview has certain limits
  • Add clustergroup support to preview target
  • Fix preview when the application's index name is not the same as the name attribute
  • Force strings in extraParametersNested
  • In ACM policies do not use $ARGOCD_APP_SOURCE_ variables*
  • Add support for custom CAs
  • Update tests
  • Support for issue #459
  • Drop unused piece of schema json
  • Small cleanup to remove unneeded log messages
  • Support for cluster-wide proxy
  • Update ESO to 0.9.14
  • Update CRD from operator v0.0.44
  • Expose main.experimentalCapabilities in operator-install
  • Release clustergroup v0.8.3
  • feat: add support for hive clusterdeployments creating spokes
  • test: regenerated tests after clusterdeployment commit
  • Support remote repoURL when previewing templates
  • test: updated test-cased and regeneated expectations
  • Moved CLUSTERGROUP declaration to restore make preview-% functionality
  • Namespace argocd.argoproj.io/managed-by label issue
  • Fixed indenting and duplicate entries in application-policies.yaml
  • Add main.experimentalCapabilities to values.schema.json
  • Release clustergroup v0.8.4
  • Move the CLUSTERGROUP env variable for previews inside its section
  • Fix up tests after upstream PR merge
  • Drop old comments
  • bug: Fix to generate OperatorGroup definition when namespaces definition has labels
  • Try fallbacks for /etc/pki when it does not exist
  • Fix for multiple OperatorGroup rendering
  • Updated tests for CI
  • Update ESO to 0.9.16
  • Updated vault-helm to v0.28.0
  • Use --recurse-submodules when cloning
  • Drop support for IIB on OCP 4.12
  • Stop saving stderr in a file when running skopeo
  • Rearchitect the code so we can print out the actual error
  • Check for prereqs for IIB
  • chore: added annotations controling gitops and fail for missing meta for clusterdeployments
  • chore: removed managedclusterset spec
  • Rework IIB loading support
  • Switch to gitops-1.12
  • Add overrides for MCE IIB
  • Update README for IIB changes
  • Release clustergroup v0.8.5
  • New global scope argocdServer section for values-global.yaml
  • Improve readme for ACM IIB
  • Drop gitopsspec from pattern's CR
  • Allow customizing the VP operator subscription
  • Add retries when checking oc version
  • Add an imperative-admin-sa service account
  • Added support to label/annotate nodes
  • Added support to enable user workloads in control plane nodes
  • Added full support for the scheduler
  • Simplified PR for auto approve install plans
  • Switch to registry.redhat.io for the initContainer image
  • Update for ACM chart to application-policies.yaml
  • Add extraParameters to values.schema.json
  • Update ESO to 0.9.18
  • Update vault to 1.16.2
  • Feat: Followup to definition of extraParameters under the main section of a values file.
  • Use golang-external-secrets for the acm hub-ca bits
  • Only do the acm hub ca policy when vault is the backend
  • fix: when using clusterdeployments, secrets should exist in the cluster-namespace
  • Force rolebindings as early as possible
  • bug: Invalid OperatorGroup generated when ommitting targetNamespaces
  • Fix CI issue
  • Actually use adminServiceAccountName for the auto approve job
  • Make sure that the if condition on chart split is not always true
  • Bump super-linter from 5 to 6
  • Drop some validations for now
  • Add some debugging to the chart split action
  • Use a specific git version when running git subtree split
  • Release clustergroup v0.8.6
  • Add a sudo to apt-get command
  • Add some READMEs in the individual charts
  • Fix super-linter issues and upgrade local super-linter target
  • Skip unreachable spokes when setting up vault
  • Add no_log to spokes initialization task
  • Drop initContainers variable and make it the default
  • Update tests after dropping initContainers
  • Release clustergroup v0.8.7
  • Upgrade ESO to v0.9.19
  • Update helm version in CI to 3.14.0
  • Drop imperative.initcontainers.gitinit
  • Have unseal work whenever we specifiy HEAD
  • make resourceExclusion configurable
  • Update tests after upgrading resourceExclusions tweak
  • Make resourcehealthchecks configurable
  • Update tests after upgrading resourceHealthChecks change
  • Do not quote $BRANCH variable
  • Fix initcontainer race on spokes
  • Release clustergroup v0.8.8
  • configure annotation based resource tracking
  • Drop imperative.volumes and imperative.volumemounts
  • Properly quote vault kv command
  • Make HUB work when spokes point to in hub cluster gitea
  • update tests
  • ACM chart version 0.0.2
  • golang-external-secrets chart version 0.0.4
  • hashicorp-vault chart version 0.0.2
  • Implement multi-source
  • Add tests for proper multisource support on spokes
  • Release clustergroup v0.8.9
  • Fix multisource indent error
  • Release clustergroup v0.8.10
  • Release acm v0.0.3
  • Fix missing chart field
  • Release acm v0.0.4
  • Update chart versions
  • Update vault-helm to v0.28.1 and vault to 1.17.2
  • Update ESO to 0.9.20
  • Release new hashicorp-vault and golang-external-secrets charts
  • Add Ansible playbook
  • Put the playbook in a more normal location for us
  • Exclude new playbook
  • Allow for choice in where file is written
  • Fix pki bind mount when using podman machine
  • Allow originRepo to be set via make install and main.git parameters
  • resolves #21 rag-llm-gitops
  • Fix vars that were erroneously dropped
  • Allow more flexibility with multiSourceConfig schema
  • Inject VALUES_SECRET env var
  • Add helmRepoUrl variable
  • Update letsencrypt to v0.1.1
  • Allow overriding gitops source on spokes
  • Update acm chart to v0.1.1
  • Update clustergroup chart to 0.8.11
  • Extend the schema for disconnected
  • Update clustergroup chart to 0.8.12
  • Update super-linter to v7
  • Fix action path
  • Fix action path v2
  • Upgrade ESO to v0.10.0
  • Upgrade vault to 1.17.3
  • Update hashicorp-vault to 0.1.2
  • Update golang-external-secrets to 0.1.2
  • Switch to gitops-1.13
  • Add a pushsecrets policy and vault path for ESO syncing
  • Fix PyInk warnings
  • Removed previous version of common to convert to subtree from https://github.com/validatedpatterns/common.git main
  • Simplify testing workflow for now

mbaldessari and others added 30 commits March 29, 2024 10:51
Add help and message clarifying that preview has certain limits
With this the preview all should be a lot more complete and useful.

Closes: validatedpatterns/common#452
Add clustergroup support to preview target
…name attribute

This covers the following case:
foobar:
  name: foo
  namespace: foo
  project: foo
  path: charts/all/foo

The preview.sh script is passed the name attribute of the application
`foo`. So now we first find the key which corresponds to the attribute
name `foo` and then use that when looking up the other attributes like
path, etc.

Closes: validatedpatterns/multicloud-gitops#351
Fix preview when the application's index name is not the same as the name attribute
Otherwise if we pass a boolean in the extraParametersNested we will get:

  spec.source.helm.parameters[10].value: Invalid value: "boolean":
    spec.source.helm.parameters[10].value in body must be of type string:
    "boolean"
Force strings in extraParametersNested
The reason for this is the following:
When changing the repo on the hub (by editing the pattern), the
expectation is that the repo change will replicate from the hub to the
spokes managed by ACM.

Today this is very unlikely to happen because changing the repo on the
hub will not change the policy and so ACM will not reapply it on the
spokes. (I believe there is like a daily repush that happens even when
the policy has not changed, but that is way too slow to be relied upon).

By using the actual variable the policy will actually change, ACM will
notice this and push the change on the spokes.

Found while testing disconnected mode.

I am not replacing them everywhere because I am not sure yet if there are
additional semantics in common/clustergroup that I am unaware of.
In ACM policies do not use $ARGOCD_APP_SOURCE_* variables
This feature relies on the VP operator version >= 0.0.44 in order to
work.

The way to enable this is to add a feature flag called 'initcontainers'
in the VP operator. Once this is enabled, we will detect this and take
over the all ArgoCD instances' definition and add initContainers which
will inject the CAs contained in the trusted-bundle-ca configmap and
also the openshift internal CA.

Testing protocol:

  1. (Operator 0.0.44) MCG deployment with experimentalCapabilities set
     to '' and using a github main upstream (i.e. without this PR)

  2. (Operator 0.0.44) MCG deployment with experimentalCapabilities set
     to 'initcontainers' and using a github diconnected common upstream
     (requiring a custom CA) (i.e. with this PR)

  3. (Operator 0.0.44) MCG deployment with experimentalCapabilities set
     to '' and using a github diconnected common upstream. (same as 1.2)
     and then set the initcontainer capability on the hub. Checked that
     the .global.experimentalCapabilities property replicated from hub to
     spoke and the initcontainers have been generated correctly

  3.1 (Operator 0.0.44) Change the repo from github to an internal one
      that does need the custom ca to work

  4. (Operator 0.0.43) Test an old operator with a newer common that
      contains this very branch

Note: Once we will make initcontainers a default feature of the operator
we will remove the ifs added in this PR and just make it the defaut
behaviour.
The changes here support the "Support for merging of namespaces, projects,
subscriptions and application in overrides/values-common.yaml #459" issue that was opened by
Northrop Grumman

Files that were changed are:
clustergroup/templates/_helpers.tpl
clustergroup/templates/core/namespaces.yaml
clustergroup/templates/core/operatorgroup.yaml
clustergroup/templates/plumbing/projects.yaml
clustergroup/values.schema.json
examples/values-example.yaml

The idea is that if you define the projects section, or the namespaces section, in two different
values files using a map construct we will be able to merge both definition of projects into
the final rendering of the manifests.

The new structure for projects is as follows:
```
clusterGroup:
  ...
  projects:
    project1:
```

The new structure for namespaces is as follows:
```
clusterGroup:
  ...
  namespaces:
    namespace1:
    open-cluster-management:
      labels:
        openshift.io/node-selector: ""
        kubernetes.io/os: linux
      annotations:
        openshift.io/cluster-monitoring: "true"
        owner: "namespace owner"
```
The user would need to choose to use a list or a hashmap object.  The user would not be able to use a
mix of hashes and list to describe projects or namespaces.
Not entirely sure how this slipped in.

Reported-by: Martin Jackson <[email protected]>
Drop unused piece of schema json
Small cleanup to remove unneeded log messages
If the clusterwide proxy object is configured, let's support it when
we clone the git repos for the imperative framework.
Expose main.experimentalCapabilities in operator-install
Co-authored-by: Alejandro Villegas <[email protected]>
Signed-off-by: Tomer Figenblat <[email protected]>
Co-authored-by: Alejandro Villegas <[email protected]>
Signed-off-by: Tomer Figenblat <[email protected]>
mbaldessari and others added 29 commits August 22, 2024 16:10
This is needed on spokes when installing in a disconnected environment
A couple of small updated and new releases
Release new hashicorp-vault and golang-external-secrets charts
It is supported all the way back to OCP 4.12

Tested on sno hub + spoke successfully
See the README for more details, but TLDR: you can use
`secret/pushsecrets` to push secrets from any node to the vault.
This secret can then be retrieved from either a different namespace
or a different cluster node.

Tested this with a pushsecret as follows:
```
apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
  name: pushsecret
  namespace: hello-world
spec:
  data:
    - conversionStrategy: None
      match:
        remoteRef:
          property: baz
          remoteKey: pushsecrets/testme
        secretKey: bar
  deletionPolicy: Delete
  refreshInterval: 10s
  secretStoreRefs:
    - kind: ClusterSecretStore
      name: vault-backend
  selector:
    secret:
      name: existing-secret
  updatePolicy: Replace
```

The above takes the property called `baz` of an existing secret called `existing-secret` in
the `hello-world` namespace and pushes it to the `secret/pushsecrets/testme` vault path.

Suggested-By: Chris Butler <[email protected]>

Closes: MBP-641
Add a pushsecrets policy and vault path for ESO syncing
git-subtree-dir: common
git-subtree-mainline: a03f7de
git-subtree-split: 5acb722
@mbaldessari mbaldessari merged commit a04d997 into validatedpatterns:main Sep 5, 2024
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants