From d710e80bd8f9ba2e212fd06400f1a936c0c943a6 Mon Sep 17 00:00:00 2001 From: Nicolas Bigler Date: Wed, 20 Dec 2023 11:52:30 +0100 Subject: [PATCH] Update documentation Signed-off-by: Nicolas Bigler --- .../ROOT/pages/explanations/data-usage.adoc | 2 + .../ROOT/pages/how-tos/installation.adoc | 30 +-- .../ROOT/pages/how-tos/multi-instance.adoc | 44 ---- docs/modules/ROOT/pages/index.adoc | 6 +- .../ROOT/pages/references/parameters.adoc | 219 ------------------ 5 files changed, 6 insertions(+), 295 deletions(-) delete mode 100644 docs/modules/ROOT/pages/how-tos/multi-instance.adoc delete mode 100644 docs/modules/ROOT/pages/references/parameters.adoc diff --git a/docs/modules/ROOT/pages/explanations/data-usage.adoc b/docs/modules/ROOT/pages/explanations/data-usage.adoc index 0b1cf2f..10c37b6 100644 --- a/docs/modules/ROOT/pages/explanations/data-usage.adoc +++ b/docs/modules/ROOT/pages/explanations/data-usage.adoc @@ -1,5 +1,7 @@ = Data Usage +WARNING: This page is outdated and the information below is not correct anymore. + This page gives a brief overview how resources data usage (e.g. buckets) is saved to the postgres billing database. == Data flow diff --git a/docs/modules/ROOT/pages/how-tos/installation.adoc b/docs/modules/ROOT/pages/how-tos/installation.adoc index cb6fa2a..5895b2f 100644 --- a/docs/modules/ROOT/pages/how-tos/installation.adoc +++ b/docs/modules/ROOT/pages/how-tos/installation.adoc @@ -2,13 +2,11 @@ == Requirements -This component requires https://github.com/appuio/component-appuio-cloud-reporting[component-appuio-cloud-reporting] and is installed into the same namespace. -This is required for this component to be able to access the billing database and its connection secrets. -It also requires an Exoscale IAMKey and a Kubernetes/OpenShift Service Account token in the target cluster to get resources (e.g. SOS buckets, Postgres, etc). +This controller is deployed with https://github.com/vshn/component-appcat[component-appcat]. == Sources -The data is matched from a k8s cluster and an Exoscale organization. +The data is matched from the local k8s cluster and the associated Exoscale organization. The Kubernetes Service Account token is required to have `get` permissions on `Namespaces` and to the following managed resources: * `buckets.exoscale.crossplane.io` @@ -20,27 +18,3 @@ The Kubernetes Service Account token is required to have `get` permissions on `N === Exoscale organization The Access Key (IAM Key) from an Exoscale organization is required to have read access across all managed resources (e.g. SOS buckets, Postgres, etc). - -=== K8s cluster -The access to the desired cluster from where the metrics should be collected has to be configured. The current configuration for each instance is done in the -`syn-tenant-repo` under `manifests//billing` - -== Example - -[source,yaml] ----- -applications: - - billing-collector-cloudservices - -parameters: - metrics_collector: - exoscale: - enabled: true - objectStorage: - schedule: '10 10,16,20 * * *' - dbaas: - enabled: true # must be manually enabled to scrape DBaas metrics - schedule: '*/15 * * * *' ----- - -See the xref:references/parameters.adoc[parameters] reference for a full list of parameters. diff --git a/docs/modules/ROOT/pages/how-tos/multi-instance.adoc b/docs/modules/ROOT/pages/how-tos/multi-instance.adoc deleted file mode 100644 index 7205459..0000000 --- a/docs/modules/ROOT/pages/how-tos/multi-instance.adoc +++ /dev/null @@ -1,44 +0,0 @@ -= Deploy multiple instances - -This guide provides an example how to deploy multiple instances of this component. - -== Requirements - - -. Prepare catalog by configuring 2 instances -+ -[source,yaml] ----- -applications: - - billing-collector-cloudservices as collector-exoscale-ch-gva-2 <1> - - billing-collector-cloudservices as collector-cloudscale-rma-0 <2> -parameters: - appuio_cloud_reporting: - namespace: 'appuio-cloud-reporting' - database: - name: 'reporting' - host: 'reporting-db.appuio-reporting.svc' - parameters: 'sslmode=disable' - password: 'passw0rd' - port: 5432 ----- -<1> Instance one with alias name collector-exoscale-ch-gva-2 -<2> Instance two with alias name collector-cloudscale-rma-0 -+ - -. Add relevant entries to Vault -+ -[source,bash] ----- -parent="clusters/kv/${TENANT_ID}/${CLUSTER_ID}" -instance_1="collector-exoscale-ch-gva-2" -instance_2="collector-cloudscale-rma-0" - -vault kv put "${parent}/billing-collector-cloudservices/${instance_1}" exoscale-key= exoscale-secret= cluster-server= cluster-token= -vault kv put "${parent}/billing-collector-cloudservices/${instance_2}" exoscale-key= exoscale-secret= cluster-server= cluster-token= ----- -+ - -. Compile and push the cluster catalog -. Wait until changes are applied -. Verify that the instances are up and configured correctly diff --git a/docs/modules/ROOT/pages/index.adoc b/docs/modules/ROOT/pages/index.adoc index 6782803..af47c81 100644 --- a/docs/modules/ROOT/pages/index.adoc +++ b/docs/modules/ROOT/pages/index.adoc @@ -1,9 +1,7 @@ = billing-collector-cloudservices -Batch job to sync usage data from the Exoscale and Cloudscale metrics APIs to the https://github.com/appuio/appuio-cloud-reporting/[APPUiO Cloud Reporting] database. +Controller to sync usage data from the Exoscale and Cloudscale metrics APIs to Odoo16 Metrics are collected taking into account product (e.g. `object-storage-storage:exoscale`), source (e.g. `exoscale:namespace`), tenant (organization) and date time. -On DBaaS, we also gather information by Plan. That can be seeing in the product. For example, in the case of a PostgreSQL database service, product may look like `postgres:exoscale:*:*:hobbyist-2`. - -See the xref:references/parameters.adoc[parameters] reference for further details. +On DBaaS, we also gather information by Plan. That can be seeing in the product. For example, in the case of a PostgreSQL database service, product may look like `appcat-exoscale-pg-hobbyist-2`. diff --git a/docs/modules/ROOT/pages/references/parameters.adoc b/docs/modules/ROOT/pages/references/parameters.adoc deleted file mode 100644 index f777453..0000000 --- a/docs/modules/ROOT/pages/references/parameters.adoc +++ /dev/null @@ -1,219 +0,0 @@ -= Parameters - -Because this component depends on the component https://hub.syn.tools/appuio-cloud-reporting/references/parameters.html[appuio-cloud-reporting], some parameters are taken from that component and are not individually configurable in this component. -In particular: - -* https://hub.syn.tools/appuio-cloud-reporting/references/parameters.html#_namespace[namespace] -* https://hub.syn.tools/appuio-cloud-reporting/references/parameters.html#_database[database] - -The following list includes only parameters of this component. -The parent key for all of them is `metrics_collector`. - -See xref:how-tos/installation.adoc[Installation] for a full example. - -== `namespace` - -[horizontal] -type:: string -default:: `${_instance}` - -The namespace in which to deploy this component. - -== `database` - -[horizontal] -type:: dictionary -default:: -+ -[source,yaml] ----- -database: ${appuio_cloud_reporting:database} ----- - -See https://hub.syn.tools/appuio-cloud-reporting/references/parameters.html#_database[appuio-cloud-reporting docs] for reference. - - -== `database_secret` - -[horizontal] -type:: dictionary -default:: -+ -[source,yaml] ----- -database_secret: ${appuio_cloud_reporting:database_secret} ----- - -See https://hub.syn.tools/appuio-cloud-reporting/references/parameters.html#_database_secret[appuio-cloud-reporting docs] for reference. - -== `database_env` - -[horizontal] -type:: dictionary -default:: -+ -[source,yaml] ----- -database_env: ${appuio_cloud_reporting:database_env} ----- - -See https://hub.syn.tools/appuio-cloud-reporting/references/parameters.html#_database_env[appuio-cloud-reporting docs] for reference. - - -== `extra_volumes` - -[horizontal] -type:: dictionary -default:: -+ -[source,yaml] ----- -extra_volumes: ${appuio_cloud_reporting:extra_volumes} ----- - -See https://hub.syn.tools/appuio-cloud-reporting/references/parameters.html#_extra_volumes[appuio-cloud-reporting docs] for reference. - - -== `images` - -[horizontal] -type:: dictionary -default:: https://github.com/vshn/billing-collector-cloudservices/blob/master/component/class/defaults.yml[See class/defaults.yml]. - -Dictionary containing the container images used by this component. - -== `exoscale.enabled` - -[horizontal] -type:: bool -default:: `false` - -Set to true if you want to deploy the exosacle metrics collector components. - -=== `exoscale.objectStorage.enabled` - -[horizontal] -type:: bool -default:: `true` - -Enable exoscale object storage cronjob. - -=== `exoscale.objectStorage.schedule` - -[horizontal] -type:: string -default:: `10 10,16,20 * * *` - -The cron schedule at which the object storage metrics collection job is spawned. - -=== `exoscale.dbaas.enabled` - -[horizontal] -type:: bool -default:: `false` - -Whether to add the DBaaS cronjob, too. - -=== `exoscale.dbaas.schedule` - -[horizontal] -type:: string -default:: `*/15 * * * *` - -The cron schedule at which the DBaaS metrics collection job is spawned, if enabled. - -=== `secrets.exoscale.credentials.stringData.EXOSCALE_API_KEY` - -[horizontal] -type:: string -default:: Required. - -The Exoscale API key. - -You need to get the token from the https://portal.exoscale.com[Exoscale Console]. -You need to select the correct account (token is limited to one account), choose "IAM" in the menu and generate a new key pair. - -=== `secrets.exoscale.credentials.stringData.EXOSCALE_API_SECRET` - -[horizontal] -type:: string -default:: Required. - -The Exoscale API secret. - -Second part of the Exoscale API credentials. -See EXOSCALE_API_KEY for instructions. - -=== `secrets.exoscale.credentials.stringData.KUBERNETES_SERVER_URL` - -[horizontal] -type:: string -default:: Required. - -The Kubernetes server URL. - -=== `secrets.exoscale.credentials.stringData.KUBERNETES_SERVER_TOKEN` - -[horizontal] -type:: string -default:: Required. - -The token to connect to a Kubernetes cluster. - -The Service Account connected to this token should have `get` and `list` permissions to the following managed resources: - -* `buckets.exoscale.crossplane.io` -* `postgresqls.exoscale.crossplane.io` - -== `cloudscale.enabled` - -[horizontal] -type:: bool -default:: `false` - -Set to true if you want to deploy the cloudscale metrics collector components. - -=== `cloudscale.objectStorage.enabled` - -[horizontal] -type:: bool -default:: `true` - -Enable cloudscale object storage cronjob. - -=== `cloudscale.objectStorage.schedule` - -[horizontal] -type:: string -default:: `10 4,10,16 * * *` - -The cron schedule at which the object storage metrics collection job is spawned. - -== `secrets.cloudscale.credentials.stringData.CLOUDSCALE_API_TOKEN` - -[horizontal] -type:: string -default:: Required. - -The cloudscale API token. - -You need to get the token from the https://control.cloudscale.ch[Cloudscale Control Panel]. -You need to select the correct Project (token is limited to one project), choose "API Tokens" in the menu and generate a new one. - -== `secrets.cloudscale.credentials.stringData.KUBERNETES_SERVER_URL` - -[horizontal] -type:: string -default:: Required. - -The Kubernetes server URL. - -== `secrets.cloudscale.credentials.stringData.KUBERNETES_SERVER_TOKEN` - -[horizontal] -type:: string -default:: Required. - -The token to connect to a Kubernetes cluster. - -The Service Account connected to this token should have `get` and `list` permissions to `buckets.cloudscale.crossplane.io` managed resource, and `get` and `list` permissions for namespaces.