Document using Altinity.Cloud with posthog helm charts (#3078)

* Document some more env vars * Add initial instructions on how to deploy Altinity Cloud * Fix typos * Update marketplace docs * Update contents/docs/self-host/configure/using-altinity-cloud.md Co-authored-by: Joe Martin <[email protected]> * Update contents/docs/self-host/configure/using-altinity-cloud.md Co-authored-by: Joe Martin <[email protected]> * Update contents/docs/self-host/configure/using-altinity-cloud.md Co-authored-by: Joe Martin <[email protected]> * Update contents/docs/self-host/configure/using-altinity-cloud.md Co-authored-by: Joe Martin <[email protected]> * Update contents/docs/self-host/configure/using-altinity-cloud.md Co-authored-by: Joe Martin <[email protected]> * Update contents/docs/self-host/configure/using-altinity-cloud.md Co-authored-by: Joe Martin <[email protected]> * Update contents/marketplace/altinity.md * Update contents/marketplace/altinity.md * Update contents/docs/self-host/deploy/configuration.md Co-authored-by: Joe Martin <[email protected]>
PostHog · Mar 3, 2022 · 0fab983 · 0fab983
1 parent 60a14f8
commit 0fab983
Show file tree

Hide file tree

Showing 5 changed files with 153 additions and 4 deletions.
diff --git a/contents/docs/self-host/configure/environment-variables.md b/contents/docs/self-host/configure/environment-variables.md
@@ -33,11 +33,14 @@ Some variables here are default Django variables. This [Django Docs page](https:
 | `DATABASE_URL`| [Database URL](https://github.com/jacobian/dj-database-url#url-schema) pointing to your PostgreSQL instance.  | `postgres://localhost:5432/posthog` if PostHog is running in DEBUG or TEST mode, must be specified otherwise.
 | `DEBUG_QUERIES`| Whether debugging queries (ClickHouse) is enabled in the Command Palette.| `False`
 | `DEBUG`                    | Determines if PostHog should run in [DEBUG mode](https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-DEBUG). You can set this to a truthy value when developing, but disable this in production!  | `False` |
+| `CLICKHOUSE_DISABLE_EXTERNAL_SCHEMAS` | If set, disables using ProtoBuf schemas for kafka communication. Needs to be set when using an external ClickHouse service provider during initial deploy. | `False`
 | `DISABLE_PAID_FEATURE_SHOWCASING`| Whether any showcasing of a paid feature should be disabled. Useful if running a free open source version of PostHog and are not interested in premium functionality. | `False`
 | `DISABLE_SECURE_SSL_REDIRECT` | Disables automatic redirect from port 80 (HTTP) to port 443 (HTTPS).                           | `False`
 | `GITHUB_TOKEN`| GitHub personal access token, used to prevent rate limiting when using plugins and to allow installation of plugins from private repos                      | `None`
 | `GITLAB_TOKEN`| GitLab personal access token, used to prevent rate limiting when using plugins and to allow installation of plugins from private repos                      | `None`
 | `JS_URL`                   | URL used by Webpack for loading external resources like images and files.                         | `http://localhost:8234` if PostHog is running in DEBUG mode, must be specified otherwise.
+| `KAFKA_URL` | Address used by the application to contact kafka | `kafka://kafka`
+| `KAFKA_URL_FOR_CLICKHOUSE` | Address used by ClickHouse to read from kafka. Falls back to `KAFKA_URL` | `None`
 | `MATERIALIZE_COLUMNS_ANALYSIS_PERIOD_HOURS` | Diagnostic for what columns to materialize | `168`
 | `MATERIALIZE_COLUMNS_BACKFILL_PERIOD_DAYS` | How far back backfill materialized columns | `90`
 | `MATERIALIZE_COLUMNS_MAX_AT_ONCE` | How many columns to materialize at once | `10`

diff --git a/contents/docs/self-host/configure/using-altinity-cloud.md b/contents/docs/self-host/configure/using-altinity-cloud.md
@@ -0,0 +1,137 @@
+---
+title: Deploying ClickHouse using Altinity.Cloud
+sidebar: Docs
+showTitle: true
+---
+
+This document outlines how to deploy PostHog using Altinity Cloud ClickHouse clusters.
+
+## Prerequisites
+
+- Altinity.Cloud ClickHouse cluster:
+    - Minimum ClickHouse version: 21.8.13
+    - Single shard and no data replication
+    - No dashes (`-`) in cluster name
+- PostHog helm chart version >= 16.1.1
+- PostHog version >= 1.33.0
+
+## Deployment instructions
+
+PostHog uses Kafka to send data from the app to ClickHouse. For that reason, Kafka needs to be accessible to ClickHouse during deployment.
+
+### Deploying using external Kafka
+
+```yaml
+env:
+  - name: CLICKHOUSE_DISABLE_EXTERNAL_SCHEMAS
+    value: "1"
+
+kafka:
+  enabled: false
+
+externalKafka:
+  brokers:
+    - "broker-1.posthog.kafka.us-east-1.amazonaws.com:9094"
+    - "broker-2.posthog.kafka.us-east-1.amazonaws.com:9094"
+    - "broker-3.posthog.kafka.us-east-1.amazonaws.com:9094"
+
+clickhouse:
+  enabled: false
+
+externalClickhouse:
+  host: "somecluster.demo.altinity.cloud"
+  user: "admin"
+  password: "password"
+  cluster: "clustername"
+  secure: true
+```
+
+Read more about how to configure external Kafka in the chart [in our deployment documentation](https://posthog.com/docs/self-host/deploy/configuration#kafka).
+
+### Using internal Kafka
+
+To deploy using a version of Kafka managed by the PostHog Helm chart, follow these three steps:
+
+1. [Deploy your Helm chart](/docs/self-host) initially with the following values.yaml:
+
+```yaml
+kafka:
+  enabled: true
+  externalAccess:
+    enabled: true
+    service:
+      type: LoadBalancer
+      ports:
+        external: 9094
+    autoDiscovery:
+      enabled: true
+  serviceAccount:
+    create: true
+  rbac:
+    create: true
+
+
+clickhouse:
+  enabled: false
+
+redis:
+  enabled: false
+
+postgresql:
+  enabled: false
+
+pgbouncer:
+  enabled: false
+
+plugins:
+  enabled: false
+
+worker:
+  enabled: false
+
+web:
+  enabled: false
+
+events:
+  enabled: false
+
+migrate:
+  enabled: false
+```
+
+2. Get the external Kafka IP via `kubectl get svc -n posthog | grep kafka-0-external`
+
+3. [Deploy PostHog using helm](/docs/self-host) with new values.yaml (fill in placeholder values)
+
+```yaml
+env:
+  - name: KAFKA_URL_FOR_CLICKHOUSE
+    value: "kafka://KAFKA_IP:9094"
+  - name: CLICKHOUSE_DISABLE_EXTERNAL_SCHEMAS
+    value: "1"
+
+clickhouse:
+  enabled: false
+
+externalClickhouse:
+  host: "somecluster.demo.altinity.cloud"
+  user: "admin"
+  password: "password"
+  cluster: "clustername"
+  secure: true
+
+kafka:
+  enabled: true
+  externalAccess:
+    enabled: true
+    service:
+      type: LoadBalancer
+      ports:
+        external: 9094
+    autoDiscovery:
+      enabled: true
+  serviceAccount:
+    create: true
+  rbac:
+    create: true
+```
diff --git a/contents/docs/self-host/deploy/configuration.md b/contents/docs/self-host/deploy/configuration.md
@@ -207,6 +207,11 @@ ClickHouse is the datastore system that does the bulk of heavy lifting with rega
 By default, ClickHouse is installed as a part of the chart, powered by [clickhouse-operator](https://github.com/Altinity/clickhouse-operator/). We are currently working to add the possibility to use an external ClickHouse service (see [issue #279](https://github.com/PostHog/charts-clickhouse/issues/279) for more info).
 
 
+#### Use an external service
+To use an external ClickHouse service, please set `clickhouse.enabled` to `false` and then configure the `externalClickhouse` values.
+
+Find out how to deploy PostHog using Altinity Cloud [in our deployment configuration docs](/docs/self-host/configure/using-altinity-cloud).
+
 #### Custom settings
 
 It's possible to pass custom settings to ClickHouse. This might be needed to e.g. set query time limits or increase max memory usable by clickhouse.

diff --git a/contents/marketplace/altinity.md b/contents/marketplace/altinity.md
@@ -8,21 +8,21 @@ hideLastUpdated: true
 
 <img src="/images/marketplace/alinity.png" alt="Altinity Logo" width="100" style={{float: 'right', margin: '0 0 1rem 1rem'}} />
 
-[Altinity](https://altinity.com) helps enterprises deliver real-time analytics based on ClickHouse anywhere and for any business purpose. Our offerings cover everything needed from project inception to prodution operation. 
+[Altinity](https://altinity.com) helps enterprises deliver real-time analytics based on ClickHouse anywhere and for any business purpose. Altinity covers everything needed from project inception to production operation.
 
 1. Altinity.Cloud platform offering fully supported and managed ClickHouse clusters in AWS and GCP
 2. 24/7 enterprise support for ClickHouse in any environment
 3. Training for analytic developers and administrators
 4. Altinity Stable builds for ClickHouse
 
-Altinity customers range from startups to Fortune 10 enterprises. 
+Altinity customers range from startups to Fortune 10 enterprises.
 
 ## Services offered
 
 ### Support
-- Coming soon (likely from PostHog 1.33.0): Altinity.Cloud ClickHouse backends for PostHog users who prefer to use ClickHouse as a hands-off service - get in touch now if you are interested
 - Enterprise support for users who prefer to operate ClickHouse in self-managed environments
+- Altinity.Cloud ClickHouse backends for PostHog users who prefer to use ClickHouse as a hands-off service. Find out more about using Altinity.Cloud to manage your self-hosted instance in [our deployment configuration docs.](/docs/self-host/configure/using-altinity-cloud).
 
 ## Contact
 
-[Speak to Altinity](mailto:[email protected])
+[Speak to Altinity](mailto:[email protected])
diff --git a/src/sidebars/sidebars.json b/src/sidebars/sidebars.json
@@ -536,6 +536,10 @@
                                 {
                                     "name": "Helm chart configuration",
                                     "url": "/docs/self-host/deploy/configuration"
+                                },
+                                {
+                                    "name": "Deploying ClickHouse using Altinity.Cloud",
+                                    "url": "/docs/self-host/configure/using-altinity-cloud"
                                 }
                             ]
                         },