Document using Altinity.Cloud with posthog helm charts

contents/docs/self-host/configure/environment-variables.md

@@ -33,11 +33,14 @@ Some variables here are default Django variables. This [Django Docs page](https:
 | `DATABASE_URL`| [Database URL](https://github.com/jacobian/dj-database-url#url-schema) pointing to your PostgreSQL instance.  | `postgres://localhost:5432/posthog` if PostHog is running in DEBUG or TEST mode, must be specified otherwise.
 | `DEBUG_QUERIES`| Whether debugging queries (ClickHouse) is enabled in the Command Palette.| `False`
 | `DEBUG`                    | Determines if PostHog should run in [DEBUG mode](https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-DEBUG). You can set this to a truthy value when developing, but disable this in production!  | `False` |
+| `CLICKHOUSE_DISABLE_EXTERNAL_SCHEMAS` | If set, disables using ProtoBuf schemas for kafka communication. Needs to be set when using an external ClickHouse service provider during initial deploy. | `False`
 | `DISABLE_PAID_FEATURE_SHOWCASING`| Whether any showcasing of a paid feature should be disabled. Useful if running a free open source version of PostHog and are not interested in premium functionality. | `False`
 | `DISABLE_SECURE_SSL_REDIRECT` | Disables automatic redirect from port 80 (HTTP) to port 443 (HTTPS).                           | `False`
 | `GITHUB_TOKEN`| GitHub personal access token, used to prevent rate limiting when using plugins and to allow installation of plugins from private repos                      | `None`
 | `GITLAB_TOKEN`| GitLab personal access token, used to prevent rate limiting when using plugins and to allow installation of plugins from private repos                      | `None`
 | `JS_URL`                   | URL used by Webpack for loading external resources like images and files.                         | `http://localhost:8234` if PostHog is running in DEBUG mode, must be specified otherwise.
+| `KAFKA_URL` | Address used by the application to contact kafka | `kafka://kafka`
+| `KAFKA_URL_FOR_CLICKHOUSE` | Address used by ClickHouse to read from kafka. Falls back to `KAFKA_URL` | `None`
 | `MATERIALIZE_COLUMNS_ANALYSIS_PERIOD_HOURS` | Diagnostic for what columns to materialize | `168`
 | `MATERIALIZE_COLUMNS_BACKFILL_PERIOD_DAYS` | How far back backfill materialized columns | `90`
 | `MATERIALIZE_COLUMNS_MAX_AT_ONCE` | How many columns to materialize at once | `10`

contents/docs/self-host/configure/using-altinity-cloud.md

@@ -0,0 +1,137 @@
+---
+title: Deploying ClickHouse using Altinity.Cloud
+sidebar: Docs
+showTitle: true
+---
+
+This document outlines how to deploy PostHog using Altinity Cloud ClickHouse clusters.
+
+## Prerequisites
+
+- Altinity.Cloud ClickHouse cluster:
+    - Minimum ClickHouse version: 21.8.13
+    - Single shard and no data replication
+    - No dashes (`-`) in cluster name
+- PostHog helm chart version >= 16.1.1
+- PostHog version >= 1.33.0
+
+## Deployment instructions
+
+PostHog uses Kafka to send data from the app to ClickHouse. For that reason, Kafka needs to be accessible for ClickHouse during deployment.
+
+### Deploying using external Kafka
+
+```yaml
+env:
+  - name: CLICKHOUSE_DISABLE_EXTERNAL_SCHEMAS
+    value: "1"
+
+kafka:
+  enabled: false
+
+externalKafka:
+  brokers:
+    - "broker-1.posthog.kafka.us-east-1.amazonaws.com:9094"
+    - "broker-2.posthog.kafka.us-east-1.amazonaws.com:9094"
+    - "broker-3.posthog.kafka.us-east-1.amazonaws.com:9094"
+
+clickhouse:
+  enabled: false
+
+externalClickhouse:
+  host: "somecluster.demo.altinity.cloud"
+  user: "admin"
+  password: "password"
+  cluster: "clustername"
+  secure: true
+```
+
+Read more on configuring external Kafka in the chart [here](https://posthog.com/docs/self-host/deploy/configuration#kafka)
+
+### Using internal Kafka
+
+Deployment using Kafka managed by Posthog Helm chart requires three steps:
+
+1. [Deploy helm](/docs/self-host) initially with the following values.yaml:
+
+```yaml
+kafka:
+  enabled: true
+  externalAccess:
+    enabled: true
+    service:
+      type: LoadBalancer
+      ports:
+        external: 9094
+    autoDiscovery:
+      enabled: true
+  serviceAccount:
+    create: true
+  rbac:
+    create: true
+
+
+clickhouse:
+  enabled: false
+
+redis:
+  enabled: false
+
+postgresql:
+  enabled: false
+
+pgbouncer:
+  enabled: false
+
+plugins:
+  enabled: false
+
+worker:
+  enabled: false
+
+web:
+  enabled: false
+
+events:
+  enabled: false
+
+migrate:
+  enabled: false
+```
+
+2. Get the external kafka IP via `kubectl get svc -n posthog | grep kafka-0-external`
+
+3. [Deploy posthog using helm](/docs/self-host) with new values.yaml (fill in placeholder values)
+
+```yaml
+env:
+  - name: KAFKA_URL_FOR_CLICKHOUSE
+    value: "kafka://KAFKA_IP:9094"
+  - name: CLICKHOUSE_DISABLE_EXTERNAL_SCHEMAS
+    value: "1"
+
+clickhouse:
+  enabled: false
+
+externalClickhouse:
+  host: "somecluster.demo.altinity.cloud"
+  user: "admin"
+  password: "password"
+  cluster: "clustername"
+  secure: true
+
+kafka:
+  enabled: true
+  externalAccess:
+    enabled: true
+    service:
+      type: LoadBalancer
+      ports:
+        external: 9094
+    autoDiscovery:
+      enabled: true
+  serviceAccount:
+    create: true
+  rbac:
+    create: true
+```

contents/docs/self-host/deploy/configuration.md

@@ -207,6 +207,11 @@ ClickHouse is the datastore system that does the bulk of heavy lifting with rega
 By default, ClickHouse is installed as a part of the chart, powered by [clickhouse-operator](https://github.com/Altinity/clickhouse-operator/). We are currently working to add the possibility to use an external ClickHouse service (see [issue #279](https://github.com/PostHog/charts-clickhouse/issues/279) for more info).
 
 
+#### Use an external service
+To use an external ClickHouse service, please set `clickhouse.enabled` to `false` and then configure the `externalClickhouse` values.
+
+Read more on how to deploy using Altinity Cloud [here](/docs/self-host/configure/using-altinity-cloud).
+
 #### Custom settings
 
 It's possible to pass custom settings to ClickHouse. This might be needed to e.g. set query time limits or increase max memory usable by clickhouse.

src/sidebars/sidebars.json

@@ -536,6 +536,10 @@
                                 {
                                     "name": "Helm chart configuration",
                                     "url": "/docs/self-host/deploy/configuration"
+                                },
+                                {
+                                    "name": "Deploying ClickHouse using Altinity.Cloud",
+                                    "url": "/docs/self-host/configure/using-altinity-cloud"
                                 }
                             ]
                         },