Skip to content

Commit

Permalink
Add volume expansion docs
Browse files Browse the repository at this point in the history
Signed-off-by: Tapajit Chandra Paul <[email protected]>
  • Loading branch information
tapojit047 committed Oct 23, 2024
1 parent 88e2e20 commit 356a441
Show file tree
Hide file tree
Showing 7 changed files with 538 additions and 0 deletions.
10 changes: 10 additions & 0 deletions docs/guides/druid/volume-expansion/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Volume Expansion
menu:
docs_{{ .version }}:
identifier: guides-druid-volume-expansion
name: Volume Expansion
parent: guides-druid
weight: 44
menu_name: docs_{{ .version }}
---
409 changes: 409 additions & 0 deletions docs/guides/druid/volume-expansion/guide.md

Large diffs are not rendered by default.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
56 changes: 56 additions & 0 deletions docs/guides/druid/volume-expansion/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
title: Druid Volume Expansion Overview
menu:
docs_{{ .version }}:
identifier: guides-druid-volume-expansion-overview
name: Overview
parent: guides-druid-volume-expansion
weight: 10
menu_name: docs_{{ .version }}
section_menu_id: guides
---

> New to KubeDB? Please start [here](/docs/README.md).
# Druid Volume Expansion

This guide will give an overview on how KubeDB Ops-manager operator expand the volume of various component of `Druid` like:. (Combined and Topology).

## Before You Begin

- You should be familiar with the following `KubeDB` concepts:
- [Druid](/docs/guides/druid/concepts/druid.md)
- [DruidOpsRequest](/docs/guides/druid/concepts/druidopsrequest.md)

## How Volume Expansion Process Works

The following diagram shows how KubeDB Ops-manager operator expand the volumes of `Druid` database components. Open the image in a new tab to see the enlarged version.

<figure align="center">
  <img alt="Volume Expansion process of Druid" src="/docs/guides/druid/volume-expansion/images/druid-volume-expansion.png">
<figcaption align="center">Fig: Volume Expansion process of Druid</figcaption>
</figure>

The Volume Expansion process consists of the following steps:

1. At first, a user creates a `Druid` Custom Resource (CR).

2. `KubeDB` Provisioner operator watches the `Druid` CR.

3. When the operator finds a `Druid` CR, it creates required number of `PetSets` and related necessary stuff like secrets, services, etc.

4. Each PetSet creates a Persistent Volume according to the Volume Claim Template provided in the petset configuration. This Persistent Volume will be expanded by the `KubeDB` Ops-manager operator.

5. Then, in order to expand the volume of the druid data components (ie. Historicals, MiddleManagers) of the `Druid`, the user creates a `DruidOpsRequest` CR with desired information.

6. `KubeDB` Ops-manager operator watches the `DruidOpsRequest` CR.

7. When it finds a `DruidOpsRequest` CR, it halts the `Druid` object which is referred from the `DruidOpsRequest`. So, the `KubeDB` Provisioner operator doesn't perform any operations on the `Druid` object during the volume expansion process.

8. Then the `KubeDB` Ops-manager operator will expand the persistent volume to reach the expected size defined in the `DruidOpsRequest` CR.

9. After the successful Volume Expansion of the related PetSet Pods, the `KubeDB` Ops-manager operator updates the new volume size in the `Druid` object to reflect the updated state.

10. After the successful Volume Expansion of the `Druid` components, the `KubeDB` Ops-manager operator resumes the `Druid` object so that the `KubeDB` Provisioner operator resumes its usual operations.

In the next docs, we are going to show a step-by-step guide on Volume Expansion of various Druid database components using `DruidOpsRequest` CRD.
16 changes: 16 additions & 0 deletions docs/guides/druid/volume-expansion/yamls/deep-storage-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: v1
kind: Secret
metadata:
name: deep-storage-config
namespace: demo
stringData:
druid.storage.type: "s3"
druid.storage.bucket: "druid"
druid.storage.baseKey: "druid/segments"
druid.s3.accessKey: "minio"
druid.s3.secretKey: "minio123"
druid.s3.protocol: "http"
druid.s3.enablePathStyleAccess: "true"
druid.s3.endpoint.signingRegion: "us-east-1"
druid.s3.endpoint.url: "http://myminio-hl.demo.svc.cluster.local:9000/"

34 changes: 34 additions & 0 deletions docs/guides/druid/volume-expansion/yamls/druid-cluster.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
apiVersion: kubedb.com/v1alpha2
kind: Druid
metadata:
name: druid-cluster
namespace: demo
spec:
version: 28.0.1
deepStorage:
type: s3
configSecret:
name: deep-storage-config
topology:
historicals:
replicas: 1
storage:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: standard
middleManagers:
replicas: 1
storage:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: standard
routers:
replicas: 1
deletionPolicy: Delete

13 changes: 13 additions & 0 deletions docs/guides/druid/volume-expansion/yamls/volume-expansion-ops.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: ops.kubedb.com/v1alpha1
kind: DruidOpsRequest
metadata:
name: dr-volume-exp
namespace: demo
spec:
type: VolumeExpansion
databaseRef:
name: druid-cluster
volumeExpansion:
historicals: 3Gi
middleManagers: 2Gi
mode: Online

0 comments on commit 356a441

Please sign in to comment.