Based on Kubernetes operators, this control plane for apache pinot is responsible for deploying and managing heterogenous apache pinot kubernetes clusters and its operations including schema, table and tenants management. This control plane is based on Dsoi-Spec and is built using operator-runtime. This is a radical new approach that brings ease of use and decouples application and kubernetes in a way that it becomes easier for day 2 operations. The underlying controllers are built on observed state (conditions) and not state machines.
- Installation of heterogeneous pinot clusters.
- Rolling Upgrades - Incremental
- Ordered Deployment
- Seperation of pinot specific configurations with k8s configurations.
- Table Management
- Schema Management
- Tenant Management (experimental)
helm repo add datainfra https://charts.datainfra.io
helm upgrade --install pinot-control-plane datainfra/pinot-control-plane
- Getting Started With Heterogeneous Pinot Clusters
- Getting Started With Schema Management
- Getting Started With Table Management
- Getting Started With Tenant Management
We believe that Kubernetes can serve as a control plane for any application, including those with data and stateful sets. While Helm charts are useful for configuration management, they don't maintain the state of the application. That's why we've built a control plane based on kubernetes operator pattern that acts as a bridge between your application's requirements and Kubernetes infrastructure.
Pinot control plane for k8s is specifically designed to improve the user experience of running Apache Pinot clusters. As a distributed database, Pinot can be challenging to run on Kubernetes without the right tools. Our project is based on the DSOI Spec and built using the Operator-Runtime library to provide a more user-friendly and Kubernetes-friendly experience.
- For questions and feedback please feel free to reach out to us on Slack ↗︎.
- For bugs, please create issue on GitHub ↗︎.
- For commercial support and consultation, please reach out to us at
[email protected]↗︎.
This project is based on the Kubernetes operator pattern, but it is not exclusively limited to this pattern. Given the complexity of Pinot, relying solely on Kubernetes operators may or may not be sufficient to effectively manage its operations. Our vision for the project is to create a comprehensive set of tools and utilities that enable seamless deployment and operation of Pinot on Kubernetes.
Helm is configuration management tool, it does not maintain the state of the application. When building controllers, there is clear abstraction and concern on
- who is responsible for applying configuration ?
- who is responsible for reconciling configuration ?
Helm can template out any yaml, in our case its CR's/operator deployment etc. Once configs are applied its the responsibility of the underlying controllers to reconcile the configuration to achieve desired state.
- The underlying controllers are based on conditions and NOT state machines. The status of objects is constructable by observation. This project is solely built on observed state, the underlying functions follow the k8s native pattern of
CreateOrUpdate. Each resource whether k8s native or in this case of pinot specific resources ie schema/table/tenant, is checked for existense, if not existed created else check for updates and updated. States taken into consideration are orginal, desired and current.
Apache®, Apache Pinot, Pinot® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. This project, pinot-control-plane-k8s, is not an Apache Software Foundation project.