Charmed Spark bundle as well as Spark Client Snap and spark8t form the Charmed Spark K8s solution that makes operating Spark workloads on Kubernetes seamless, secure, and production-ready.
This repository contains relevant artifacts for deploying and testing the Charmed Spark bundle in the following directories:
- python — contains the
spark-testpackage that provides utilities and fixtures to easily implement Charmed Spark tests. Find more information in its readme file - releases — contains the artifacts for the different channels, supporting the following backends:
- yaml — using Juju YAML bundles to easily deploy Charmed Spark on K8s
- terraform — using Terraform scripts to deploy Charmed Spark using the Juju Terraform provider
The Charmed Spark K8s bundle is also available on Charmhub.
The minimum requirements are as follows:
- 8 GB of HDD.
- 2 CPU threads per host.
- Access to the internet for downloading the Canonical Spark Image.
- Access to a Kubernetes cluster, e.g. MicroK8s or Charmed Kubernetes.
A production-ready solution might require more resources.
Charmed Spark supports native integration with the Canonical Observability Stack (COS). If you want to enable monitoring on top of Charmed Spark, make sure that you have a Juju model with COS correctly deployed and see the How to enable monitoring guide. To deploy COS on MicroK8s, follow the step-by-step tutorial. For more information about Charmed Spark and COS integration, refer to the COS documentation and the monitoring explanation section.
For information on security features and the use of cryptography, see the Security explanation page.
Security issues in the Charmed Spark K8s bundle can be reported through LaunchPad. Please do not file GitHub issues about security issues.
Canonical welcomes contributions to the Charmed Spark bundle. Please check out our contribution guidelines if you're interested in contributing to the solution. If you truly enjoy working on open-source projects like this one and you would like to be part of the OSS revolution, please don't forget to check out the career opportunities we have at Canonical.
The Charmed Spark bundle is free software, distributed under the Apache Software License, version 2.0.
See LICENSE for more information.