From 4245007efc0c7a665e327ff99e840b7cfec12123 Mon Sep 17 00:00:00 2001 From: David Morrison Date: Thu, 29 Feb 2024 14:53:00 -0800 Subject: [PATCH] first steps at getting started docs --- README.md | 73 ++-------- ctrl/controller.rs | 1 - docs/adv/autoscaling.md | 92 ++++++++++++ docs/adv/metrics.md | 8 ++ docs/{ => components}/sk-ctrl.md | 0 docs/{ => components}/sk-driver.md | 4 +- docs/{ => components}/sk-tracer.md | 0 docs/{ => components}/skctl.md | 22 ++- docs/{ => dev}/api_changes.md | 0 docs/{ => dev}/contributing.md | 0 docs/faq.md | 8 ++ docs/intro/concepts.md | 45 ++++++ docs/intro/installation.md | 223 +++++++++++++++++++++++++++++ docs/intro/running.md | 129 +++++++++++++++++ docs/mkdocs.yml | 23 +-- 15 files changed, 547 insertions(+), 81 deletions(-) create mode 100644 docs/adv/autoscaling.md create mode 100644 docs/adv/metrics.md rename docs/{ => components}/sk-ctrl.md (100%) rename docs/{ => components}/sk-driver.md (94%) rename docs/{ => components}/sk-tracer.md (100%) rename docs/{ => components}/skctl.md (88%) rename docs/{ => dev}/api_changes.md (100%) rename docs/{ => dev}/contributing.md (100%) create mode 100644 docs/faq.md create mode 100644 docs/intro/concepts.md create mode 100644 docs/intro/installation.md create mode 100644 docs/intro/running.md diff --git a/README.md b/README.md index d8dd7e5a..1e7771f2 100644 --- a/README.md +++ b/README.md @@ -30,69 +30,16 @@ This package provides the following components: [![Watch the video](https://img.youtube.com/vi/Q1XpH1H4It8/hqdefault.jpg)](https://www.youtube.com/watch?v=Q1XpH1H4It8) -## Installation +## Documentation -### Prerequisites +Full [documentation for SimKube](https://appliedcomputing.io/docs/simkube/index.html) is available on Applied +Computing's website. Here are some quick links to select topics: -The following prereqs are required for all components: - -- Rust >= 1.71 (needed if you want to build outside of Docker) -- Docker -- kubectl >= 1.27 -- Kubernetes >= 1.27 - -Additional prerequisites are necessary for your simulation cluster: - -- [KWOK](https://kwok.sigs.k8s.io) >= 0.4.0 -- [CertManager](https://cert-manager.io) for setting up mutating webhook certificates -- [The Promtheus operator](https://github.com/prometheus-operator/prometheus-operator); we recommend configuring this - via the [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) project - -### Optional Prerequisites - -SimKube uses [🔥Config](https://github.com/acrlabs/fireconfig) to generate Kubernetes manifests from definitions located -in `./k8s/`. If you want to use this mechanism for generating Kubernetes manifests, you will need to install the -following additional dependencies: - -- Python 3.10 -- Python Poetry (https://python-poetry.org/docs/) - -Additionally, if you want to run SimKube on a local development cluster, [kind](https://kind.sigs.k8s.io) >= 0.19 is the -supported tooling for doing so. - -If you want to test autoscaling, SimKube currently supports either the [Kubernetes Cluster Autoscaler](https://github.com/kubernetes/autoscaler) -or [Karpenter](https://karpenter.sh). You will need to install and configure these applications to use the -corresponding KWOK provider. For the Kubernetes Cluster Autoscaler, a KWOK [cloud provider](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler/cloudprovider/kwok) -is available, and for Karpenter, a basic [KWOK provider](https://github.com/kubernetes-sigs/karpenter/tree/main/kwok) is -used. Configuring these applications in your environment is beyond the scope of this documentation. - -If you intend to save metrics or data from a simulation, you will need to configure Prometheus with a [remote write -endpoint](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write). One option here is -the [prom2parquet writer](https://github.com/acrlabs/prom2parquet). See [the docs](docs/sk-ctrl.md) for more -information on how to set this up. - -### Building - -To build all SimKube artifacts for the first time run: - -- `git submodule init && git submodule update` -- `make build` from the root of this repository. - -For all subsequent builds of SimKube artifacts, run only `make build` from the root of this repository. - -### Docker images - -To build and push Docker images for all the artifacts, run `DOCKER_REGISTRY=path_to_your_registry:5000 make image` - -### Deploying - -You will need a KUBECONFIG file with cluster admin permissions; `make run` will use 🔥Config to generate the Kubernetes -manifests and deploy all SimKube artifacts to the specified cluster. - -### Cleaning up - -All build artifacts are placed in the `.build/` directory. You can remove this directory or run `make clean` to clean -up. +- [Installation](https://appliedcomputing.io/docs/simkube/intro/installation.html) +- [Autoscaling](http://appliedcomputing.io/docs/simkube/adv/autoscaling.html) +- [Metrics Collection](http://appliedcomputing.io/docs/simkube/adv/metrics..html) +- [Component Reference](http://appliedcomputing.io/docs/simkube/sk-ctrl.html) +- [Developing SimKube](http://appliedcomputing.io/docs/simkube/dev/contributing.html) ## Contributing @@ -121,7 +68,3 @@ default](https://docs.github.com/en/site-policy/github-terms/github-terms-of-ser > Due to the uncertain nature of copyright and IP law, this repository does not accept contributions that have been all > or partially generated with GitHub Copilot or other LLM-based code generation tools. Please disable any such tools > before authoring changes to this project. - -### Contributor's Guide - -Please see the [Contributor's Guide](./docs/contributing.md) for more information on setting up and building SimKube. diff --git a/ctrl/controller.rs b/ctrl/controller.rs index 6aec0bc0..2d1886c2 100644 --- a/ctrl/controller.rs +++ b/ctrl/controller.rs @@ -199,7 +199,6 @@ pub(super) async fn cleanup(ctx: &SimulationContext, sim: &Simulation) { } } if let Err(e) = prom_api.delete(&ctx.prometheus_name, &Default::default()).await { - println!("{e:?}"); if matches!(e, Api(ErrorResponse { code: 404, .. })) { warn!("prometheus object not found; maybe already cleaned up?"); } else { diff --git a/docs/adv/autoscaling.md b/docs/adv/autoscaling.md new file mode 100644 index 00000000..619bad7f --- /dev/null +++ b/docs/adv/autoscaling.md @@ -0,0 +1,92 @@ + + +# Autoscaling with SimKube + +When running your simulations, you probably don't want to have to manually create a bunch of KWOK nodes every time. +Fortunately, both [Cluster Autoscaler](https://github.com/kubernetes/autoscaler) and [Karpenter](https://karpenter.sh) +(the two most popular cluster autoscalers for Kubernetes) support KWOK, which means you can have them autoscaler your +simulated cluster so you don't have to manually create virtual nodes. + +## Cluster Autoscaler instructions + +You will need to run Cluster Autoscaler with the `--cloud-provider kwok` argument. The KWOK Cluster Autoscaler provider +expects two ConfigMaps to be present; the first tells the KWOK cloudprovider what to use for its Cluster Autoscaler +NodeGroups: + +```yaml +# provider-config.yml +apiVersion: v1 +kind: ConfigMap +metadata: + name: kwok-provider-config + namespace: kube-system +data: + config: | + --- + apiVersion: v1alpha1 + readNodesFrom: configmap + nodegroups: + fromNodeLabelKey: "kwok-nodegroup" + configmap: + name: kwok-provider-templates +``` + +The second enumerates the node types that the KWOK cloudprovider supports: + +```yaml +# provider-templates.yml +apiVersion: v1 +kind: ConfigMap +metadata: + name: kwok-provider-templates + namespace: kube-system +data: + templates: | + --- + apiVersion: v1 + kind: List + items: + - apiVersion: v1 + kind: Node + metadata: + annotations: + kwok.x-k8s.io/node: fake + kowk-nodegroup: node-group-1 + labels: + node.kubernetes.io/instance-type: c5d.9xlarge + topology.kubernetes.io/zone: us-west-1a + type: virtual + status: + allocatable: + cpu: 31 + ephemeral-storage: 900Gi + memory: 71Gi + pods: 110 + capacity: + cpu: 36 + ephemeral-storage: 900Gi + memory: 72Gi + pods: 110 +``` + +The KWOK cloudprovider will automatically apply a `kwok-provider: true` taint to the nodes it generates with a +`NoSchedule` effect on them. SimKube will likewise apply the corresponding toleration to the virtual pods it creates. + +For more information on running and configuring KWOK for Cluster Autoscaler, see the +[README](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler/cloudprovider/kwok). + +## Karpenter instructions + +The core [karpenter repo](https://github.com/kubernetes-sigs/karpenter) includes a KWOK provider for karpenter. There +are some initial instructions in there for installing the karpenter+KWOK binary into your cluster. Once it's installed, +it will automatically use KWOK to scale up nodes in the cluster just like Cluster Autoscaler. As with Cluster +Autoscaler, KWOK applies the `kwok-provider=true:NoSchedule` taint to the nodes it creates. + +> [!NOTE] +> Unlike Cluster Autoscaler, karpenter does not take in a list of Kubernetes Node specs to determine what instances it +> launches. Instead, it uses a hard-coded list of "generic" instance types which roughly map to standard instance +> offerings by the major cloud providers. There is an [open PR](https://github.com/kubernetes-sigs/karpenter/pull/1048) +> to enable configuring node types via an injected file. diff --git a/docs/adv/metrics.md b/docs/adv/metrics.md new file mode 100644 index 00000000..e4ffc17e --- /dev/null +++ b/docs/adv/metrics.md @@ -0,0 +1,8 @@ + + +# Metrics Collection and Data Analysis + +Coming Soon diff --git a/docs/sk-ctrl.md b/docs/components/sk-ctrl.md similarity index 100% rename from docs/sk-ctrl.md rename to docs/components/sk-ctrl.md diff --git a/docs/sk-driver.md b/docs/components/sk-driver.md similarity index 94% rename from docs/sk-driver.md rename to docs/components/sk-driver.md index db871315..7c103e6c 100644 --- a/docs/sk-driver.md +++ b/docs/components/sk-driver.md @@ -14,7 +14,7 @@ selectors, and tolerations to ensure that the simulated pods end up on virtual n ``` Usage: sk-driver [OPTIONS] --sim-name --sim-root --virtual-ns-prefix \ - --cert-path --key-path --trace-path + --cert-path --key-path --trace-mount-path Options: --sim-name @@ -23,7 +23,7 @@ Options: --admission-webhook-port [default: 8888] --cert-path --key-path - --trace-path + --trace-mount-path -v, --verbosity [default: info] -h, --help Print help ``` diff --git a/docs/sk-tracer.md b/docs/components/sk-tracer.md similarity index 100% rename from docs/sk-tracer.md rename to docs/components/sk-tracer.md diff --git a/docs/skctl.md b/docs/components/skctl.md similarity index 88% rename from docs/skctl.md rename to docs/components/skctl.md index 41b47cca..7edd70e3 100644 --- a/docs/skctl.md +++ b/docs/components/skctl.md @@ -110,7 +110,11 @@ that isn't accepted or is parsed incorrectly, please [file an issue](https://git ``` run a simulation -Usage: skctl run [OPTIONS] --name +Usage: skctl run [OPTIONS] --name [DURATION] + +Arguments: + [DURATION] + duration of the simulation Options: --name @@ -121,6 +125,16 @@ Options: [default: simkube] + --metrics-namespace + namespace to launch monitoring utilities in + + [default: monitoring] + + --metrics-service-account + service account with monitoring permissions + + [default: prometheus-k8s] + --trace-file location of the trace file for sk-driver to read @@ -136,11 +150,9 @@ Options: ## skctl snapshot ``` -Usage: skctl snapshot [OPTIONS] --config-file +take a point-in-time snapshot of a cluster (does not require sk-tracer to be running) -Arguments: - - duration of the generated trace file +Usage: skctl snapshot [OPTIONS] --config-file Options: -c, --config-file diff --git a/docs/api_changes.md b/docs/dev/api_changes.md similarity index 100% rename from docs/api_changes.md rename to docs/dev/api_changes.md diff --git a/docs/contributing.md b/docs/dev/contributing.md similarity index 100% rename from docs/contributing.md rename to docs/dev/contributing.md diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 00000000..0cee2f5d --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,8 @@ + + +# Frequently Asked Questions + +Coming Soon diff --git a/docs/intro/concepts.md b/docs/intro/concepts.md new file mode 100644 index 00000000..bf2fc929 --- /dev/null +++ b/docs/intro/concepts.md @@ -0,0 +1,45 @@ + + +# SimKube Concepts + +SimKube is designed to allow users to simulate the behaviour of Kubernetes control plane components in a safe, isolated +local environment. It is a "record-and-replay" simulator, which means that users can record the behaviour of a production +cluster and then save that data for future analysis. Below we describe some of the key concepts of SimKube + +## What components are simulated? + +Typically when we talk about the Kubernetes control plane, we are talking about the API server, scheduler, and +controller manager. SimKube expands this definition to include anything that can impact the behaviour of a cluster, +including projects like Cluster Autoscaler, descheduler, and others. + +SimKube accomplishes this by running in a cluster with a real control plane; however, all pod behaviours are mocked out +using [Kubernetes WithOut Kubelet (KWOK)](https://kwok.sigs.k8s.io). This means that anything that happens _inside_ a +pod is effectively out of scope of the simulation. KWOK does have utilities to mock out some aspects of pod lifecycle, +but these are not (currently) supported by SimKube. Crucially, this means that simulations that rely on the Horizontal +Pod Autoscaler (for example) will not currently work. + +Note also that, unlike some simulation solutions, we are not mocking out any aspects of the control plane. This means +that simulations of cluster behaviour take place in real-time, and we do not have any hooks into or control over what +messages are seen by various control plane components. Thus, running the exact same simulation repeatedly may yield +different results on each run, depending on timing fluctuations and other challenges of distributed systems. + +## How does it work? + +SimKube has a number of components that it uses to record data and run simulations: + +- _Tracer_: The `sk-tracer` program is a lightweight pod that runs in the cluster you wish to record the behaviour of. + It saves cluster events into a in-memory event stream, that is, a timeline of "important" changes in the cluster. You + can configure `sk-tracer` to tell it what events you consider important. Subsets of this event stream can be saved + into a _trace file_, which can be replayed later in a simulated environment. +- _Controller_: The simulation controller `sk-ctrl` runs in a separate Kubernetes cluster and is responsible for setting + up simulations in that cluster. The separate cluster can be a local cluster running on your laptop using + [kind](https://kind.sigs.k8s.io), or it can be a test cluster running in the cloud. The only requirement is that the + simulation must have all the components present that you wish to simulate. The controller watches for `Simulation` + custom resources to configure and start a new simulation run. It sets up metrics collection and other required tools, + and then creates a simulation driver Job, which actually reads the specified trace file and replays the events within + against the simulated cluster. +- _CLI_: SimKube comes with an CLI utility called `skctl`, which can be used to export trace data from the cluster under + study, as well as running new simulations in your simulated environment. diff --git a/docs/intro/installation.md b/docs/intro/installation.md new file mode 100644 index 00000000..8d4bc9c5 --- /dev/null +++ b/docs/intro/installation.md @@ -0,0 +1,223 @@ + + +# Installing SimKube + +This guide will walk you through installing the various SimKube components + +## Prerequisites + +The following prereqs are required for all components: + +- Rust >= 1.71 (needed if you want to build outside of Docker) +- Docker +- kubectl >= 1.27 +- Kubernetes >= 1.27 + +Additional prerequisites are necessary for your simulation cluster: + +- [KWOK](https://kwok.sigs.k8s.io) >= 0.4.0 +- [CertManager](https://cert-manager.io) for setting up mutating webhook certificates +- [The Promtheus operator](https://github.com/prometheus-operator/prometheus-operator); we recommend configuring this + via the [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) project + +### Optional Prerequisites + +SimKube uses [🔥Config](https://github.com/acrlabs/fireconfig) to generate Kubernetes manifests from definitions located +in `./k8s/`. If you want to use this mechanism for generating Kubernetes manifests, you will need to install the +following additional dependencies: + +- Python 3.10 +- Python Poetry (https://python-poetry.org/docs/) + +Additionally, if you want to run SimKube on a local development cluster, [kind](https://kind.sigs.k8s.io) >= 0.19 is the +supported tooling for doing so. + +If you want to test autoscaling, SimKube currently supports either the [Kubernetes Cluster Autoscaler](https://github.com/kubernetes/autoscaler) +or [Karpenter](https://karpenter.sh). You will need to install and configure these applications to use the +corresponding KWOK provider. For the Kubernetes Cluster Autoscaler, a KWOK [cloud provider](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler/cloudprovider/kwok) +is available, and for Karpenter, a basic [KWOK provider](https://github.com/kubernetes-sigs/karpenter/tree/main/kwok) is +used. See [Autoscaling](../adv/autoscaling.md) for more information on configuring these tools. + +If you intend to save metrics or data from a simulation, you will need to configure Prometheus with a [remote write +endpoint](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write). One option here is +the [prom2parquet writer](https://github.com/acrlabs/prom2parquet). See [Metrics Collection](../adv/metrics.md) for more +information on how to set this up. + +## Building SimKube + +To build all SimKube artifacts for the first time run: + +- `git submodule init && git submodule update` +- `make build` from the root of this repository. + +For all subsequent builds of SimKube artifacts, run only `make build` from the root of this repository. + +### Docker images + +To build and push Docker images for all the artifacts, run `DOCKER_REGISTRY=path_to_your_registry:5000 make image` + +### Cleaning up + +All build artifacts are placed in the `.build/` directory. You can remove this directory or run `make clean` to clean +up. + +## Configuring your simulation cluster + +### Local cluster via kind + +This section explains how to create a [`kind`](https://kind.sigs.k8s.io) cluster on your local machine for running simulations. +If you have a pre-existing Kubernetes cluster that you will be using for your simulation environment, you can skip this +step. + +From the `kind` website: + +> kind is a tool for running local Kubernetes clusters using Docker container “nodes”. kind was primarily designed for +> testing Kubernetes itself, but may be used for local development or CI. + +You'll want to create a kind cluster with at least two nodes; moreover, you need to provide the cluster with access to a +directory on your local hard drive from which it can read trace files. Here's an example configuration: + +```yaml +# kind.yml +kind: Cluster +apiVersion: kind.x-k8s.io/v1alpha4 +nodes: + - role: control-plane + - role: worker + extraMounts: + - hostPath: /tmp/kind-node-data + containerPath: /data +``` + +If you are pushing the SimKube docker images to a local docker registry, you will additionally need to follow the steps +in [Create A Cluster and Registry](https://kind.sigs.k8s.io/docs/user/local-registry/) to enable `kind` to access your +images. + +Create the cluster by running + +``` +> kind create cluster --name simkube --config kind.yml +``` + +### Install Required Dependencies + +**KWOK**: + +``` +> KWOK_REPO=kubernetes-sigs/kwok +> KWOK_LATEST_RELEASE=$(curl "https://api.github.com/repos/${KWOK_REPO}/releases/latest" | jq -r '.tag_name') +> kubectl apply -f "https://github.com/${KWOK_REPO}/releases/download/${KWOK_LATEST_RELEASE}/kwok.yaml" +> kubectl apply -f "https://github.com/${KWOK_REPO}/releases/download/${KWOK_LATEST_RELEASE}/stage-fast.yaml" +``` + +**Prometheus Operator**: + +``` +> git clone https://github.com/prometheus-operator/kube-prometheus.git +> cd kube-prometheus +> kubectl create -f manifests/setup +> until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ""; done +No resources found # this message is expected +> kubectl create -f manifests/ +``` + +**cert-manager**: + +``` +> kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.3/cert-manager.yaml +> kubectl wait --for=condition=Ready -l app=webhook -n cert-manager pod --timeout=60s +``` + +We're going to use self-signed certificates for this, so apply the following file to your cluster: + +```yaml +# self-signed.yml +apiVersion: cert-manager.io/v1 +kind: ClusterIssuer +metadata: + name: selfsigned + namespace: kube-system +spec: + selfSigned: {} +``` + +``` +> kubectl apply -f self-signed.yml +``` + +## Deploying SimKube + +### Installing `sk-tracer` + +The SimKube tracer runs in a real cluster and collects data about changes to objects in that cluster. You can configure +what objects it watches via a config file. Here is an example config file you can use to watch changes to Deployments, Jobs, +and StatefulSets: + +```yaml +trackedObjects: + apps/v1.Deployment: + podSpecTemplatePath: /spec/template + batch/v1.Job: + podSpecTemplatePath: /spec/template + apps/v1.StatefulSet: + podSpecTemplatePath: /spec/template +``` + +> [!NOTE] +> SimKube does some sanitization of the resources it watches, which is why it needs to know where the +> `podSpecTemplatePath` is; especially for custom resources, the path to the `podSpecTemplate` is not necessarily +> standard or well-known. In a future version of SimKube we'll make this parameter optional for all "standard" +> Kubernetes objects. + +`sk-tracer` needs an RBAC policy that grants "get", "list" and "watch" access to all configured objects in the cluster, +as well as pods. For example, if you use the above configuration, you will need the following RBAC policy attached to +the service account used by `sk-tracer`: + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: sk-tracer +rules: +- apiGroups: [""] + resources: ["pods"] + verbs: ["get", "watch", "list"] +- apiGroups: ["apps/v1"] + resources: ["deployment", "statefulset"] + verbs: ["get", "watch", "list"] +- apiGroups: ["batch/v1"] + resources: ["job"] + verbs: ["get", "watch", "list"] +``` + +SimKube currently uses [🔥Config](https://github.com/acrlabs/fireconfig) to generate Kubernetes manifests. You can +generate the required manifests for all SimKube components by running `make k8s` from the root of this repository. To +install `sk-tracer` into your cluster, run + +``` +> kubectl apply -f .build/manifests/0000-global.k8s.yaml +> kubectl apply -f .build/manifests/0001-sk-tracer.k8s.yaml +``` + +> [!WARNING] +> The generated manifests are fairly primitive right now and you may need to customize them in order to get them to +> install in your environment. We also don't (currently) have any Helm charts available, but we'd welcome a +> contribution! (See this [GitHub issue](https://github.com/acrlabs/simkube/issues/97)) + +### Installing `sk-ctrl` + +The SimKube controller just needs the SimKube custom resources installed in the target environment, and needs no other +configuration. After running `make k8s`, run the following commands to install everything: + +``` +> kubectl apply -f .build/manifests/raw +> kubectl apply -f .build/manifests/0000-global.k8s.yaml +> kubectl apply -f .build/manifests/0002-sk-ctrl.k8s.yaml +``` + +The SimKube controller needs, at a minimum, write access for all of the objects that it will be simulating. In theory, +since this is an isolated (or potentially even local) environment, it should be safe to give it `cluster-admin`, which +is probably the easiest way to configure it. diff --git a/docs/intro/running.md b/docs/intro/running.md new file mode 100644 index 00000000..d6c594d8 --- /dev/null +++ b/docs/intro/running.md @@ -0,0 +1,129 @@ + + +# Running your first simulation + +Once you've gone through the steps to install SimKube in your environment, you're ready to run your first simulation! + +## Step 1: Collect a trace + +Use the `skctl` CLI tool to collect a trace from your production cluster (where you're running `sk-tracer`): + +``` +> kubectl port-forward -n simkube pod/sk-tracer-depl-6d559b799-ln8gk 7777:7777 +> skctl export +``` + +Alternately, if you don't have `sk-tracer` running anywhere, you can generate a "point-in-time" snapshot of your +production cluster with the following command: + +``` +> skctl snapshot -c config.yml +``` + +The config file referenced should be in the same format as expected by `sk-tracer`. Here's a basic one you can use: + +```yaml +# config.yml +trackedObjects: + apps/v1.Deployment: + podSpecTemplatePath: /spec/template +``` + +## Step 2: Copy the trace into your cluster + +> [!NOTE] +> Yes, we know this step is very annoying. We'll fix it soon---the plan is that the tracer will save the trace to a +> blob store like S3, and the simulation driver will download the trace from the store, so you won't have to do this +> step. + +The Simulation driver needs to have _local_ access to the generated trace file. If you're running a local kind cluster +as configured in the [installation](installation.md) section, you can copy the trace file into `/tmp/kind-node-data`. +Otherwise, you'll need to upload it to the correct place in your Kubernetes cluster. + +## Step 3: Create some virtual nodes + +SimKube is going to create a bunch of fake pods during the simulation run, and it will need someplace to schedule them. +We're going to create a virtual node, managed by KWOK, for this. + +```yaml +# node.yml +apiVersion: v1 +kind: Node +metadata: + annotations: + kwok.x-k8s.io/node: fake + labels: + node.kubernetes.io/instance-type: c5d.9xlarge + topology.kubernetes.io/zone: us-west-1a + type: virtual + name: fake-node-1 +spec: + taints: + - effect: NoSchedule + key: kwok-provider + value: "true" +status: + allocatable: + cpu: 35 + ephemeral-storage: 900Gi + memory: 71Gi + pods: 110 + capacity: + cpu: 36 + ephemeral-storage: 900Gi + memory: 72Gi + pods: 110 +``` + +``` +> kubectl apply -f node.yml +``` + +Now, you should see your node appear in the list of cluster nodes, posting a "Ready" status: + +``` +> kubectl get nodes +NAME STATUS ROLES AGE VERSION +fake-node-1 Ready 82s kwok-v0.5.1 +simkube-control-plane Ready control-plane 12m v1.27.3 +simkube-worker Ready 12m v1.27.3 +``` + +## Step 4: Run your simulation! + +``` +> skctl run --name my-first-simulation +5m +running simulation my-first-simulation +``` + +You should see that the simulation object has been created: + +``` +> kubectl get simulation my-first-simulation +NAME START TIME END TIME STATE +testing Initializing +``` + +During the "Initializing" phase, `sk-ctrl` is setting up a temporary high-resolution Prometheus pod to scrape data from +the simulation, as well as configuring other needed components. After 30-50 seconds, you should see the simulation +transition to "Running": + +``` +> kubectl get simulation my-first-simulation +NAME START TIME END TIME STATE +testing 2024-03-01T04:58:48Z Running +``` + +Once your simulation is over, it should move into the "Finished" state: + +``` +> kubectl get simulation my-first-simulation +NAME START TIME END TIME STATE +testing 2024-03-01T04:58:48Z 2024-03-01T04:59:06Z Finished +``` + +Congratulations! You did it! Read on to learn more about how you can do some more advanced things in your simulated +environment. diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index dbd81f92..75e40de7 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -1,12 +1,19 @@ --- nav: - Home: README.md - - Usage: - - Components: - - sk-ctrl: docs/sk-ctrl.md - - sk-driver: docs/sk-driver.md - - sk-tracer: docs/sk-tracer.md - - skctl: docs/skctl.md + - FAQ: docs/faq.md + - Getting Started: + - Concepts: docs/intro/concepts.md + - Installation: docs/intro/installation.md + - Running a Simulation: docs/intro/running.md + - Advanced: + - Autoscaling: docs/adv/autoscaling.md + - Collecting Metrics: docs/adv/metrics.md + - Components: + - sk-ctrl: docs/components/sk-ctrl.md + - sk-driver: docs/components/sk-driver.md + - sk-tracer: docs/components/sk-tracer.md + - skctl: docs/components/skctl.md - Contributing: - - Developer's Guide: docs/contributing.md - - API changes: docs/api_changes.md + - Developer's Guide: docs/dev/contributing.md + - API changes: docs/dev/api_changes.md