Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: adjust memory overhead #394

Merged
merged 1 commit into from
Apr 26, 2024
Merged

runtime: adjust memory overhead #394

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions dev-docs/aks/nested-virt-internals.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,12 +155,12 @@ This file is a serialization of [katautils.tomlConfig] and contains decent inlin
option.

<details>
<summary>Example of a useful drop-in fragment</summary>
<summary>Example of a drop-in fragment raising the default memory per VM</summary>

```sh
cat >/opt/confidential-containers/share/defaults/kata-containers/config.d/10-memory.toml <<EOF
[hypervisor.clh]
default_memory = 256
default_memory = 512
EOF
```
</details>
Expand All @@ -173,9 +173,11 @@ EOF

There's [AKS documentation for resource management] which explains the basics of how CPU and
memory are allocated for a Kata VM.
The default memory allocation is quite high at 2GiB, which fills up the node fast.
It's unclear why this default is chosen, given that the container limit's added on top of this
value. Forcing a size with the pod annotation
The default memory overhead added by the `RuntimeClass` is quite high at 2GiB, which fills up the node fast.
It's unclear why this default is chosen, given that the container limit is added on top of this
value and that the VMs are created with a 256MiB overhead.

Forcing a size with the pod annotation
`io.katacontainers.config.hypervisor.default_memory` would be possible, but the annotation would
need to be allow-listed in the config setting `enable_annotations`.

Expand All @@ -186,7 +188,7 @@ need to be allow-listed in the config setting `enable_annotations`.

```toml
[hypervisor.clh]
default_memory = 2048 # MiB! Minimum seems to be around 256.
default_memory = 256
default_vcpus = 1
enable_annotations = ["enable_iommu"]

Expand Down
2 changes: 1 addition & 1 deletion e2e/internal/kuberesource/parts.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ func ContrastRuntimeClass() *RuntimeClassConfig {
r := RuntimeClass(runtimeHandler).
WithHandler(runtimeHandler).
WithLabels(map[string]string{"addonmanager.kubernetes.io/mode": "Reconcile"}).
WithOverhead(Overhead(corev1.ResourceList{"memory": resource.MustParse("2Gi")})).
WithOverhead(Overhead(corev1.ResourceList{"memory": resource.MustParse("1152Mi")})).
WithScheduling(Scheduling(map[string]string{"kubernetes.azure.com/kata-cc-isolation": "true"}))

return &RuntimeClassConfig{r}
Expand Down