v0.7 Release Blog
KubeArmor is increasingly deployed in near-edge and far-edge scenarios for enhanching security for IoT/Edge aspects. KubeArmor has added support to handle the characteristics of edge scenarios such as:
- Support for heterogenous platforms (most of the edge is deployed on ARM, Intel). KubeArmor ensures that the same policy enforcement techniques are delivered across different linux kernel versions. It does this by abstracting the policy layer from the enforcement layer and the enforcement layer uses the techniques that are available in the context of the deployed worker node.
- Edge devices are constrained in terms of CPU and memory availability. The overall resource usage on KubeArmor has been drastically reduced. This has not only helped edge scenarios but cloud based deployments as well.
- KubeArmor supports the philosophy of "Do not assume always-on connectivity with the cloud platform".
Recently KubeArmor finished its integration with LF Edge Open Horizon. KubeArmor is also now listed on Intel SmartEdge Marketplace. LF Edge Open Horizon and Intel® Smart Edge are edge computing platforms for deploying edge networks and delivering multi-access edge computing (MEC) for applications, containers, k8s, and virtual machines.
- Raspberry Pi (RPi) - Both as a systemd service and on k8s
- Azure ARM VM
- Apple Macbook M1 laptops
In Kubernetes, the network policy resource is a set of network traffic rules that are applied to a group of pods in a Kubernetes cluster. The network policy specifies how a pod is allowed to communicate with others. Network policy controllers (running as pods in the Kubernetes cluster) convert the requirements and restrictions of the network policies that are retrieved from the Kubernetes API into the network infrastructure.
KubeArmor has visibility into the network connections made into or from the pods. It has visibility across socket(), bind(), connect(), accept() calls. KubeArmor along with the discovery-engine could now auto-generate kubernetes network policy which are enforceable using most of the k8s CNIs (Container Network Interface). Note that the discovered policies use k8s abstractions for identifying services, deployments, and pods such as k8s labels, namespaces.
This release allows kubearmor and discovery-engine to do automatic network microsegmentation by detecting the network connections and creating appropriate network policies based on it. These auto-generated network policies are enforceable using any CNI that supports k8s network policy namely, Cilium, Calico, AWS VPC CNI, Weave.
KubeArmor currently applies policy at the pod level i.e. in the KubeArmorPolicy you specify the selector labels which selects a set of pods across the cluster. However, a single pod may contain multiple containers and each container usually has a very different purpose and the set of binaries or file paths may vary across multiple containers within the pod.
This release adds support to apply the policies at specific container level by extending the selector label policy construct. Following is an instantiation of this construct:
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: block-ls-at-container-1
namespace: wordpress-mysql
spec:
severity: 2
selector:
matchLabels:
app: wordpress
kubearmor.io/container.name: "[container-1]"
process:
matchPaths:
- path: /bin/ls
action:
BlockNote the use of kubearmor.io/container.name: "[container-1]" construct as part of the selector labels that allows KubeArmor to select specific container(s) within the pod.
The file system paths and binaries present in different containers might be completely different. Without this feature, the block of a process operation in one container would also be blocked in other containers. The discovery-engine automatically discovers the security posture of the k8s application discovers what operations should be allowed. These operations are container specific and should be restricted to specific containers. Thus without the ability to specify container specific rules, there are chances that either the security posture is too generic (in the case of allow based policies) or too restrictive (in the case of block based policies).
Specifying container specific labels are optional i.e., if the container information is not specified then the previous way of applying the policies at pod level is retained.
KubeArmor is a security policy engine and it needs to be ensured that the engine itself follows all the right security practices. Privileged containers are usually frowned upon. Almost every static scanning engine will flag this as an issue. In lot of cases, organizations deploy admission controllers that would not allow containers to be installed in privileged mode.
This release achieves following in the context:
- Removes the use of
privilege: trueflag in the context of kubearmor and associated accessory pods. - Enables individual capabilities in place of enabling all capabilities for the KubeArmor containers.
- Allows KubeArmor to be installed in non-kube-system namespace.
Kudos to Anurag Kumar who achieved this as part LFX Mentorship Project.
KubeArmor can observe system events at different dimensions including, Process, File, and Network events. Discovery Engine connects to KubeArmor and does aggregation of this information over a time period to provide app behavior summary. The use-cases for this summary view are:
- User can view what are the different process getting spawned in the given workload. Workload could be filtered based on namespace, labels etc.
- User can view what file system paths are accessed. For e.g., user can check which processes within the workload are accessing service account token.
- User can check what network connections are handled by the workload. Note that the ingress/egress connections are pinned down to the final process which handles it.
❯ karmor summary -n wordpress-mysql -l app=wordpress --agg
local port to be used for port forwarding knoxautopolicy-8587dfd464-28dpq: 9089
Pod Name wordpress-bf95888cb-rkcwm
Namespace Name wordpress-mysql
Cluster Name default
Container Name wordpress
Labels app=wordpress
Process Data
+------------------------------------------------------------------------------------------------------------------------+-----------------------------------+-------+------------------------------+--------+
| SRC PROCESS | DESTINATION PROCESS PATH | COUNT | LAST UPDATED TIME | STATUS |
+------------------------------------------------------------------------------------------------------------------------+-----------------------------------+-------+------------------------------+--------+
| /bin/bash | /bin/mkdir | 1 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /bin/bash | /bin/sed | 19 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /bin/bash | /usr/bin/cut | 3 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /bin/bash | /usr/bin/dirname | 4 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /bin/bash | /usr/bin/head | 3 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /bin/bash | /usr/bin/sha1sum | 3 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /bin/bash | /usr/local/bin/php | 5 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /var/lib/rancher/k3s/data/8307e9b398a0ee686ec38e18339d1464f75158a8b948b059b564246f4af3a0a6/bin/containerd-shim-runc-v2 | /usr/local/bin/apache2-foreground | 1 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /var/lib/rancher/k3s/data/8307e9b398a0ee686ec38e18339d1464f75158a8b948b059b564246f4af3a0a6/bin/containerd-shim-runc-v2 | /usr/sbin/apache2 | 1 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
+------------------------------------------------------------------------------------------------------------------------+-----------------------------------+-------+------------------------------+--------+
File Data
+-----------------------------------+--------------------------------------------------------------------+-------+------------------------------+--------+
| SRC PROCESS | DESTINATION FILE PATH | COUNT | LAST UPDATED TIME | STATUS |
+-----------------------------------+--------------------------------------------------------------------+-------+------------------------------+--------+
| /bin/mkdir | /etc/ | 1 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /bin/mkdir | /lib/x86_64-linux-gnu/ | 4 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /bin/rm | /lib/x86_64-linux-gnu/ | 1 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /bin/sed | /etc/ | 10 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /bin/sed | /lib/x86_64-linux-gnu/ | 25 | Mon Nov 14 03:49:35 UTC 2022 | Allow |
| /usr/bin/dirname | /etc/ | 2 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /usr/bin/dirname | /lib/x86_64-linux-gnu/ | 2 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /usr/bin/head | /dev/urandom | 1 | Mon Nov 14 03:49:34 UTC 2022 | Allow |
| /usr/bin/head | /etc/ | 1 | Mon Nov 14 03:49:35 UTC 2022 | Allow |
| /usr/bin/sha1sum | /etc/ | 2 | Mon Nov 14 03:49:34 UTC 2022 | Allow |
| /usr/bin/sha1sum | /lib/x86_64-linux-gnu/ | 1 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /usr/local/bin/apache2-foreground | /dev/tty | 1 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /usr/local/bin/apache2-foreground | /lib/x86_64-linux-gnu/ | 4 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /usr/local/bin/php | /dev/urandom | 3 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /usr/local/bin/php | /etc/ | 11 | Mon Nov 14 03:49:35 UTC 2022 | Allow |
| /usr/local/bin/php | /lib/x86_64-linux-gnu/ | 31 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /usr/local/bin/php | /tmp/ | 4 | Mon Nov 14 03:49:35 UTC 2022 | Allow |
| /usr/local/bin/php | /usr/lib/ssl/openssl.cnf | 3 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /usr/local/bin/php | /usr/lib/x86_64-linux-gnu/ | 29 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /usr/local/bin/php | /usr/local/etc/php/conf.d | 2 | Mon Nov 14 03:49:34 UTC 2022 | Allow |
| /usr/local/bin/php | /usr/local/etc/php/conf.d/ | 10 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /usr/local/bin/php | /usr/local/lib/php/extensions/no-debug-non-zts-20131226/gd.so | 2 | Mon Nov 14 03:49:35 UTC 2022 | Allow |
| /usr/local/bin/php | /usr/local/lib/php/extensions/no-debug-non-zts-20131226/mysqli.so | 3 | Mon Nov 14 03:49:35 UTC 2022 | Allow |
| /usr/local/bin/php | /usr/local/lib/php/extensions/no-debug-non-zts-20131226/opcache.so | 3 | Mon Nov 14 03:49:36 UTC 2022 | Allow |
| /usr/sbin/apache2 | /dev/urandom | 1 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /usr/sbin/apache2 | /etc/ | 8 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /usr/sbin/apache2 | /lib/x86_64-linux-gnu/ | 4 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
| /usr/sbin/apache2 | /usr/lib/ssl/openssl.cnf | 1 | Mon Nov 14 03:49:44 UTC 2022 | Allow |
+-----------------------------------+--------------------------------------------------------------------+-------+------------------------------+--------+
Egress connections
+----------+--------------------+------------+------+-----------------+-----------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+--------------------+------------+------+-----------------+-----------+-------+------------------------------+
| TCP | /usr/local/bin/php | svc/mysql | 3306 | wordpress-mysql | app=mysql | 1 | Mon Nov 14 03:49:44 UTC 2022 |
+----------+--------------------+------------+------+-----------------+-----------+-------+------------------------------+
KubeArmor already had a community driven curated list of System and Network policy templates at policy-template repository.
With the templates, it was upto the user to change values like namespace labels etc to make sure that the policies are actually enforcing on their cluster.
With the new karmor recommend it is made sure that the user doesn't have to change anything on the policy but rather simply apply them to get a secure environment for the Kubernetes deployments.
karmor recommend recommends policies based on container image, k8s manifest or the actual runtime environment itself.
$ karmor recommend --help
Recommend policies based on container image, k8s manifest or the actual runtime env
Usage:
karmor recommend [flags]
karmor recommend [command]
Available Commands:
update Updates policy-template cache
Flags:
-c, --config string absolute path to image registry configuration file
-h, --help help for recommend
-i, --image strings Container image list (comma separated)
-l, --labels strings User defined labels for policy (comma separated)
-n, --namespace string User defined namespace value for policies
-o, --outdir string output folder to write policies (default "out")
-r, --report string report file (default "report.txt")
-t, --tag strings tags (comma-separated) to apply. Eg. PCI-DSS, MITRE
Global Flags:
--context string Name of the kubeconfig context to use
--kubeconfig string Path to the kubeconfig file to use
Use "karmor recommend [command] --help" for more information about a command.- Ability to specific LSM order (#788). On certain platforms it is possible that multiple LSMs are available for enforcing a policy. For e.g., in case of Google COS (Container Optimized OS), both AppArmor and BPF-LSM are available. By default, KubeArmor chooses BPF-LSM if available, but with this change it is possible to specify the LSM order that could be used for policy enforcement.
- Support for readOnly flag with BPF-LSM (#962)
- Fix AppArmor policies behavior (#954)
- Discovery engine fixes
- Relative path are not shown anymore
- Improved
karmor summaryoutput - Improved discovered policies rules to include all necessary paths
