POC: Skip kaniko build in serverless #941
Comments
kwiatekus
changed the title
|
Additionally @kwiatekus estimate the gain:
|
kwiatekus
changed the title
kwiatekus
changed the title
|
if I understand well the whole ideas is pretty much similar to the existing base function image override. ? |
POC v1 resultsActual solutionThe whole concept is a little bit simpler than the previous solution. Right now we have to deploy in-cluster docker-registry in the
additionally, we have to control three phases of function lifecycle (configuration, building, running) and in the separate phase, we create separate resources belong to the functions:
The diagram below describes the actual implementation in general:
New solutionConceptPOC proved that it's possible to skip the building phase and mount function sources directly to the functions' pods. The idea was to register central PVC, expose it to other namespaces using NFS server, and then use dedicated (for every function) PV and PVC to mount function sources to the functions pods. The diagram below describes the idea in general:
In the following proposition we can highlight two function lifecycle phases:
ImplementationTo prove the idea we decided to create our own, simpler, and not full-featured controller to a little bit speed up the whole process. Implementation is split into two folders:
We decided to support happy-path, without:
To test POC run the following script in the checkouted docker build -t buildless-serverless:alpha1 -f components/buildless-serverless/Dockerfile .
k3d cluster create
k3d image import buildless-serverless:alpha1
kubectl create ns kyma-system
helm template --namespace kyma-system config/buildless-serverless | kubectl apply -f -
sleep 5
kubectl wait --for condition=ready pod -l app=buildless-serverless -n kyma-system --timeout 3mnow we can create a function and port-forward it: cat <<EOF | kubectl apply -f -
apiVersion: serverless.kyma-project.io/v1alpha2
kind: Function
metadata:
name: test-function-nodejs-cjs
spec:
runtime: nodejs20
source:
inline:
dependencies: |
{
"name": "test-function-nodejs",
"version": "1.0.0",
"dependencies": {
"lodash":"^4.17.20"
}
}
source: |
const _ = require('lodash')
module.exports = {
main: function(event, context) {
return _.kebabCase('Hello World from Node.js 20 Function');
}
}
EOF
sleep 5
kubectl wait --for condition=ready pod -l function=test-function-nodejs-cjs --timeout 3m
kubectl port-forward svc/test-function-nodejs-cjs 8080:80now we can call a function in a second terminal: curl localhost:8080result: Conclusions
|
Given you have removed all the real features that make serverless what was the purpose of this POC ? |
We have not removed them, we do not support them in the POC implementation. That does not mean that this POC expects to get rid of these features. We decided to prove that it's possible to skip the build phase and we need PVC to keep all features. It's described in the |
POC v2 resultsIntroductionAfter proposing the POC v1 we got tons of feedback and together with architects decided to simplify the whole concept. General feedback:
ResultNow new serverless architecture is much, much simpler in comparison to the actual implementation and POC v1:
No NFS serverThe first huge change is the lack of the NFS server. This change allows us to run functions much faster and immediately react to changes in code, but the disadvantage is that there is no possibility to support user dependencies and download them in runtime (possibility always exists but in this case, without NFS server or PVC for every function, it's not possible to synchronize dependencies between functions pods and keep them for next function versions). No PVCThe second change is that we decided to get rid of storing functions code and dependencies in one, dedicated storage shared between function pods and it's versions. This is related to NFS server removal. No dependenciesThe first and the second change forced us that we should not allow users to pass custom dependencies in the functions spec. Because in POCv2 without an NFS server and common PVC, there is no possibility to update functions code without changing pod's definition, for example when a user updates the code but dependencies are still the same. More runtimes!In the final shape of the POC, we can't support user dependencies, so we have to prepare more runtimes with built-in dependencies. For example, for it can look like this:
We can still support fields like # import base nodejs20 image with all features
FROM europe-docker.pkg.dev/kyma-project/prod/function-runtime-nodejs20:2.0.0
# add custom dependencies
RUN npm install lodashHot DeployIn this case, we can offer really simple and usable hot-reload functionality. If we offer functions as simple code runners, that can be ready to go almost immediately after applying it on the cluster, then we can pass code to the pod as an argument for the apiVersion: v1
kind: Pod
metadata:
name: test-function
namespace: default
spec:
containers:
- command:
- "bash"
- "-c"
- "echo <FUNCTION_CODE> > handler.js; npm run"
...
We also know that we can keep code in configmap and mount it to every function pod, but this solution has huge disadvantages. The time needed for Kubernetes to update mounted files in pods collapses from 5sec to 30sec. We aim for a solution that allows us to run functions in less than 5sec, so we can't see configmaps. Function v2The POC v1 aims to be fully compatible with the actual solution so as a result we've got a solution proposition that does not require any changes in the Functions API (CRD). The POC v2 changes too many things and concepts as a result we should prepare totally new Functions CR version, for example, the Unfortunately, migration from version Conclusions
|
Function v2alpha1In this POC we mentioned that we should prototype and implement another version of our Functions. I would propose the very first shape of such an API. I prepared it based on the existing solution, but without fields that we can't support (like things about build configuration like apiVersion: serverless.kyma-project.io/v2alpha2
kind: Function
metadata:
name: function
spec:
replicas: 1
runtime: nodejs20
runtimeImageOverride: pprecel/nodejs20-custom:0.1
source:
inline: |- # or code:
function main() {...
# or
# gitRepository: ...
env:
- name: MY_ENV
value: MY_VALUE
resourceConfiguration:
profile: XL
# or
# requests...
# limits...
secretMounts:
- secretName: my-secret
mountPath: /tmp/secret
labels:
test-label: test-label-value
annotations:
test-annotation: test-annotation-valueNew HorizonThese changes are the very end of certain features but the beginning of a new, better buildless-serverless life, and a new horizon. We can support users who have prepared custom runtime bases with additional functionalities or dependencies and want to pull these runtime bases from private registries. Previously we had to connect our ServiceAccounts to every functions pod because these ServiceAccount allows a pod to pull images from a private in-cluster registry where we store such images. Right now we can allow users to use their own ServiceAccount for every function to allow users to pull images from private registries and for example to configure RBACs for every function. For example: A user may want to deploy a function that has to update configmap or any other resource after getting a request. Then user can define a custom ServiceAccount with a Role and RoleBinding that allows to do such operations on the cluster. When a user wants to deploy a function with a custom base runtime image, and such image is located inside the private registry, then ServiceAccount with imagePullSecrets:
- name: private-registry-configurationNew ChallengesRight now we keep a kind of hot-potato in our serverless manager, called If we want to propose and still support git-sourced functions I would suggest thinking about alternatives. The huge problem is the best (from my perspective) alternative is go-git but as far as I know the input data needed for connections looks a little bit different and we should prototype and refactor our actual API under the New PossibilitiesFrom our perspective, the actual Serverless is the developer-only tool. It means that is does not meet production-grade requirements like SAP-SLC29. The good news is in our new architecture we may re-investigate this topic and maybe we can meet such requirenments with git-sourced functions. This is only a quick thought and as I said it requires re-investigation in this area, but hope is our fuel. |
|
Having consulted @pbochynski and @varbanv, we decided that possibility for users to define custom dependencies should be preserved. As a next step, please implement a prototype (v3) with the following assumption:
Please prove that the same assumptions are applicable for python functions as well. The possibility of skipping runtime dependency resolution in favor of using SAP provided runtime images (with a set of commonly used libraries) will be considered in the next steps (as a path towards enabling SLC-29 compliance) |
POC v3 resultIntroductionAs @kwiatekus mentioned, the goal of the next POC iteration is to prove that it's possible to implement a new serverless without the build phase, without the NFS server but still compatible with old functionality. The plan is to support flow described in the POC v2 but without getting rid of support of Functions dependencies (in the Function CR spec). ImplementationThe POC implementation lands in this PR and can be run using these spells: docker buildx build --platform linux/amd64 -t pprecel/buildless-serverless:alpha1 -f components/buildless-serverless/Dockerfile .
k3d cluster create
k3d image import pprecel/buildless-serverless:alpha1
kubectl create ns kyma-system
helm template --namespace kyma-system config/buildless-serverless | kubectl apply -f -
sleep 5
kubectl wait --for condition=ready pod -l app=buildless-serverless -n kyma-system --timeout 3mHot-ReloadIn my implementation I decided to keep sources (code and deps) inside the Pod definition to perform hot-reload functionality for all functions pods at the same time. For my example function with dependencies, the pods definition looks like this: apiVersion: v1
kind: Pod
metadata:
name: nodejs-deps-5b76877c74-jjkz6
namespace: default
spec:
containers:
- command:
- sh
- -c
- |2
printf "${FUNC_HANDLER_SOURCE}" > handler.js;
printf "${FUNC_HANDLER_DEPENDENCIES}" > package.json;
npm install --prefer-offline --no-audit --progress=false;
cd ..;
npm start;
env:
...
- name: FUNC_HANDLER_SOURCE
value: |
const _ = require('lodash');
var hana = require('@sap/hana-client');
module.exports = {
main: function(event, context) {
return _.kebabCase('Hello World from Node.js 20 Function');
}
};
- name: FUNC_HANDLER_DEPENDENCIES
value: |
{
"name": "nodejs-deps",
"version": "1.0.0",
"dependencies": {
"lodash":"^4.17.20",
"@sap/xsenv":"5.3.0",
"@sap/hana-client":"2.22.27"
}
}
image: europe-docker.pkg.dev/kyma-project/prod/function-runtime-nodejs20:main
imagePullPolicy: IfNotPresent
name: function-container
securityContext:
capabilities:
drop:
- ALL
privileged: false
procMount: Default
readOnlyRootFilesystem: true
startupProbe:
failureThreshold: 300
httpGet:
path: /healthz
port: 8080
scheme: HTTP
periodSeconds: 1
successThreshold: 1
timeoutSeconds: 1
...
volumeMounts:
- mountPath: /usr/src/app/function
name: sources
- mountPath: /.local
name: local
- mountPath: /tmp
name: tmp-dir
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-m8bkd
readOnly: true
workingDir: /usr/src/app/function
...
securityContext:
runAsGroup: 1001
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
...
volumes:
- emptyDir: {}
name: sources
- emptyDir: {}
name: local
- emptyDir:
sizeLimit: 100M
name: tmp-dir
...As we can see in this example source code and dependencies are stored in the FeaturesThe POC v3 is almost the same as v2, but we decided to check how resource and time-consuming will be adding dependencies resolving s functions pod startup (in runtime). The second important difference is support for python3.12 functions to check if there will be any differences in behavior or further problems. The second change is support for resource specification MeasurementsThis section describes how I tested POC v3 implementation on BTP cluster FunctionsI've decided to deploy four small functions to check how fast these will be fully operational (running):
All functions were deployed at once with modified resource configuration in a few variants described below: XS:
requestCpu: "50m"
requestMemory: "64Mi"
limitCpu: "100m"
limitMemory: "128Mi"
S:
requestCpu: "100m"
requestMemory: "128Mi"
limitCpu: "200m"
limitMemory: "256Mi"
M:
requestCpu: "200m"
requestMemory: "256Mi"
limitCpu: "400m"
limitMemory: "512Mi"
L:
requestCpu: "400m"
requestMemory: "512Mi"
limitCpu: "800m"
limitMemory: "1024Mi"
XL:
requestCpu: "800m"
requestMemory: "1024Mi"
limitCpu: "1600m"
limitMemory: "2048Mi"ResultThe measurement process was really simple and based on the startupProbe looking like this: startupProbe:
failureThreshold: 300
httpGet:
path: /healthz
port: 8080
scheme: HTTP
periodSeconds: 1
successThreshold: 1
timeoutSeconds: 1This allows using the
As we can see running package managers (npm, pip) requires more time to fully run the function's code. But in my opinion, this time is still satisfactory during we are talking about seconds, not minutes. Of course dependencies I chose for the test are relatively light and probably running functions with the AI toolset will require much more memory, CPU, and time but this is obvious and in my opinion, this is not the use case for Kyma Functions. Conclusions
|
|
why k3d and not kyma as a testbed? |
I only showed how to run POC on k3d, because it doesn't require any additional action. Personally, I used BTP to run and test POC implementation. To do this, switch to sources from this PR, export KUBECONFIG env with the path to your cluster kubeconfig file, and run: helm template --namespace default config/buildless-serverless | kubectl apply -f -
sleep 5
kubectl wait --for condition=ready pod -l app=buildless-serverless -n default --timeout 3m |
Description
Verify the following ideas:
into serverless provided base image.
Document the outcome as a issue comment and discuss with the team (derive recommendations for git functions).
If the ideas are validated, draw necessary diagrams explaining necessary changes in the function processing flow.
Reason
If the POC validates the idea we could eliminate the dependencies to kaniko build and docker registry. This could greatly speed up redeploying new versions of code and reduce necessary resources required to develop ideas as functions (no extra memory, cpu time or Volume storage for images).
Additionally the POC could help positioning serverless module as development tool that helps users to prove their ideas in the kyma runtimes as functions and later "eject" them into manifests that declare production-grade workloads
The text was updated successfully, but these errors were encountered: