Skip to content
This repository has been archived by the owner on Nov 12, 2023. It is now read-only.

Experimental project plugging Tekton yaml behind KFP API and UI engine

License

Notifications You must be signed in to change notification settings

kubeflow/kfp-tekton-backend

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Kubeflow Pipelines Backend for Tekton

Kubeflow Pipelines backend for Tekton. This project is a follow-on to KFP-Tekton Compiler project, and will allow to take the compiled Tekton yaml to Kubeflow Pipelines (KFP) engine and run it through KFP API and UI using the SDK

This project is in very early development phase. Instructions will be improved as we make more progress with implementation.

Getting Started

Prequisites

  1. Install Tekton v0.13.0 or above
  2. Install Kubeflow if you want to leverage the Kubeflow stack
  3. Clone this repository
    git clone github.com/kubeflow/kfp-tekton-backend
    cd kfp-tekton-backend
    

Install Tekton KFP with pre-built images

  1. Remove the old version of KFP from a previous Kubeflow deployment if it exists on your cluster. Also clean up the old webhooks if you were using KFP 0.4 or above.

    kubectl delete -k manifests/kustomize/env/platform-agnostic
    kubectl delete MutatingWebhookConfiguration cache-webhook-kubeflow
  2. Once the old KFP deployment is removed, run the below command to deploy this modified Kubeflow Pipeline version with Tekton backend.

    kubectl apply -k manifests/kustomize/env/platform-agnostic

    Check the new KFP deployment, it should take about 5 to 10 minutes.

    kubectl get pods -n kubeflow

    Now go ahead and access the pipeline in the Kubeflow dashboard. It should be accessible from the istio-ingressgateway which is the <public_ip>:31380

Development: Building from source code

Prerequites

  1. NodeJS 12 or above
  2. Golang 1.13 or above

Frontend

The development instructions are under the frontend directory. Below are the commands for building the frontend docker image.

cd frontend
npm run docker

Backend

The KFP backend with Tekton needs to modify the Kubeflow Pipelines api-server and persistent agent.

  1. To build these two images, clone this repository under the GOPATH and rename it to pipelines.

    cd $GOPATH/src/go/github.com/kubeflow
    git clone github.com/kubeflow/kfp-tekton-backend
    mv kfp-tekton-backend pipelines
    cd pipelines
  2. For local binary builds, use the go build commands

    go build -o apiserver ./backend/src/apiserver
    go build -o agent ./backend/src/agent/persistence
  3. For Docker builds, use the below docker build commands

    docker build -t api-server -f backend/Dockerfile .
    docker build -t persistenceagent -f backend/Dockerfile.persistenceagent .
  4. Push the images to registry and modify the Kustomization to use your own built images.

    Modify the newName under the images section in manifests/kustomize/base/kustomization.yaml.

    Now you can follow the Install Tekton KFP with pre-built images instructions to install your own KFP backend.