This document describes the features supported by the KFP-Tekton compiler. With the current implementation, the KFP-Tekton compiler is capable of compiling 80% of approximately 90 sample and test pipelines found in the KFP repository.
- Pipeline DSL Features with Native Tekton Implementation
- Pipeline DSL Features with a Custom Tekton Implementation
Below are the features using Tekton's native support without any custom workaround.
pod_annotations and pod_labels are for assigning custom annotations or labels to a pipeline component. They are implemented with
Tekton's task metadata field under Tekton
Task. The pipeline transformers example shows how to apply
custom annotations and labels to one or more components in the pipeline.
Retry feature allows users to specify the number of times a particular component should retry its execution when it fails. It is implemented with Tekton's task spec under Tekton Pipeline. The retry python test is an example of how to use this feature.
Volumes are for mounting existing Kubernetes resources onto the components. It is implemented with Tekton's task volumes feature under Tekton Task. The volume python test is an example of how to use this feature.
Timeout can be used for setting the amount of time allowed on executing a component within the Pipeline or setting the amount of time allowed on executing the whole pipeline. The task timeout is implemented with Tekton's task failure timeout under Tekton Pipeline, and pipeline timeout is implemented with Tekton's pipeline failure timeout under Tekton PipelineRun. The timeout python test is an example of how to use this feature.
RunAfter is for indicating that a component must execute after one or more other components. It is implemented with Tekton's runAfter feature under Tekton Pipeline. The sequential python test is an example of how to use this feature.
Input Parameters are for passing pipeline parameters or other component outputs to the next running component. It is implemented with Tekton's parameters features under Tekton task. The parallel_join python test is an example of how to use this feature.
ContainerOp defines the container spec for a pipeline component. It is implemented with Tekton's steps features under Tekton task. The generated Tekton task name is the same as the containerOp name whereas the step name is always called "main". The sequential python test is an example of how to use this feature.
Affinity, Node Selector, and Tolerations are Kubernetes spec for selecting which node should run the component based on user-defined constraints. They are implemented with Tekton's PipelineRunTaskSpec features under Tekton PipelineRun. The affinity, node_selector, and tolerations Python tests are examples of how to use these features.
This feature has been implemented in Tekton version 0.13.0.
ImagePullSecret is a feature for the components to know which secret to use when pulling container images from private registries. It is implemented with Tekton's podTemplate field under Tekton PipelineRun. The imagepullsecrets Python test is an example of how to use this feature.
This feature has been available since Tekton version 0.13.0.
An exit handler is a component that always executes, irrespective of success or failure,
at the end of the pipeline. It is implemented using Tekton's
finally
section under the Pipeline spec. An example of how to use an exit handler can be found in
the exit_handler compiler test.
The finally syntax is supported since Tekton version 0.14.0.
Below are the features that don't have one to one mapping to Tekton's native implementation, but the same behaviors can be replicated with extra custom processing code or workaround within the compiler.
InitContainers are containers that executed before the main component within the same pod. Since Tekton already has the concept of task steps, initContainers are placed as steps before the main component task under Tekton Task. The init_container python test is an example of how to use this feature.
Conditions are for determining whether to execute certain components based on the output of the condition checks. Since Tekton required users to define an image for doing the condition check, we created a custom python image to replicate the same condition checks from Argo and made it as the default in our compiler. The flip-coin example demonstrates how to use multiple conditions within the same pipeline.
Please be aware that the current Condition feature is using Tekton V1alpha1 API because the Tekton community is still designing the V1beta1 API. We will be migrating to the V1beta1 API once it's available in Tekton. Please refer to the design proposal for more details.
ResourceOp, VolumeOp, and VolumeSnapshotOp are special operations for creating Kubernetes resources on the pipeline cluster. ResourceOp is a basic operation for manipulating any Kubernetes resource. VolumeOp and VolumeSnapshotOp are operations for creating a unique Volume/VolumeSnapshot per pipeline and can be used as volume/snapshot with any pipeline ContainerOp. Because Tekton has no support for these features natively, we have to implement our own custom task called kubectl-wrapper for creating these Kubernetes resources. The resourceop_basic, volume_op, and volume_snapshot_op python tests are examples of how to use these features.
One thing to pay attention to is that VolumeSnapshot won't be available by default on all the supported Kubernetes versions. Therefore, VolumeSnapshotDataSource feature gate must be enabled in order to use VolumeSnapshotOp.
Output parameters are a dictionary of string files that users can define as a component's outputs. In Tekton, we implemented this with Tekton task results under Tekton Task. Since Tekton task results are only able to output parameters to its /tekton/results directory, we add an extra ending step to copy any non-Tekton user output to the /tekton/results directory. The parallel_join python test is an example of how to use this feature.
Input Artifacts in Kubeflow pipelines are used for passing raw text or local files as files placed in the component pod. Since Input Artifacts can only be raw or in a compressed format as strings, we created a custom step for passing these strings as files before the main task is executed. The input_artifact_raw_value python test is an example of how to use this feature.
Output Artifacts are files that need to be persisted to the default/destination object storage. Additionally, by default, all Kubeflow pipeline 'Output Parameters' are also stored as output artifacts. Since Tekton deprecated pipelineResource and the recommended gsutil task is not capable of moving files to the minio object storage without proper DNS address, we decided to create a step based on the minio mc image for moving output artifacts. This feature also includes the ArtifactLocation support where users can set their own object storage credentials during execution. The artifact_location python test is an example of how to use this feature.
It also includes two annontations tekton.dev/input_artifacts and tekton.dev/output_artifacts for metadata tracking. Refer to the
Tetkon Artifact design proposal for more details.
The current implementation is relying on the existing KFP's minio setup for getting the default credentials. This feature probably needs to be deprecated and merged with the output parameters once KFP finalizes the artifact management for the multi-user scenario.
Below are the features that have certain limitation in the current Tekton release.
ParallelFor is a feature for running the same component multiple times in parallel. Because Tekton has no looping or recursion feature for running tasks, right now the ParallelFor pipelines are flattened into multiple same components when using static parameters. The loop_static python test is an example of how to use this feature.
However, when using dynamic parameters, the number of parallel tasks is determined during runtime. Therefore, we are tracking the looping support issue in Tekton and implement ParallelFor with dynamic parameters once ready.
Here is the list of Tekton variables that will get substituted during pipeline execution. In addition, the compiler will automatically to map the below list of Argo variables to Tekton variables:
argo -> tekton
{{inputs.parameters.%s}} -> $(inputs.params.%s)
{{outputs.parameters.%s}} -> $(results.%s.path)
parallel_join_with_argo_vars is an example of how Argo variables are used and it can still be converted to Tekton variables with our Tekton compiler. However, other Argo variables will throw out an error because those Argo variables are very unique to Argo's pipeline system.
Below are the KFP-Tekton features with different behavior than Argo.
Sidecars are containers that are executed in parallel with the main component within the same pod. It is implemented with Tekton's task sidecars features under Tekton Task. The sidecar python test is an example of how to use this feature.
However, when you run kfp-tekton pipeline with sidecars, you may notice a completed pod and a sidecar pod with an error. There are known issues with the existing implementation of sidecars:
When the nop image does provide the sidecar's command, the sidecar will continue to run even after nop has been swapped into the sidecar container's image field. Until this issue is resolved the best way to avoid it is to avoid overriding the nop image when deploying the tekton controller, or ensuring that the overridden nop image contains as few commands as possible.
kubectl get pods will show a 'Completed' pod when a sidecar exits successfully but an Error when the sidecar exits with an error. This is only apparent when using kubectl to get the pods of a TaskRun, not when describing the Pod using kubectl describe pod ... nor when looking at the TaskRun, but can be quite confusing. However, it has no functional impact.
Tekton pipeline readme has documented this limitation.