Override default pod spec with StrategicMergePatch (kanisterio#321)

* Use StrategicMergePatch to patch default pod spec

Signed-off-by: Prasad Ghangal <[email protected]>

* Add support to override default Pod Specs in KubeTask function

Signed-off-by: Prasad Ghangal <[email protected]>

* Add support to override pod specs in all Kanister functions

Signed-off-by: Prasad Ghangal <[email protected]>

* Add unit tests for merge patch functions

Signed-off-by: Prasad Ghangal <[email protected]>

* Use strategicpatch.JSONMap type for PodOverride specs
instead of map[string]interface{}

Signed-off-by: Prasad Ghangal <[email protected]>

* Refactoring - move common code to a function

Signed-off-by: Prasad Ghangal <[email protected]>

* UT: Add test for Blueprint spec override with Actionset

Signed-off-by: Prasad Ghangal <[email protected]>

* Update docs for PodOverride support

Signed-off-by: Prasad Ghangal <[email protected]>

* e2e_test: Add e2e test for KubeTask function

Signed-off-by: Prasad Ghangal <[email protected]>

* Addressed review comments

Signed-off-by: Prasad Ghangal <[email protected]>

docs/architecture.rst

@@ -153,6 +153,7 @@ as follows:
       Secrets map[string]ObjectReference    `json:"secrets"`
       Options map[string]string             `json:"options"`
       Profile *ObjectReference              `json:"profile"`
+      PodOverride map[string]interface{}    `json:"podOverride,omitempty"`
   }
 
 - ``Name`` is required and specifies the action in the Blueprint.
@@ -168,6 +169,8 @@ as follows:
 - ``Profile`` is a reference to a :ref:`Profile<profiles>` Kubernetes
   CustomResource that will be made available to the Blueprint.
 - ``Options`` is used to specify additional values to be used in the Blueprint
+- ``PodOverride`` is used to specify pod specs that will override default specs
+  of the Pod created while executing functions like KubeTask, PrepareData, etc.
 
 As a reference, below is an example of a ActionSpec.
 

docs/functions.rst

@@ -121,6 +121,7 @@ This allows you to run a new Pod from a Blueprint.
    `namespace`, Yes, `string`, namespace in which to execute
    `image`, Yes, `string`, image to be used for executing the task
    `command`, Yes, `[]string`,  command list to execute
+   `podOverride`, No, `map[string]interface{}`, specs to override default pod specs with
 
 Example:
 
@@ -132,6 +133,10 @@ Example:
     args:
       namespace: "{{ .Deployment.Namespace }}"
       image: busybox
+      podOverride:
+        containers:
+        - name: container
+          imagePullPolicy: IfNotPresent
       command:
         - sh
         - -c
@@ -225,6 +230,7 @@ ScaleWorkload.
    `volumes`, No, `map[string]string`, Mapping of ``pvcName`` to ``mountPath`` under which the volume will be available.
    `command`, Yes, `[]string`,  command list to execute
    `serviceaccount`, No, `string`,  service account info
+   `podOverride`, No, `map[string]interface{}`, specs to override default pod specs with
 
 .. note::
    The ``volumes`` argument does not support ``subPath`` mounts so the
@@ -407,6 +413,7 @@ and restores data to the specified path.
    `pod`, No, `string`, pod to which the volumes are attached
    `volumes`, No, `map[string]string`, Mapping of `pvcName` to `mountPath` under which the volume will be available
    `encryptionKey`, No, `string`, encryption key to be used during backups
+   `podOverride`, No, `map[string]interface{}`, specs to override default pod specs with
 
 .. note::
    The ``image`` argument requires the use of ``kanisterio/kanister-tools``
@@ -481,6 +488,7 @@ respective PVCs and restores data to the specified path.
    `pods`, No, `string`, pods to which the volumes are attached
    `encryptionKey`, No, `string`, encryption key to be used during backups
    `backupInfo`, Yes, `string`, snapshot info generated as output in BackupDataAll function
+   `podOverride`, No, `map[string]interface{}`, specs to override default pod specs with
 
 .. note::
    The `image` argument requires the use of `kanisterio/kanister-tools`
@@ -549,6 +557,7 @@ Arguments:
    `volume`, Yes, `string`, name of the source PVC
    `dataArtifactPrefix`, Yes, `string`, path on the object store to store the data in
    `encryptionKey`, No, `string`, encryption key to be used during backups
+   `podOverride`, No, `map[string]interface{}`, specs to override default pod specs with
 
 Outputs:
 
@@ -591,6 +600,7 @@ This function deletes the snapshot data backed up by the BackupData function.
    `backupIdentifier`, No, `string`, (required if backupTag not provided) unique snapshot id generated during backup
    `backupTag`, No, `string`, (required if backupIdentifier not provided) unique tag added during the backup
    `encryptionKey`, No, `string`, encryption key to be used during backups
+   `podOverride`, No, `map[string]interface{}`, specs to override default pod specs with
 
 Example:
 
@@ -625,6 +635,7 @@ BackupDataAll function.
    `backupInfo`, Yes, `string`, snapshot info generated as output in BackupDataAll function
    `encryptionKey`, No, `string`, encryption key to be used during backups
    `reclaimSpace`, No, `bool`, provides a way to specify if space should be reclaimed
+   `podOverride`, No, `map[string]interface{}`, specs to override default pod specs with
 
 Example:
 

docs/templates.rst

@@ -24,6 +24,7 @@ The TemplateParam struct is defined as:
       Options      map[string]string
       Object       map[string]interface{}
       Phases       map[string]*Phase
+      PodOverride  map[string]interface{}
   }
 
 Rendering Templates

go.mod

@@ -40,7 +40,7 @@ require (
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
 	github.com/jarcoal/httpmock v1.0.4 // indirect
 	github.com/jpillora/backoff v0.0.0-20170918002102-8eab2debe79d
-	github.com/json-iterator/go v1.1.6 // indirect
+	github.com/json-iterator/go v1.1.6
 	github.com/kelseyhightower/envconfig v1.4.0 // indirect
 	github.com/kubernetes-csi/external-snapshotter v1.1.0
 	github.com/luci/go-render v0.0.0-20160219211803-9a04cc21af0f

pkg/apis/cr/v1alpha1/types.go

@@ -22,11 +22,10 @@ which also has the apache 2.0 license.
 package v1alpha1
 
 import (
-	sp "k8s.io/apimachinery/pkg/util/strategicpatch"
-
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/runtime"
 	"k8s.io/apimachinery/pkg/runtime/schema"
+	sp "k8s.io/apimachinery/pkg/util/strategicpatch"
 )
 
 const (
@@ -100,7 +99,7 @@ type ActionSpec struct {
 	// Profile is use to specify the location where store artifacts and the
 	// credentials authorized to access them.
 	Profile *ObjectReference `json:"profile"`
-	// PodOverride is use to specify pod specs that will override the
+	// PodOverride is used to specify pod specs that will override the
 	// default pod specs
 	PodOverride sp.JSONMap `json:"podOverride,omitempty"`
 	// Options will be used to specify additional values

pkg/function/args.go

@@ -17,6 +17,10 @@ package function
 import (
 	"github.com/mitchellh/mapstructure"
 	"github.com/pkg/errors"
+	sp "k8s.io/apimachinery/pkg/util/strategicpatch"
+
+	"github.com/kanisterio/kanister/pkg/kube"
+	"github.com/kanisterio/kanister/pkg/param"
 )
 
 // Arg returns the value of the specified argument
@@ -45,3 +49,22 @@ func ArgExists(args map[string]interface{}, argName string) bool {
 	_, ok := args[argName]
 	return ok
 }
+
+// GetPodSpecOverride merges PodOverride specs passed in args and TemplateParams and returns combined Override specs
+func GetPodSpecOverride(tp param.TemplateParams, args map[string]interface{}, argName string) (sp.JSONMap, error) {
+	var podOverride sp.JSONMap
+	var err error
+	if err = OptArg(args, KubeTaskPodOverrideArg, &podOverride, tp.PodOverride); err != nil {
+		return nil, err
+	}
+
+	// Check if PodOverride specs are passed through actionset
+	// If yes, override podOverride specs
+	if tp.PodOverride != nil {
+		podOverride, err = kube.CreateAndMergeJsonPatch(podOverride, tp.PodOverride)
+		if err != nil {
+			return nil, err
+		}
+	}
+	return podOverride, nil
+}

pkg/function/copy_volume_data.go

@@ -43,6 +43,7 @@ const (
 	CopyVolumeDataOutputBackupArtifactLocation = "backupArtifactLocation"
 	CopyVolumeDataEncryptionKeyArg             = "encryptionKey"
 	CopyVolumeDataOutputBackupTag              = "backupTag"
+	CopyVolumeDataPodOverrideArg               = "podOverride"
 )
 
 func init() {
@@ -57,7 +58,7 @@ func (*copyVolumeDataFunc) Name() string {
 	return "CopyVolumeData"
 }
 
-func copyVolumeData(ctx context.Context, cli kubernetes.Interface, tp param.TemplateParams, namespace, pvc, targetPath, encryptionKey string) (map[string]interface{}, error) {
+func copyVolumeData(ctx context.Context, cli kubernetes.Interface, tp param.TemplateParams, namespace, pvc, targetPath, encryptionKey string, podOverride map[string]interface{}) (map[string]interface{}, error) {
 	// Validate PVC exists
 	if _, err := cli.CoreV1().PersistentVolumeClaims(namespace).Get(pvc, metav1.GetOptions{}); err != nil {
 		return nil, errors.Wrapf(err, "Failed to retrieve PVC. Namespace %s, Name %s", namespace, pvc)
@@ -70,6 +71,7 @@ func copyVolumeData(ctx context.Context, cli kubernetes.Interface, tp param.Temp
 		Image:        kanisterToolsImage,
 		Command:      []string{"sh", "-c", "tail -f /dev/null"},
 		Volumes:      map[string]string{pvc: mountPoint},
+		PodOverride:  podOverride,
 	}
 	pr := kube.NewPodRunner(cli, options)
 	podFunc := copyVolumeDataPodFunc(cli, tp, namespace, mountPoint, targetPath, encryptionKey)
@@ -133,11 +135,16 @@ func (*copyVolumeDataFunc) Exec(ctx context.Context, tp param.TemplateParams, ar
 	if err = OptArg(args, CopyVolumeDataEncryptionKeyArg, &encryptionKey, restic.GeneratePassword()); err != nil {
 		return nil, err
 	}
+	podOverride, err := GetPodSpecOverride(tp, args, CopyVolumeDataPodOverrideArg)
+	if err != nil {
+		return nil, err
+	}
+
 	cli, err := kube.NewClient()
 	if err != nil {
 		return nil, errors.Wrapf(err, "Failed to create Kubernetes client")
 	}
-	return copyVolumeData(ctx, cli, tp, namespace, vol, targetPath, encryptionKey)
+	return copyVolumeData(ctx, cli, tp, namespace, vol, targetPath, encryptionKey, podOverride)
 }
 
 func (*copyVolumeDataFunc) RequiredArgs() []string {

pkg/function/delete_data.go

@@ -20,6 +20,7 @@ import (
 
 	"github.com/pkg/errors"
 	v1 "k8s.io/api/core/v1"
+	sp "k8s.io/apimachinery/pkg/util/strategicpatch"
 	"k8s.io/client-go/kubernetes"
 
 	kanister "github.com/kanisterio/kanister/pkg"
@@ -42,7 +43,9 @@ const (
 	DeleteDataEncryptionKeyArg = "encryptionKey"
 	// DeleteDataReclaimSpace provides a way to specify if space should be reclaimed
 	DeleteDataReclaimSpace = "reclaimSpace"
-	deleteDataJobPrefix    = "delete-data-"
+	// DeleteDataPodOverrideArg contains pod specs to override default pod specs
+	DeleteDataPodOverrideArg = "podOverride"
+	deleteDataJobPrefix      = "delete-data-"
 )
 
 func init() {
@@ -57,12 +60,13 @@ func (*deleteDataFunc) Name() string {
 	return "DeleteData"
 }
 
-func deleteData(ctx context.Context, cli kubernetes.Interface, tp param.TemplateParams, reclaimSpace bool, namespace, encryptionKey string, targetPaths, deleteTags, deleteIdentifiers []string, jobPrefix string) (map[string]interface{}, error) {
+func deleteData(ctx context.Context, cli kubernetes.Interface, tp param.TemplateParams, reclaimSpace bool, namespace, encryptionKey string, targetPaths, deleteTags, deleteIdentifiers []string, jobPrefix string, podOverride sp.JSONMap) (map[string]interface{}, error) {
 	options := &kube.PodOptions{
 		Namespace:    namespace,
 		GenerateName: jobPrefix,
 		Image:        kanisterToolsImage,
 		Command:      []string{"sh", "-c", "tail -f /dev/null"},
+		PodOverride:  podOverride,
 	}
 	pr := kube.NewPodRunner(cli, options)
 	podFunc := deleteDataPodFunc(cli, tp, reclaimSpace, namespace, encryptionKey, targetPaths, deleteTags, deleteIdentifiers)
@@ -156,6 +160,11 @@ func (*deleteDataFunc) Exec(ctx context.Context, tp param.TemplateParams, args m
 	if err = OptArg(args, DeleteDataReclaimSpace, &reclaimSpace, false); err != nil {
 		return nil, err
 	}
+	podOverride, err := GetPodSpecOverride(tp, args, DeleteDataPodOverrideArg)
+	if err != nil {
+		return nil, err
+	}
+
 	// Validate profile
 	if err = validateProfile(tp.Profile); err != nil {
 		return nil, err
@@ -164,7 +173,7 @@ func (*deleteDataFunc) Exec(ctx context.Context, tp param.TemplateParams, args m
 	if err != nil {
 		return nil, errors.Wrapf(err, "Failed to create Kubernetes client")
 	}
-	return deleteData(ctx, cli, tp, reclaimSpace, namespace, encryptionKey, strings.Fields(deleteArtifactPrefix), strings.Fields(deleteTag), strings.Fields(deleteIdentifier), deleteDataJobPrefix)
+	return deleteData(ctx, cli, tp, reclaimSpace, namespace, encryptionKey, strings.Fields(deleteArtifactPrefix), strings.Fields(deleteTag), strings.Fields(deleteIdentifier), deleteDataJobPrefix, podOverride)
 }
 
 func (*deleteDataFunc) RequiredArgs() []string {

pkg/function/delete_data_all.go

@@ -38,7 +38,9 @@ const (
 	DeleteDataAllReclaimSpace = "reclaimSpace"
 	// DeleteDataAllBackupInfo provides backup info required for delete
 	DeleteDataAllBackupInfo = "backupInfo"
-	deleteDataAllJobPrefix  = "delete-data-all-"
+	// DeleteDataAllPodOverrideArg contains pod specs to override default pod specs
+	DeleteDataAllPodOverrideArg = "podOverride"
+	deleteDataAllJobPrefix      = "delete-data-all-"
 )
 
 func init() {
@@ -72,6 +74,11 @@ func (*deleteDataAllFunc) Exec(ctx context.Context, tp param.TemplateParams, arg
 	if err = OptArg(args, DeleteDataAllReclaimSpace, &reclaimSpace, false); err != nil {
 		return nil, err
 	}
+	podOverride, err := GetPodSpecOverride(tp, args, DeleteDataAllPodOverrideArg)
+	if err != nil {
+		return nil, err
+	}
+
 	// Validate profile
 	if err = validateProfile(tp.Profile); err != nil {
 		return nil, err
@@ -92,7 +99,7 @@ func (*deleteDataAllFunc) Exec(ctx context.Context, tp param.TemplateParams, arg
 		deleteIdentifiers = append(deleteIdentifiers, info.BackupID)
 	}
 
-	return deleteData(ctx, cli, tp, reclaimSpace, namespace, encryptionKey, targetPaths, nil, deleteIdentifiers, deleteDataAllJobPrefix)
+	return deleteData(ctx, cli, tp, reclaimSpace, namespace, encryptionKey, targetPaths, nil, deleteIdentifiers, deleteDataAllJobPrefix, podOverride)
 }
 
 func (*deleteDataAllFunc) RequiredArgs() []string {

pkg/function/kube_task.go

@@ -19,6 +19,7 @@ import (
 
 	"github.com/pkg/errors"
 	"k8s.io/api/core/v1"
+	sp "k8s.io/apimachinery/pkg/util/strategicpatch"
 	"k8s.io/client-go/kubernetes"
 
 	kanister "github.com/kanisterio/kanister/pkg"
@@ -30,10 +31,11 @@ import (
 )
 
 const (
-	jobPrefix            = "kanister-job-"
-	KubeTaskNamespaceArg = "namespace"
-	KubeTaskImageArg     = "image"
-	KubeTaskCommandArg   = "command"
+	jobPrefix              = "kanister-job-"
+	KubeTaskNamespaceArg   = "namespace"
+	KubeTaskImageArg       = "image"
+	KubeTaskCommandArg     = "command"
+	KubeTaskPodOverrideArg = "podOverride"
 )
 
 func init() {
@@ -48,7 +50,7 @@ func (*kubeTaskFunc) Name() string {
 	return "KubeTask"
 }
 
-func kubeTask(ctx context.Context, cli kubernetes.Interface, namespace, image string, command []string) (map[string]interface{}, error) {
+func kubeTask(ctx context.Context, cli kubernetes.Interface, namespace, image string, command []string, podOverride sp.JSONMap) (map[string]interface{}, error) {
 	var serviceAccount string
 	var err error
 	if namespace == "" {
@@ -67,7 +69,9 @@ func kubeTask(ctx context.Context, cli kubernetes.Interface, namespace, image st
 		Image:              image,
 		Command:            command,
 		ServiceAccountName: serviceAccount,
+		PodOverride:        podOverride,
 	}
+
 	pr := kube.NewPodRunner(cli, options)
 	podFunc := kubeTaskPodFunc(cli)
 	return pr.Run(ctx, podFunc)
@@ -109,11 +113,16 @@ func (ktf *kubeTaskFunc) Exec(ctx context.Context, tp param.TemplateParams, args
 	if err = OptArg(args, KubeTaskNamespaceArg, &namespace, ""); err != nil {
 		return nil, err
 	}
+	podOverride, err := GetPodSpecOverride(tp, args, KubeTaskPodOverrideArg)
+	if err != nil {
+		return nil, err
+	}
+
 	cli, err := kube.NewClient()
 	if err != nil {
 		return nil, errors.Wrapf(err, "Failed to create Kubernetes client")
 	}
-	return kubeTask(ctx, cli, namespace, image, command)
+	return kubeTask(ctx, cli, namespace, image, command, podOverride)
 }
 
 func (*kubeTaskFunc) RequiredArgs() []string {

pkg/function/kube_task_test.go

@@ -22,6 +22,7 @@ import (
 	. "gopkg.in/check.v1"
 	"k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	sp "k8s.io/apimachinery/pkg/util/strategicpatch"
 	"k8s.io/client-go/kubernetes"
 
 	kanister "github.com/kanisterio/kanister/pkg"
@@ -126,6 +127,14 @@ func (s *KubeTaskSuite) TestKubeTask(c *C) {
 		StatefulSet: &param.StatefulSetParams{
 			Namespace: s.namespace,
 		},
+		PodOverride: sp.JSONMap{
+			"containers": []map[string]interface{}{
+				{
+					"name":            "container",
+					"imagePullPolicy": "Always",
+				},
+			},
+		},
 	}
 	action := "test"
 	for _, tc := range []struct {