Add merge strategy annotations to managed resources

[negz]

What problem are you facing?

The new FunctionComposer added in crossplane/crossplane#4500 uses server-side-apply (crossplane/crossplane#4047). This means that if you're using a Composition in mode: Pipeline Crossplane (technically, the API server) is able to correctly merge arrays of objects, among other things.

For this to work, the CRD of the composed resource needs to include merge strategy OpenAPI extensions. For example by default arrays of objects will be replaced entirely - the CRD needs to specify //listType: map and a //listMapKey to change this behavior.

How could Crossplane help solve your problem?

We should start adding these annotations to Crossplane types. The core types defined in crossplane-runtime (e.g. ConditionedStatus) would be a great start, plus very common types like AWS tags.

[negz]

CC @ulucinar - just an FYI.

[negz]

@ulucinar would it be relatively straightforward to add the //+mapType: granular comment marker to all AWS tags, like https://github.com/upbound/provider-aws/blob/v0.43.1/apis/acm/v1beta1/zz_certificate_types.go#L50?

I think this would have a big impact for folks, e.g. in crossplane-contrib/function-patch-and-transform#50.

[CodyRay]

Will this be possible for uses like helm-provider that can contain arbitrary data. Our usecase is with patch and transform we have a base containing an array inside the value and we'd like to patch in an additional array from the XRD claim. Today we'd do that with resources[i].patches[i].policy.mergeOptions but I noticed that is one of the things that is not supported with function-patch-and-transform

[negz]

@CodyRay No, I don't think this will work for a runtime.RawExtension since the API server has no idea about the structure of the data inside that field and thus SSA can't merge it.

I'm not entirely sure how to handle that case. We could reintroduce mergeOptions, but I'm a little reluctant to do that given that in most cases folks should not use it and should use SSA instead.

Could you raise an issue to track your use case WRT merging/appending RawExtension fields?

[sttts]

This might be a naive question, but who is the other actor you want to merge with? The provider that completes lists provided by a function?

[negz]

@sttts Yes, it's usually (always?) the provider. There's still parts of spec that are "shared" - e.g. AWS tags.

[CodyRay]

Could you raise an issue to track your use case WRT merging/appending RawExtension fields?

Would that issue belong in https://github.com/crossplane/upjet or https://github.com/crossplane-contrib/function-patch-and-transform ?

[negz]

@CodyRay Option 3 - https://github.com/crossplane/crossplane. 😉

[erikgb]

@CodyRay Did you ever create an issue for this?

[CodyRay]

I didn't end up creating a ticket. We found some really problematic behavior before we made it to production with mergeOptions where array items weren't removed from the destination when they are deleted from the source composite resource. Since I think that in the long term server-side apply is going to be better, we ended up pivoting to use a function and I suspect we'll eventually use https://github.com/crossplane-contrib/function-go-templating or do the appending within our helm chart instead.