Multi-Domain Support for single uds-core cluster

[joelmccoy]

Is your feature request related to a problem? Please describe.

Currently I don't have a method to host UDS Packages under multiple domains on a single uds-core cluster. I would like to be able to host UDS_PACKAGE_1 under app.domain1.com and UDS_PACKAGE_2 under app.domain2.com using the native package CRD.

This ask is coming from this issue: defenseunicorns/uds-marketplace#204

Describe the solution you'd like

• Given a single uds core cluster is deployed
• When when I deploy two different packages
• Then I have the ability to deploy these packages under different domains

Describe alternatives you've considered

• Deploying a custom Istio Ingress and not use the UDS Package CR to create the virtual services and netpols
• Deploying another cluster to host this new domain

Additional context

I see two paths forward for implementation that would be satisfactory:

Allow the Istio Tenant and Admin Gateway to support multiple domains

• Implementation might look like specifying UDS_DOMAIN=domain1.com,domain2.com
• The Istio gateways created by uds-core would need to be updated to support *.domain1.com and *.domain2.com
• This would require updates the the UDS Operator to create virtual services that allow both of these domains (or allow selecting a domain in the Package CRD)
• This would require a Wildcard SAN TLS cert to be generated by the end user to use on the gateways (I tested this and it works fine)

Add UDS Package Operator support for another gateway

• This requires the consumer of core to develop and deploy another package on top of core that creates an Istio Gateway
• Would need to update the UDS Operator logic to select the custom gateway and pick the appropriate domain for the virtual services (I believe this is hardcoded right now?)
• Maybe some netpols stuff would have to change with the operator as well?

[mjnagel]

I think I may advocate for the listed alternative:

Deploying a custom Istio Ingress and not use the UDS Package CR to create the virtual services and netpols

Unless there is a clear ask for multi-domain support from a significant portion of our user-base, this feels like a potential one-off that we could support building out as conditional manifests in the Marketplace package. It would still run on core and benefit from pieces of core integration, but would require gateway, virtualservice, and network policy to be defined and deployed with the marketplace package.

Alternatively we could bite off parts of this, as @joelmccoy noted synchronously one thing we could support in the operator would be allowing use of additional gateways, specifically for expose entries in the Package CR. Core wouldn't necessarily create/manage additional Gateway resources, but could allow them to be used via expose. This would effectively reduce the number of resources needed for this from 3 (Gateway, VS, NetPol) to 1 (Gateway). Would have to lean more into validations and pepr looking up existing gateways, but it's overall not too complex to support.