feat: support shared components and providers

[brooksmtownsend]

Feature or Problem

This PR adds support to wadm for two major changes:

1. Allow applications to attach a special wasmcloud.dev/shared: 'true' annotation, which marks an application as shared to the entire lattice
2. Allow applications to link components and providers to/from components in a different deployed application that is shared

Components and capability providers can now specify a shared application in their properties rather than supplying an image reference. When an application is deployed that references a component in a different application, wadm will validate that the application is deployed and that the referenced component exists.

You cannot use the shared component feature to alter configuration, identifiers, or scaling properties of a shared component. I could see a feature in the future to add additional scalers, but it might cause undesirable behavior and it would be generally just better to supply your own component instead of using a shared one at this point.

This PR is getting really large, so I'm going to push off additional validation in the handlers to a separate PR that can be reviewed independently.

Here are things that should be done before this feature lands, but based on their checked status aren't completed in this PR:

(Captured these in #451)

• Ensure applications that have dependents can't undeploy or delete
• Ensure applications that are shared can't change to be unshared if there are dependents
• Ensure applications that are shared can't remove the shared components if there are dependents of a component
• Surface shared component status in applications that depend on that component (this is hard and I'm currently thinking of the best way to do this)
• Double check that applications can't be deployed with both image/application reference or neither
• Update WIT representation of the component/capability properties

Related Issues

Fixes #277

Release Information

wadm 0.18

This feature will be released with 0.18.0 and denoted an experimental feature for wadm. The cross-manifest linking works well and is e2e tested. There are additional guardrails we may want to introduce here, or change the way the feature works.

Consumer Impact

This doesn't include any breaking changes for existing manifests

Testing

Unit Test(s)

I'd like to include additional unit tests in the convert.rs file that prove that the conversion between a manifest + scalers is properly handled, given how critical and complex that code is. This is made harder by the need for a bunch of generics

Acceptance or Integration

Added the e2e_shared integration test suite that tests sharing components, providers, and multiple aspects of linking to/from shared components.

• I'd also like to include some failure scenarios in this integration test suite.

Manual Verification

[joonas]

I'm not sure that it necessarily makes sense to address in this PR, but something that I think needs to be addressed before this feature makes it into a release / becomes available for use is the reporting from a shared application/component's perspective.

If I'm providing a shared component to the lattice, I would like to be able to query what all applications are linking against my shared component so that I can have a conversation with the teams responsible for those applications about the changes I may need to make.

As I currently understand things, this change works well from the perspective of establishing a link to a shared component, but as of right now there is nothing to help the teams providing these shared components understand their "dependents", which is (imo) an absolute necessity for any platform-ish teams to be able to successfully operate.

The other thing that I'm not quite clear on is how would an upgrade to/of a shared component be facilitated, would you be forced to spin up a new application (or perhaps add a new shared component to your existing application) with a different/newer version and then encourage the internal teams to upgrade?

[brooksmtownsend]

As I currently understand things, this change works well from the perspective of establishing a link to a shared component, but as of right now there is nothing to help the teams providing these shared components understand their "dependents", which is (imo) an absolute necessity for any platform-ish teams to be able to successfully operate.

Yeah I agree, I captured that in #451 as a followup since this is too hard (imo) to find out just by querying NATS

The other thing that I'm not quite clear on is how would an upgrade to/of a shared component be facilitated, would you be forced to spin up a new application (or perhaps add a new shared component to your existing application) with a different/newer version and then encourage the internal teams to upgrade?

Yeah so here, right now, there's no validation rules and the expectation is you could updated a shared component to a newer version just fine. It will be up to the deployer to make sure they aren't breaking dependents. I don't think that this is ideal, but I worry that overprescribing deployment rules will make using this feature harder.

[thomastaylor312]

Approved with some nits