Application endpoint discovery

[arne-broering]

How to inform about exposed app endpoints?
How do I query the application?

[phil-abb]

@gunjald can you provide more details on what you mean by this? I watched the recording of Wednesday's call and didn't understand what you meant. It wasn't clear to me who needs to know this information and what they would do with it.

For example:

• Are you talking about endpoints an end-user would need to know about after deploying an application so they could do things like access the application's UI or web service?
• Are you talking about how different applications, having previous knowledge of each other, discover the endpoints when the applications are deployed on the same device/cluster?
• Are you talking about how applications, having NO previous knowledge of each other, figure out how to talk to each other?
• Are you talking about how the different containers in a single application deployment figure out how to communicate with each other?

[gunjald]

My comment was mainly for the two points and one of them you captured already while other are also valid but we have not defined use cases for them so far as I understand. The two points would be then,

1. endpoints an end-user would need to know about after deploying an application so they could do things like access the application's UI or web service
2. endpoints that client applications outside of margo edge device may need to know about of deployed application. These client applications on industrial devices as an example a sensor which could submit the temperature or pressure reading to edge application on desired endpoint of the edge application instance

In general there would be endpoints for different reasons that one or other clients in industrial environment will use to connect with the edge applications and it could be a FQDN (DNS name of the application) representing the edge application endpoint but how this FQDN is generated/managed and resolved by those clients who needs it? And if there are other means than DNS which could be evaluated for this purpose?

[phil-abb]

@gunjald

The application vendor's end-user documentation should provide information about accessing any UIs and APIs in the deployed application. Of course, how the application exposes these is not so straightforward under Margo.

For Kubernetes, I'm aware of the following ways to expose UIs and APIs:

• By deploying a service manifest with the type LoadBalancer as part of your helm chart.
• By using an ingress controller and deploying an ingress manifest as part of your helm chart.
• By deploying manifest to use the Gateway API as part of your helm chart.

For Docker Compose, I'm aware of the following ways to expose UIs and APIs:

• By exposing ports that map to the container's internal ports.

Margo complicates this because the application vendors don't know what else is being deployed onto the device:

1. Exposing ports using the Kubernetes service manifest or via Docker Compose

   • The service manifest or Docker Compose file provided by the application vendor indicates what port should be exposed. This means multiple application vendors could try exposing the same port.

2. Defining and Ingress manifest

   • This is the approach I've always used on Kubernetes.
   • The ingress manifest uses annotations to control different aspects of the ingress controls (e.g., TLS termination rules). The problem with this is each ingress controller, of which there are several, uses different annotations. This means application vendors either need to know which ingress controller is on the device or deploy one as part of their application which doesn't seem like a good idea.
   • The other issue occurs if multiple applications deploy ingress manifests using the same hostname. Several problems can occur if this happens. As a result, each application namespace needs to use its own hostname. One example of a problem is if you're using TLS. The certificate referenced in the ingress manifest belongs to the application's namespace. I've addressed this previously by using an app-specific prefix added to the machine's hostname (e.g. myapp.edge.company.com) as the hostname in the ingress manifest.
   • Ingress routes have several issues, many of which are being addressed by the Gateway API

3. Using the Gateway API

   • I haven't used this yet but I imagine it has the same issue as the ingress manifest if the HTTPRoute hostnames are not unique.
   • This is a newer API and I'm not sure how widely supported it is right now.

[gunjald]

Hi Phil,

You have captured the different points very well and I think that has another dimension where I was focusing on that is for example if a port has been exposed by the application via a load balancer on its IP address and that port may not be for application vendor but for the other devices or external applications in that environment who may want to connect with that application. And those external application should have a way to discover which edge application they want to connect with and what is the end point of the edge application(defined in Margo.yaml).

WOS may not may what port has been defined within Helm chart or compose file that should be exposed for such purposes and similarly the external applications doesnt know where to connect with margo edge application instance.

Hypothetically, if WOS would have been aware of a port that a given edge application (A helm chart in Margo.yaml) wanted to expose, it may have created an entry to a DNS server e.g. "margo-test-app.enterprisexxx.com" pointing to the load balancer IP and port. So the external applications may have just use the "margo-test-app.enterprisexxx.com" to connect with the margo edge application. I would treat such procedures as discovery for the sake of explanation.

But so far any margo application specific information about the application endpoint seems to be contained within the artefacts like Helm charts or compose files and only known to application vendor. But then finally once they are deployed on Margo compatible devices they still need to be discoverable by other consumer application outside of margo devices in my opinion.

[phil-abb]

@gunjald

I think we can split this into two areas:

1. How does Margo deal with the different ways of exposing UIs/APIs
2. How does Margo deal with automatic service discovery