feat: support health check

[fyuan1316]

[description]
add health check for deployment

close #17

[zhekazuev]

@fyuan1316
Hi,

Maybe better to use exec or tcpProbe startupProbe instead of httpGet?

startupProbe:
  tcpSocket:
    port: 8080
  initialDelaySeconds: 60
  periodSeconds: 5

As I understand image has healthcheck for livenessProbe(source):

livenessProbe:
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 10
  httpGet:
    scheme: HTTP
    port: 8080
    path: /readyz

And readinessProbe - as you did:

readinessProbe:
  initialDelaySeconds: 10
  periodSeconds: 5
  timeoutSeconds: 10
  httpGet:
    scheme: HTTP
    port: 8080
    path: /v1/models

What do you think?

And how to check if the file has already been built?
Maybe check the process or file via exec startupProbe?
I'm not very good at using models.

[fyuan1316]

Hi, @zhekazuev thanks for reply.

As I understand image has healthcheck for livenessProbe(source):

Inspired by your comment, I looked a little deeper into the code.
https://github.com/go-skynet/LocalAI/blob/master/api/api.go#L168-#L171

Based on the code, I think we can change readinessProbe path to "/readyz" and livenessProbe to "/healthz". This way we can be consistent with the original intent of the code design.

Maybe better to use exec or tcpProbe startupProbe instead of httpGet?

Regarding startupProbe, at first I noticed that the pod status was running but when I accessed the /v1/models service it didn't return properly. So my original expectation was that the startupProbe would indicate that the model was ready for service.
If we use a tcpSocket here, I think startupProbe should behave the same as now, i.e. not wait for the model to be ready for service. Do I understand correctly?

And how to check if the file has already been built?
Maybe check the process or file via exec startupProbe?
I'm not very good at using models.

I think it might be a good suggestion to see how we can better check that the model is built out or serviced properly. Because for me the current behaviour does seem a bit disconcerting for first time users.

[mudler]

@fyuan1316 that makes sense indeed - in LocalAI those endpoints are used for docker compose to wait and check when the service is healthy. any update on this PR?

[cryptk]

Regarding startupProbe, at first I noticed that the pod status was running but when I accessed the /v1/models service it didn't return properly. So my original expectation was that the startupProbe would indicate that the model was ready for service.

This isn't what startupProbes do. A startup probe merely delays the time that a liveness/readiness probe start running and is used for services that can take a while (and an unpredictable amount of time) to be ready for health checking to begin. LocalAI starts quite quicky, so startupProbes are not needed here.

Either way, the work here should be closed as superseded by #45 which has liveness/readiness probes as well as quite a few other enhancements.