feat(prometheus): add AI metrics and fix #9320

[AntoineJac]

Summary

This PR is:

• adding AI metrics enable for Prometheus
• fixing a name in the metrics with s missing in tokenS
• simplify the logics

Logs format will be:

"ai": {
    "ai-proxy": {
      "cache": {
        "cache_status": "",
        "vector_db": "",
        "embeddings_provider": "",
        "embeddings_model": "",
      },
      "usage": {
        "total_tokens": 40,
        "prompt_tokens": 28,
        "completion_tokens": 12,
        "cost": 0.00104
      },
      "meta": {
        "azure_api_version": "2023-05-15",
        "request_model": "gpt-35-turbo",
        "provider_name": "azure",
        "azure_instance_id": "ai-proxy-regression",
        "response_model": "gpt-35-turbo",
        "plugin_id": "1fdf6955-6243-4298-8754-5142596c9a00",
        "azure_deployment_id": "kong-gpt-3-5"
      }
    }
  },

Promotheus metrics:

kong_ai_requests_total{ai_provider="azure",ai_model="gpt-35-turbo",cache_status="",vector_db="",embeddings_provider="",embeddings_model="",workspace="default"} 1

kong_ai_cost_total{ai_provider="azure",ai_model="gpt-35-turbo",cache_status="",vector_db="",embeddings_provider="",embeddings_model="",workspace="default"} 0.00104

kong_ai_tokens_total{ai_provider="azure",ai_model="gpt-35-turbo",cache_status="",vector_db="",embeddings_provider="",embeddings_model="",token_type="prompt_tokens",workspace="default"} 12
kong_ai_tokens_total{ai_provider="azure",ai_model="gpt-35-turbo",cache_status="",vector_db="",embeddings_provider="",embeddings_model="",token_type="completion_tokens",workspace="default"} 25
kong_ai_tokens_total{ai_provider="azure",ai_model="gpt-35-turbo",cache_status="",vector_db="",embeddings_provider="",embeddings_model="",token_type="total_tokens",workspace="default"} 37

Here is the Grafana AI template for these metrics:
https://grafana.com/grafana/dashboards/21162-kong-cx-ai/

Checklist

• The Pull Request has tests
• A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
• There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference

Fix AG-56

[fffonion]

should we use llm to better describe the usage (in case we went into image/video/audio area in the future), something like
kong_ai_llm_tokens_total etc, or maybe use a type="llm" in label.

[AntoineJac]

@fffonion agree I will use llm in the metrics name so "kong_ai_llm_tokens_total" to avoid increasing the cardinality of the metrics as it impacts performance for every new addition. Thanks

[AntoineJac]

@fffonion , I spend more time on this. Actually the tokens/cost logic is currently the same even with multi modal api.

Let's keep kong_ai_tokens_total and maybe we will add a new type in the future if cost logic is evolving

[fffonion]

@AntoineJac Multi-modal or text only can all be called "LLMs", I was looking at something like image or audio only, say I send some texts to let some service to generate a TTS audio, and there might be not a standard API format (like currently those LLM models have) yet. Both LLMs and others can be categorized as "AI". So I would like keep the naming constraints
limited to our current scope. Although "AI" in Kong currently just means LLM, but it may change in the future.

[AntoineJac]

@fffonion noted thanks. I have made the changes and fixed the conflicts with the new init.lua file

[team-gateway-bot]

Cherry-pick failed for master, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally.

git remote add upstream https://github.com/kong/kong-ee
git fetch upstream master
git worktree add -d .worktree/cherry-pick-13148-to-master-to-upstream upstream/master
cd .worktree/cherry-pick-13148-to-master-to-upstream
git checkout -b cherry-pick-13148-to-master-to-upstream
ancref=$(git merge-base 99cb0608050cbd14611e7795960aade9fc6360cc 486f6ebeb03811300129d5ca10b65108dfee7633)
git cherry-pick -x $ancref..486f6ebeb03811300129d5ca10b65108dfee7633

[AntoineJac]

Here is the cherry pick:
https://github.com/Kong/kong-ee/pull/9592