request_openai removes the query parameter api-version for an AzureProvider

[pabvald]

Cc: @svilupp

While developing the support for Azure OpenAI in PromptingTools.jl, I came across the following issue. If I use the OpenAI.create_chat with an AzureProvider the ?api-version=2023-03-15-preview is removed from the URL:

julia> provider.base_url
"https://<resource-name>.openai.azure.com/openai/deployments/gpt-4o-2"

julia> OpenAI.build_url(provider, "chat/completions")
"https://<resource-name>.openai.azure.com/openai/deployments/gpt-4o-2/chat/completions?api-version=2023-03-15-preview"

julia> OpenAI.create_chat(provider, model, messages)
ERROR: HTTP.Exceptions.StatusError(404, "POST", "/openai/deployments/gpt-4o-2/chat/completions", HTTP.Messages.Response:
"""
HTTP/1.1 404 Not Found
Content-Length: 56
Content-Type: application/json
apim-request-id: 5236bf72-58a1-4440-95aa-eb5220891959
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
x-content-type-options: nosniff
Date: Tue, 24 Sep 2024 07:38:29 GMT

{"error":{"code":"404","message": "Resource not found"}}""")
...

See how even though the result of OpenAI.build_url(provider, "chat/completions") includes the query parameter:

julia> OpenAI.build_url(provider, "chat/completions")
"https://<resource-name>.openai.azure.com/openai/deployments/gpt-4o-2/chat/completions?api-version=2023-03-15-preview"

the URL used in the request doesn't:

ERROR: HTTP.Exceptions.StatusError(404, "POST", "/openai/deployments/gpt-4o-2/chat/completions", 

To get the correct URL, I need to use OpenAI.openai_request pasing the argument query = Dict("api-version" => provider.api_version):

julia> OpenAI.openai_request(
               "/chat/completions",
               provider;
               method = "POST",
               http_kwargs = NamedTuple(),
               messages = messages,
               query = Dict("api-version" => provider.api_version),
               streamcallback = nothing
           )
OpenAIResponse{JSON3.Object{Vector{UInt8}, Vector{UInt64}}}(200, {
              "choices": [
                           {
                              "finish_reason": "stop",
                                      "index": 0,
                                    "message": {
                                                  "content": "I'm just a computer program, so I don't have feelings, but I'm here and ready to help you! How can I assist you today?",
                                                     "role": "assistant"
                                               }
                           }
                         ],
              "created": 1727163808,
                   "id": "chatcmpl-AAuUik0iKDBdWfFyZBy6UZcoAFSWl",
                "model": "gpt-4o-2024-05-13",
               "object": "chat.completion",
   "system_fingerprint": "fp_67802d9a6d",
                "usage": {
                            "completion_tokens": 28,
                                "prompt_tokens": 12,
                                 "total_tokens": 40
                         }
})

The issue seems to come from the HTTP.request function, which removes the query parameters from the URL and only includes them if they are provided through the query argument. The issue doesn't occur when using HTTP.post.

[svilupp]

Thanks for filing the bug! Doesn't this belong to HTTP.jl if it's the HTTP behavior that's causing the error?

EDIT: Note that HTTP.post is just a shorthand for the same function call -- it shouldn't change the behavior: https://github.com/JuliaWeb/HTTP.jl/blob/6d4db2ad694b6a53eaaae6d54adfe1158867dfb9/src/HTTP.jl#L532

[pabvald]

It might not be a bug in HTTP.jl, but the desired behaviour of the function. OpenAI.jl are surely not aware of this.

[pabvald]

@svilupp I have just checked the functionality of HTTP.post and HTTP.request and they both work fine. The issue seems to be in how OpenAI.jl uses the functions:

Setup:

Create the provider:

julia> const AZURE_OPENAI_API_KEY = "your-api-key"
julia> const AZURE_OPENAI_HOST = "https://<resource-name>.openai.azure.com"
julia> model = "gpt-4o-2"

julia> provider = OpenAI.AzureProvider(;
              api_key = AZURE_OPENAI_API_KEY,
              base_url =  AZURE_OPENAI_HOST * "/openai/deployments/$model"
          )
julia> provider.api_version
"2023-03-15-preview"
julia> OpenAI.build_url(provider, "chat/completions")
"https://<resource-name>.openai.azure.com/openai/deployments/gpt-4o-2/chat/completions?api-version=2023-03-15-preview"

Build the request:

julia> messages = [Dict("role"=> "user", "content" => "How are you doing?")]
julia> body = JSON.json(Dict("messages" => messages))
julia> headers = Dict("Content-Type" => "application/json", "api-key" => AZURE_OPENAI_API_KEY)

HTTP.post:

julia> HTTP.post(OpenAI.build_url(provider, "chat/completions"), headers, body)
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Content-Length: 435
Content-Type: application/json
x-ms-region: Sweden Central
apim-request-id: df973a26-ba7c-4604-827e-e095129fd54c
x-ratelimit-remaining-requests: 9
x-accel-buffering: no
x-ms-rai-invoked: true
x-request-id: 19050062-405e-46c3-9091-072ade31b37e
x-ms-client-request-id: df973a26-ba7c-4604-827e-e095129fd54c
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
azureml-model-session: d162-20240922163921
x-content-type-options: nosniff
x-ratelimit-remaining-tokens: 9355
Date: Tue, 24 Sep 2024 08:08:29 GMT

{"choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hello! I'm here and ready to help you with any questions or information you may need. How can I assist you today?","role":"assistant"}}],"created":1727165309,"id":"chatcmpl-AAusvjUDl8jb8hpAy552aubHGHxCy","model":"gpt-4o-2024-05-13","object":"chat.completion","system_fingerprint":"fp_67802d9a6d","usage":{"completion_tokens":25,"prompt_tokens":12,"total_tokens":37}}
"""

HTTP.request:

julia> HTTP.request("POST", OpenAI.build_url(provider, "chat/completions"), headers, body)
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Content-Length: 446
Content-Type: application/json
x-ms-region: Sweden Central
apim-request-id: d31d754a-0922-494d-8975-8bcbb754d889
x-ratelimit-remaining-requests: 9
x-accel-buffering: no
x-ms-rai-invoked: true
x-request-id: 14e19d54-f77b-4d46-bca5-8647a85735ac
x-ms-client-request-id: d31d754a-0922-494d-8975-8bcbb754d889
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
azureml-model-session: d159-20240922131454
x-content-type-options: nosniff
x-ratelimit-remaining-tokens: 8710
Date: Tue, 24 Sep 2024 08:08:53 GMT

{"choices":[{"finish_reason":"stop","index":0,"message":{"content":"I'm just a computer program, so I don't have feelings, but I'm here and ready to help you! What can I assist you with today?","role":"assistant"}}],"created":1727165334,"id":"chatcmpl-AAutK8yYYRSQBB0jQEtEaVyX2xK5y","model":"gpt-4o-2024-05-13","object":"chat.completion","system_fingerprint":"fp_67802d9a6d","usage":{"completion_tokens":29,"prompt_tokens":12,"total_tokens":41}}
"""

No need to provide the query parameter to HTTP.request: the URL keeps the ?api-version=2023-03-15-preview

[svilupp]

I suspect it might be this override:

OpenAI.jl/src/OpenAI.jl

Line 80 in 762daad

query = isnothing(query) ? [] : query

But it was introduced after I've used Azure API, so that might be what broke it.
git blame shows #57 from @cpfiffer

@cpfiffer, is there a reason to force the empty vector in query in the above line (query = isnothing(query) ? [] : query )? We suspect it overrides default HTTP.jl behavior and breaks AzureProvider implementation.

[cpfiffer]

Not that I can recall. I think to fix it we should probably have a query parser in there (or use whatever HTTP.jl does), but it's probably fine to just remove it and see what happens. Everything in there should be tested as far as I remember.

[svilupp]

My understanding is different -- that line should be removed. This is all handled natively by HTTP.jl.

See the following:

using HTTP
using HTTP.URIs
# used by `request_uri`
target = URI("https://httpbin.org/get?a=1&b=2")
@info "URIs" target.host target.path target.query
┌ Info: URIs
│   target.host = "httpbin.org"
│   target.path = "/get"
└   target.query = "a=1&b=2"

If you leave query as nothing, URL package will extract it easily in the HTTP stack.
If you provide query=[], you override the extracted query and effectively remove it.

I'd suggest opening a PR to remove this line. Anyone against it?