Skip to content

Commit

Permalink
fix(ai): caching use deep copy for request table (#13582)
Browse files Browse the repository at this point in the history
Co-authored-by: Antoine Jacquemin <[email protected]>
  • Loading branch information
kikito and AntoineJac authored Aug 27, 2024
1 parent 7b01bad commit 8b2b384
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 2 deletions.
4 changes: 4 additions & 0 deletions changelog/unreleased/kong/ai-proxy-add-deep-copy-lib.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
message: |
**AI-proxy-plugin**: Replace the lib and use cycle_aware_deep_copy for the `request_table` object.
scope: Plugin
type: feature
4 changes: 2 additions & 2 deletions kong/llm/proxy/handler.lua
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ local cjson = require("cjson.safe")
local kong_utils = require("kong.tools.gzip")
local buffer = require "string.buffer"
local strip = require("kong.tools.utils").strip

local cycle_aware_deep_copy = require("kong.tools.table").cycle_aware_deep_copy

local EMPTY = require("kong.tools.table").EMPTY

Expand Down Expand Up @@ -346,7 +346,7 @@ function _M:access(conf)
end

request_table = kong.request.get_body(content_type, nil, conf.max_request_body_size)
llm_state.set_request_body_table(request_table)
llm_state.set_request_body_table(cycle_aware_deep_copy(request_table))
end

if not request_table then
Expand Down

0 comments on commit 8b2b384

Please sign in to comment.