feat(ffi) expose kv & metrics shms through FFI #582

casimiro · 2024-08-08T14:29:49Z

No description provided.

codecov · 2024-08-08T14:44:20Z

Codecov Report

Attention: Patch coverage is 76.79558% with 84 lines in your changes missing coverage. Please review.

Project coverage is 62.97581%. Comparing base (96b4e27) to head (12dd92b).
Report is 12 commits behind head on main.

Files with missing lines	Patch %	Lines
src/common/proxy_wasm/ngx_proxy_wasm_host.c	42.85714%	28 Missing ⚠️
src/common/shm/ngx_wa_shm_kv.c	71.64179%	19 Missing ⚠️
src/common/shm/ngx_wa_shm_queue.c	26.92308%	19 Missing ⚠️
src/common/metrics/ngx_wa_metrics.c	76.08696%	11 Missing ⚠️
src/common/metrics/ngx_wa_histogram.c	68.42105%	6 Missing ⚠️
src/common/shm/ngx_wa_shm.c	96.42857%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (96b4e27) and HEAD (12dd92b). Click for more details.

HEAD has 14 uploads less than BASE

Flag BASE (96b4e27) HEAD (12dd92b)

valgrind 5 1

unit 10 0

Additional details and impacted files

@@                 Coverage Diff                  @@
##                main        #582          +/-   ##
====================================================
- Coverage   90.60900%   62.97581%   -27.63319%     
====================================================
  Files             49          49                  
  Lines          10936        9510        -1426     
====================================================
- Hits            9909        5989        -3920     
- Misses          1027        3521        +2494

Files with missing lines	Coverage Δ
src/common/lua/ngx_wasm_lua_ffi.c	`94.05405% <100.00000%> (+1.13370%)`	⬆️
src/common/lua/ngx_wasm_lua_ffi.h	`100.00000% <100.00000%> (ø)`
src/common/metrics/ngx_wa_metrics.h	`100.00000% <100.00000%> (ø)`
src/common/shm/ngx_wa_shm.h	`100.00000% <100.00000%> (ø)`
src/ngx_wasmx.c	`82.78689% <100.00000%> (-6.60706%)`	⬇️
src/wasm/ngx_wasm.h	`90.90909% <ø> (-9.09091%)`	⬇️
src/wasm/ngx_wasm_core_module.c	`97.14286% <ø> (+4.49580%)`	⬆️
src/wasm/ngx_wasm_directives.c	`35.08772% <100.00000%> (-61.42970%)`	⬇️
src/common/shm/ngx_wa_shm.c	`95.83333% <96.42857%> (ø)`
src/common/metrics/ngx_wa_histogram.c	`67.30769% <68.42105%> (-31.78323%)`	⬇️
... and 4 more

... and 37 files with indirect coverage changes

Flag	Coverage Δ
unit	`?`
valgrind	`62.97581% <76.79558%> (-19.08449%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

src/common/shm/ngx_wasm_shm.c

src/ngx_wasmx.h

lib/resty/wasmx/shm.lua

src/common/lua/ngx_wasm_lua_ffi.c

src/common/metrics/ngx_wa_metrics.h

lib/resty/wasmx/shm.lua

docs/PROXY_WASM.md

src/common/metrics/ngx_wa_histogram.c

src/common/lua/ngx_wasm_lua_ffi.c

This corrects metrics code to properly use ngx_wasm_shm facilities for shm zone handling.

Previously `ngx_wa_metrics_get` would refuse to return a histogram and return an NGX_ABORT instead. This was due to Proxy-Wasm not supporting histogram retrieval. However, the upcoming FFI interface to shms will allow histograms to be retrieved. Also merges `ngx_wa_metrics_histogram.h` into `ngx_wa_metrics.h` as histogram declaration was already in `ngx_wa_metrics.h` and removing `ngx_wa_metrics_histogram.h` simplifies usage of the lib.

After changing the metrics library to rely on shm facilities to store its shm zone, support for the IPC subsystem in shm became necessary.

Iterating a tree in batches without knowing the number of elements in it implies an additional traversal through its elements after the last batch is retrieved. This last traversal returns no elements which signals the end of the iteration. Knowing the number of entries in the tree renders the additional traverse unnecessary, when the last batch is retrieved it can be verified that the tree has been fully iterated.

casimiro force-pushed the feat/metrics_lua_interface branch from a1b9043 to 5384812 Compare August 8, 2024 14:36

casimiro force-pushed the feat/metrics_lua_interface branch from 5384812 to 78097bb Compare August 8, 2024 14:56

thibaultcha reviewed Aug 8, 2024

View reviewed changes

src/common/shm/ngx_wasm_shm.c Outdated Show resolved Hide resolved

casimiro force-pushed the feat/metrics_lua_interface branch 2 times, most recently from f79dad6 to 239bc8c Compare August 8, 2024 22:19

casimiro added the pr/work-in-progress PR: Work in progress label Aug 8, 2024

thibaultcha reviewed Aug 9, 2024

View reviewed changes

src/ngx_wasmx.h Outdated Show resolved Hide resolved

casimiro force-pushed the feat/metrics_lua_interface branch 2 times, most recently from ae9316e to dc68e4a Compare August 12, 2024 21:14

thibaultcha reviewed Aug 13, 2024

View reviewed changes

thibaultcha changed the title ~~feat(shm/ffi) expose kv/metrics read-only operations to Lua land~~ feat(ffi) expose kv & metrics shms through FFI Aug 13, 2024

casimiro force-pushed the feat/metrics_lua_interface branch from dc68e4a to 698a670 Compare August 13, 2024 19:59

casimiro commented Aug 13, 2024

View reviewed changes