Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cross modules stream/http shared dict #36

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

cross modules stream/http shared dict #36

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 77 additions & 8 deletions src/ngx_stream_lua_shdict.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,14 +306,79 @@ ngx_stream_lua_shdict_expire(ngx_stream_lua_shdict_ctx_t *ctx, ngx_uint_t n)


void
ngx_stream_lua_inject_shdict_api(ngx_stream_lua_main_conf_t *lmcf, lua_State *L)
ngx_stream_lua_inject_shdict_api(ngx_log_t *log,
ngx_stream_lua_main_conf_t *lmcf, lua_State *L)
{
ngx_stream_lua_shdict_ctx_t *ctx;
ngx_uint_t i;
ngx_shm_zone_t **zone;
ngx_uint_t i;
ngx_array_t all_zones;
ngx_module_t *http_module = NULL;
ngx_stream_lua_shdict_ctx_t *ctx;
ngx_shm_zone_t **zone, *shm_zone;
ngx_pool_t *temp_pool;
ngx_list_part_t *part;
ngx_module_t **modules;

/* Find ngx_http_lua_module */
modules = lmcf->cycle->modules;

for (i = 0; modules[i]; i++) {

if (ngx_strcmp(modules[i]->name, "ngx_http_lua_module") == 0) {
http_module = modules[i];
break;
}
}

/* place http and stream zones in a single array */
temp_pool = ngx_create_pool(NGX_DEFAULT_POOL_SIZE, log);

if (temp_pool == NULL) {
ngx_log_error(NGX_LOG_ERR, log, 1,
"error: pool for zone array not allocated");
return;
}

if (ngx_array_init(&all_zones, temp_pool, 2,
sizeof(ngx_shm_zone_t *)) != NGX_OK)
{
ngx_destroy_pool(temp_pool);
ngx_log_error(NGX_LOG_ERR, log, 1,
"error: zone array not allocated");
return;
}

part = &lmcf->cycle->shared_memory.part;
shm_zone = part->elts;

for (i = 0; /* void */ ; i++) {

if (lmcf->shm_zones != NULL) {
lua_createtable(L, 0, lmcf->shm_zones->nelts /* nrec */);
if (i >= part->nelts) {
if (part->next == NULL) {
break;
}
part = part->next;
shm_zone = part->elts;
i = 0;
}

if ((shm_zone->tag == http_module && shm_zone->tag != NULL)
|| shm_zone->tag == &ngx_stream_lua_module)
{
zone = ngx_array_push(&all_zones);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach is risky since the shdict implementation of ngx_http_lua_module might be different than ngx_stream_lua_module (due to different versions of these modules, for example). It will lead to hard-to-debug issues when these two modules' implementations are incompatible in terms of the shm zone storage layout.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The safest way, IMHO, is to abstract the shm dict implementation out and make both these ngx_xxx_lua modules share it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@agentzh You mean share the a common header file ? Where would this file reside ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alonbg I mean creating a ngx_meta_lua_module and make both ngx_http_lua_module and ngx_stream_lua_module depend on it.

Copy link
Author

@alonbg alonbg Sep 21, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@agentzh Yes this was your original idea :) but to avoid collision [and allow coexistence of old and new] meta's shared dict directive will need to have a different name or stay with the name lua_shared_dict but restricted to the configuration main part (outside both http and stream contexts). Which option do you prefer ?

Copy link
Author

@alonbg alonbg Sep 21, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The risk is still there I believe since the ngx_meta_lua_shdict_ctx_t which would be defined in all three modules might be different.


if (zone == NULL) {
ngx_destroy_pool(temp_pool);
ngx_log_error(NGX_LOG_ERR, log, 1,
"error: zone pointer not allocated");
return;
}

*zone = shm_zone;
}
}

if (all_zones.nelts > 0) {
lua_createtable(L, 0, all_zones.nelts /* nrec */);
/* ngx.shared */

lua_createtable(L, 0 /* narr */, 13 /* nrec */); /* shared mt */
Expand Down Expand Up @@ -357,11 +422,14 @@ ngx_stream_lua_inject_shdict_api(ngx_stream_lua_main_conf_t *lmcf, lua_State *L)
lua_pushvalue(L, -1); /* shared mt mt */
lua_setfield(L, -2, "__index"); /* shared mt */

zone = lmcf->shm_zones->elts;
zone = all_zones.elts;

for (i = 0; i < lmcf->shm_zones->nelts; i++) {
for (i = 0; i < all_zones.nelts; i++) {
ctx = zone[i]->data;

dd("injecting shared dict %.*s",
(int) ctx->name.len, ctx->name.data);

lua_pushlstring(L, (char *) ctx->name.data, ctx->name.len);
/* shared mt key */

Expand All @@ -381,6 +449,7 @@ ngx_stream_lua_inject_shdict_api(ngx_stream_lua_main_conf_t *lmcf, lua_State *L)
}

lua_setfield(L, -2, "shared");
ngx_destroy_pool(temp_pool);
}


Expand Down
4 changes: 2 additions & 2 deletions src/ngx_stream_lua_shdict.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ typedef struct {
ngx_int_t ngx_stream_lua_shdict_init_zone(ngx_shm_zone_t *shm_zone, void *data);
void ngx_stream_lua_shdict_rbtree_insert_value(ngx_rbtree_node_t *temp,
ngx_rbtree_node_t *node, ngx_rbtree_node_t *sentinel);
void ngx_stream_lua_inject_shdict_api(ngx_stream_lua_main_conf_t *lmcf,
lua_State *L);
void ngx_stream_lua_inject_shdict_api(ngx_log_t *log,
ngx_stream_lua_main_conf_t *lmcf, lua_State *L);


#endif /* _NGX_STREAM_LUA_SHDICT_H_INCLUDED_ */
Expand Down
2 changes: 1 addition & 1 deletion src/ngx_stream_lua_util.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2674,7 +2674,7 @@ ngx_stream_lua_inject_ngx_api(lua_State *L, ngx_stream_lua_main_conf_t *lmcf,

ngx_stream_lua_inject_req_api(log, L);
ngx_stream_lua_inject_variable_api(L);
ngx_stream_lua_inject_shdict_api(lmcf, L);
ngx_stream_lua_inject_shdict_api(log, lmcf, L);
ngx_stream_lua_inject_socket_tcp_api(log, L);
ngx_stream_lua_inject_socket_udp_api(log, L);
ngx_stream_lua_inject_uthread_api(log, L);
Expand Down