custom proxy_access_log still not working in 3.4.*

[ravigunasekara]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

kong/kong-gateway:3.4.3.6

Current Behaviour

docker run fails with nginx: [emerg] unknown log format "custom-json" in /usr/local/kong/nginx-kong.conf

I have set environment variable KONG_PROXY_ACCESS_LOG=/dev/stdout\ customer-json \ as per the instructions in the issue #12061.

Also custom-json.conf is included in nginx.conf http block.
http { include 'custom-json.conf'; include 'nginx-kong.conf'; }

Below is how custom-json.conf looks like.
log_format custom-json escape=json '{' '"consumer":"$http_x_consumer_username",' '"consumer_id":"$http_x_consumer_id",' '}';

Expected Behaviour

docker-compose run successfully build the image and able to run the container with the new log formatter.

Steps To Reproduce

1. Update your Docker file

FROM kong/kong-gateway:3.4.3.6 
ENV APP_HOME=/app \
    KONG_DATABASE=off \
    KONG_DECLARATIVE_CONFIG=/tmp/kong.yml \
    KONG_PROXY_ACCESS_LOG=/dev/stdout\ customer-json \
    KONG_ADMIN_ACCESS_LOG=/dev/stdout \
    KONG_PREFIX="/usr/local/kong/"

2. include custom log formatter into the kong.conf

http {
    include 'custom-json.conf';
    include 'nginx-kong.conf';
}

3. For an example your custom-json.conf looks like this (this could be any format)

log_format customer-json escape=json '{'
    '"consumer":"$http_x_consumer_username",'
    '"consumer_id":"$http_x_consumer_id",'
'}';

4. run docker-compose build
5. Failed with nginx: [emerg] unknown log format "customer-json" in /usr/local/kong/nginx-kong.conf

Anything else?

It seems that Kong always expects log_format definitions to be inside the nginx-kong.conf. If we create a new log formatter and add it to the http block inside the kong.conf, Kong fails to fetch it. Normally, nginx accepts custom log formatters via the http block. There is a simple workaround to overcome this issue, but I would like to know the recommended solution.

[ravigunasekara]

I am sharing the workaround I have implemented in case others experience the same issues.
I have modified Dockerfile in a way that it override the current access_log in nginx-kong.conf file that it uses my own log formatter(customer-json).
Added below line to Dockerfile

RUN set -xe && \
    kong prepare -p "${KONG_PATH}" && \
		sed -i "s|access_log /dev/stdout kong_log_format|access_log /dev/stdout customer-json|" ${KONG_PATH}/nginx-kong.conf && \
    true	

Hope this helps :)

[samugi]

Hi @ravigunasekara, thank you for opening this issue.

I couldn't reproduce it in 3.4.3.6, I noticed, however, that you have defined a custom format named custom-json and then configured a slightly different name in your environment variable: KONG_PROXY_ACCESS_LOG=/dev/stdout\ customer-json, i.e. customer instead of custom. Could that be the problem?

Thank you.

[github-actions]

This issue is marked as stale because it has been open for 14 days with no activity.

[github-actions]

Dear contributor,

We are automatically closing this issue because it has not seen any activity for three weeks.
We're sorry that your issue could not be resolved. If any new information comes up that could
help resolving it, please feel free to reopen it.

Your contribution is greatly appreciated!

Please have a look
our pledge to the community
for more information.

Sincerely,
Your Kong Gateway team

[ravigunasekara]

Thanks and Sorry for the delayed reply @samugi. I was holidaying and just came back. I have setup my environment variable such a way (KONG_PROXY_ACCESS_LOG=/dev/stdout\ customer-json) because my custom-json.conf file defines log_format as customer-json (According to the documentation, I believe we need to set the log_format attribute name, not the file name). I will attempt to recreate this issue in a small project and provide a GitHub link for you to review. It will help to determine if I am doing something differently from your setup.