Client logging in case of errors

[miketsts]

A client calls the logger only in case of successful request completion, no matter the HTTP status is successful or not.
However, in case of an error, for example, httplib::Read the logger is not called. As a result, my application log does not contain a log record of a request content.
Well, I can of course add this capability to my application logic. But don't you think this could be better addressed in a library, if it in any case supports logging?

To tell the truth, I don't know what is the correct solution, I'm thinking loud ...

[yhirose]

@miketsts, yes, the current log interface is only for HTTP level information, not TCP/SSL level. If the TCP/SSL level logger is implemented, we have to introduce another log interface for this purpose, but we get two different log streams and hard to tell the actual order of each item in both streams. Another way is to integrate the TCP/SSL error into the current log interface, but it'll break the current interface and make it more complicated.

I don't come up with a good solution, yet... What do you know other HTTP server libraries are doing in this regard? Thanks for your help!

[miketsts]

@yhirose, unfortunately, I don't have experience with other libraries, at least on a client side. This is why I chose cpp-httplib, and I am pretty much happy with it :-)
Before I continue, let me specify yet another problem with logging that I see. The current logging approach when both request and response are provided at the end of a request processing, ignores the timestamp issue which is important for logging. I mean, a log record always has a timestamp of the end of the operation. This is often insufficient.

When I implemented logging in my application (in another interface, not HTTP), I separated the request and response. In such case it's always clear what has been sent, and what was the response, including errors.
Of course, there should be a way to make a relation between the two records. In my application it was just a serial number that I've maintained together with the request. Or it can be another application supplied ID. And then this ID appears in the log interface. Or, the log interface can create a special prefix string for the log record, etc.

[miketsts]

I actually found my own another older implementation. It is server side processing, but the principle can be the same.
In each request handling function I create an object of a type RequestLogger at the beginning of the processing. Its constructor outputs a starting log record. The object is deleted at the end of the processing - and the ending log record output from the destructor. My RequestLogger object accepts an ID as a construction parameter, it's actually a serial number maintained by the application. This ID is used as a part of a formatted log string.
This is more or less the design. Of course, there are more implementation details and decisions to make.

[miketsts]

An additional idea came to my mind. Currently, the user supplied logger callback has two parameters - request and response. We can add an additional parameter: event, which can be an enum having different values, like "start processing", "end processing", etc. The enum can be extended in the future, while the user callback can decide which events it wishes to log, and which it does not.
For completeness, there can be also a kind of "aux" parameter. While I currently don't see a need in it, maybe some future "events" would require additional information.
This is of course incompatible with the current logger interface. But I think the compatibility can be managed.

[yhirose]

@yhirose, I did some research on this topic by looking at what Apache and Nginx are doing.

They both maintain 'access log' and 'error log', and it looks the way I should also take. I don't have time to work on it right now, but I'll get back to this issue sometime in April.

[miketsts]

@yhirose I understand what you mean.
From my POV of an application developer, I have a single application log, which I can filter on different criteria like severity or event being logged. So I will route the library logging to my application log anyway, let it be two callbacks instead of one. And it's a pity I still will not be able to log start/end of HTTP processing - I will have to implement it on the application level.
Thank you for looking into this issue.

[SeanTolstoyevski]

Hi @yhirose ,

I wanted to ask here instead of opening a new issue.

Is there currently a built-in logging mechanism that provides information about requests on the client/server side?
What I want to do is to see information about the request (post/get method, path, duration for the request ex. 50 ms, etc.) based on the condition (debug or release), like Gin's debugmode in Golang.

What kind of mechanism should I set up for this?