Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce structured logging #32

Merged
merged 11 commits into from
Nov 21, 2023
Merged

Introduce structured logging #32

merged 11 commits into from
Nov 21, 2023

Conversation

bomoko
Copy link
Contributor

@bomoko bomoko commented Nov 20, 2023

Logging is currently a shambles for the insights-handler service.
This is a first PR to both clean up logging generally, and introduce structured logging into the service with log levels.

We're moving from logs that look like

2023/11/20 12:35:45 PROBLEMS FROM SBOM - enabled
Disabled S3 upload is true
2023/11/20 12:35:45 Registering Fact Filters/Transformer
2023/11/20 12:35:45 insights-handler running...
2023/11/20 12:35:45 Listening for messages in queue lagoon-insights:items
2023/11/20 12:35:58 [DEBUG] insights: {sbom-gz BINARY_PAYLOAD SBOM   FACTS json application/json false}
2023/11/20 12:35:58 [DEBUG] target: {organization-cotton main nginx }
2023/11/20 12:35:58 error: unable to determine resource destination (does organization-cotton:main exist?): returned error 404 Not Found: <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot POST /graphql</pre>
</body>
</html>

to

time=2023-11-20T11:56:42.412+13:00 level=DEBUG msg=problemsFromSBOM status=true
time=2023-11-20T11:56:42.412+13:00 level=DEBUG msg=disableS3Upload status=true
time=2023-11-20T11:56:42.416+13:00 level=INFO msg="Listening for messages" queue=lagoon-insights:items
time=2023-11-20T12:27:55.744+13:00 level=DEBUG msg="Facts info" data="{ProjectName:high-cotton EnvironmentName:Master EnvironmentId:3 Facts:[{EnvironmentId:3 ProjectName:high-cotton EnvironmentName:Master Name:drupal-core Value:9.0.1 Description:Drupal CMS version found on environment Type:TEXT Category:Framework Service:cli Source:} {EnvironmentId:3 ProjectName:high-cotton EnvironmentName:Master Name:php-version Value:8.0.3 Description:PHP version found on environment Type:TEXT Category:Programming language Service:cli Source:}] Type:direct.facts InsightsType:}"
time=2023-11-20T12:27:55.752+13:00 level=ERROR msg="Error deleting facts from source" Project=high-cotton Environment=Master Source="" Error="returned error 404 Not Found: <!DOCTYPE html>\n<html lang=\"en\">\n<head>\n<meta charset=\"utf-8\">\n<title>Error</title>\n</head>\n<body>\n<pre>Cannot POST /graphql</pre>\n</body>\n</html>\n"

@bomoko bomoko removed the request for review from shreddedbacon November 20, 2023 01:27
bomoko
bomoko commented Nov 20, 2023
View reviewed changes
Copy link
Contributor Author

@bomoko bomoko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this is merged, it will be important to spin out the Environment attributes into EnvironmentId and EnvironmentName where applicable

@bomoko
Copy link
Contributor Author

bomoko commented Nov 20, 2023 •
edited
Loading

Before this is merged, it will be important to spin out the Environment attributes into EnvironmentId and EnvironmentName where applicable

Should be done now

@bomoko bomoko merged commit 6c6a9ce into main Nov 21, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tobybellwood tobybellwood Awaiting requested review from tobybellwood

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@bomoko