feat: support monitoring WebSocket endpoints (wss) #511

heitorPB · 2023-06-27T18:00:10Z

Summary

Solves #492

Checklist

Tested and/or added tests to validate that the changes work as intended, if applicable.
Updated documentation in README.md, if applicable.

README.md

WebSocket endpoints are automatically identified by the URL protocol specification: `wss://` or `ws://`. The request body is used as the "message" written to the server, and the answer is stored in the `[BODY]`. Optionally, the user can set the `jsonrpc` flag to automatically wrap the request body in a JSON RPC 2.0 method call.

- test we can identify the endpoint type for WebSockets based on the URL supplied: `wss://` (with SSL/TLS) and `ws://` (plain text). - test we can generate a JsonRPC 2.0 message via the new endpoint flag `JsonRPC`.

See https://en.wikipedia.org/wiki/JSON-RPC#Version_2.0

heitorPB · 2023-07-23T22:20:03Z

Current tests:

$ make test
go test ./... -cover
?       github.com/TwiN/gatus/v5        [no test files]
?       github.com/TwiN/gatus/v5/alerting       [no test files]
ok      github.com/TwiN/gatus/v5/alerting/alert (cached)        coverage: 100.0% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider      (cached)        coverage: 100.0% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/custom       (cached)        coverage: 95.1% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/discord      (cached)        coverage: 95.5% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/email        (cached)        coverage: 70.0% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/github       (cached)        coverage: 82.4% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/gitlab       (cached)        coverage: 85.7% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/googlechat   (cached)        coverage: 95.9% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/matrix       (cached)        coverage: 97.0% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/mattermost   (cached)        coverage: 95.6% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/messagebird  (cached)        coverage: 90.9% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/ntfy (cached)        coverage: 51.7% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/opsgenie     (cached)        coverage: 90.0% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/pagerduty    (cached)        coverage: 79.5% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/pushover     (cached)        coverage: 92.3% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/slack        (cached)        coverage: 95.3% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/teams        (cached)        coverage: 95.5% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/telegram     (cached)        coverage: 91.7% of statements
ok      github.com/TwiN/gatus/v5/alerting/provider/twilio       (cached)        coverage: 33.3% of statements
?       github.com/TwiN/gatus/v5/config/remote  [no test files]
?       github.com/TwiN/gatus/v5/core/ui        [no test files]
?       github.com/TwiN/gatus/v5/storage        [no test files]
?       github.com/TwiN/gatus/v5/storage/store/common   [no test files]
?       github.com/TwiN/gatus/v5/test   [no test files]
--- FAIL: TestPing (0.00s)
    client_test.go:69: expected true
    client_test.go:71: Round-trip time returned on success should've higher than 0
FAIL
        github.com/TwiN/gatus/v5/client coverage: 67.4% of statements
FAIL    github.com/TwiN/gatus/v5/client 3.967s
ok      github.com/TwiN/gatus/v5/config (cached)        coverage: 83.9% of statements
ok      github.com/TwiN/gatus/v5/config/connectivity    (cached)        coverage: 100.0% of statements
ok      github.com/TwiN/gatus/v5/config/maintenance     (cached)        coverage: 100.0% of statements
ok      github.com/TwiN/gatus/v5/config/ui      (cached)        coverage: 78.3% of statements
ok      github.com/TwiN/gatus/v5/config/web     (cached)        coverage: 81.8% of statements
ok      github.com/TwiN/gatus/v5/controller     (cached)        coverage: 78.6% of statements
ok      github.com/TwiN/gatus/v5/controller/handler     (cached)        coverage: 78.9% of statements
--- FAIL: TestIntegrationEvaluateHealthForICMP (0.00s)
    endpoint_test.go:741: Conditions '[CONNECTED] == true' should have been a success
    endpoint_test.go:744: Because the connection has been established, result.Connected should've been true
    endpoint_test.go:747: Because all conditions passed, this should have been a success
FAIL
        github.com/TwiN/gatus/v5/core   coverage: 85.9% of statements
FAIL    github.com/TwiN/gatus/v5/core   2.838s
ok      github.com/TwiN/gatus/v5/jsonpath       (cached)        coverage: 97.1% of statements
ok      github.com/TwiN/gatus/v5/metrics        (cached)        coverage: 100.0% of statements
ok      github.com/TwiN/gatus/v5/pattern        (cached)        coverage: 100.0% of statements
ok      github.com/TwiN/gatus/v5/security       (cached)        coverage: 61.2% of statements
ok      github.com/TwiN/gatus/v5/storage/store  (cached)        coverage: 93.5% of statements
ok      github.com/TwiN/gatus/v5/storage/store/common/paging    (cached)        coverage: 100.0% of statements
ok      github.com/TwiN/gatus/v5/storage/store/memory   (cached)        coverage: 88.8% of statements
ok      github.com/TwiN/gatus/v5/storage/store/sql      (cached)        coverage: 81.6% of statements
ok      github.com/TwiN/gatus/v5/util   (cached)        coverage: 100.0% of statements
ok      github.com/TwiN/gatus/v5/watchdog       (cached)        coverage: 51.3% of statements
ok      github.com/TwiN/gatus/v5/web    (cached)        coverage: [no statements]
FAIL
make: *** [Makefile:16: test] Error 1

Do you have any suggestions on how to test the new code in the core/websockets.go file? I'm not familiar with Golang, any suggestions/comments are greatly appreciated.

TwiN

I forgot to submit the PR review 🤦‍♂️
A little late and some of the comments may no longer be relevant, but here goes

core/endpoint.go

README.md

TwiN · 2023-07-24T02:29:48Z

@heitorPB To answer your questions on the two failing tests about ping/icmp -- feel free to ignore them for now. They're flaky and they don't work on everybody's local environment.

Move the `defer ws.Close()` to after opening the connection, so the socket is closed also in case of errors.

heitorPB · 2023-07-25T15:32:55Z

@TwiN Thank you for the feedback :) I addressed your points in the last commits.

heitorPB · 2023-07-25T20:29:31Z

Btw, I was told that the websocket library used here is not maintained, and was suggested Gorrilla instead, which is/was also unmaintaned. What are your thoughts on this?

Edit: found a maintained fork o Gorilla/websocket: https://github.com/fasthttp/websocket

core/websocket.go

The connection should only be closed if successfully opened.

TwiN · 2023-08-03T22:31:11Z

core/websocket.go

+package core
+
+import (
+	"golang.org/x/net/websocket"
+)
+
+func queryWebSocket(endpoint *Endpoint, result *Result) {
+	const (
+		Origin             = "http://localhost/"
+		MaximumMessageSize = 1024 // in bytes
+	)


I'm a bit conflicted here, because historically, actual calls were made from the client package here: https://github.com/TwiN/gatus/blob/master/client/client.go as opposed to the core package, but if you used the dns file as an example, I can see why you'd put it in the core package instead.

Still, could you move this functionality in the client package instead? I'll take care of the dns one separately to avoid future confusion

Yeah, I used the DNS as example.

I can move the code to the client package, no problem. Will move the function there and follow the same terminology. I should rename the function to CanCreateWebSocketConnection?

Also, do you want me to rebase this branch on top of current master? I saw that you merged master in here.

Naming it QueryWebsocket is fine; you'll have to modify the function's signature so that it doesn't pass core.Endpoint and core.Result though, just like the other functions in the client package.

As for the rebase question- I'll squash when I merge anyways, so don't worry about it.

I moved the function to the client package. Had to add other arguments and return to accomododate a body and a return message. Hope it is fine, but if you know of a better way I can rework it.

And a noob question: how do I compile this branch?

$ go build -a -installsuffix cgo -o gatus . # github.com/TwiN/gatus/v5/core core/endpoint.go:347:47: undefined: client.queryWebSocket

Oh, I needed to have the function in Uppercase.

codecov-commenter · 2023-08-03T22:34:40Z

Codecov Report

Patch coverage: 5.12% and project coverage change: -0.78% ⚠️

Comparison is base (640e455) 81.65% compared to head (b562869) 80.88%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #511      +/-   ##
==========================================
- Coverage   81.65%   80.88%   -0.78%     
==========================================
  Files          56       57       +1     
  Lines        4350     4389      +39     
==========================================
- Hits         3552     3550       -2     
- Misses        617      655      +38     
- Partials      181      184       +3

Files Changed	Coverage Δ
core/websocket.go	`0.00% <0.00%> (ø)`
core/endpoint.go	`73.44% <40.00%> (-1.14%)`	⬇️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

client/client.go

TwiN · 2023-08-09T02:12:53Z

@heitorPB Thank you for the contribution!

heitorPB · 2023-08-09T12:19:19Z

Thank you for Gatus and the super valuable feedback you left here, @TwiN! I'll keep this in mind for the next PR :D

TwiN reviewed Jun 28, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

heitorPB force-pushed the websockets-support branch from 2c8af67 to ac2b573 Compare June 30, 2023 16:02

heitorPB added 2 commits July 5, 2023 10:28

core:websocket: close connection after using it

33994a9

heitorPB force-pushed the websockets-support branch from ac2b573 to 33994a9 Compare July 5, 2023 13:29

heitorPB added 2 commits July 23, 2023 19:11

test: add tests related to WebSocket support

776eb2e

- test we can identify the endpoint type for WebSockets based on the URL supplied: `wss://` (with SSL/TLS) and `ws://` (plain text). - test we can generate a JsonRPC 2.0 message via the new endpoint flag `JsonRPC`.

core:endpoint: fix name of jsonrpc parameter

41f5a3b

See https://en.wikipedia.org/wiki/JSON-RPC#Version_2.0

TwiN reviewed Jul 24, 2023

View reviewed changes

core/endpoint.go Outdated Show resolved Hide resolved

core/endpoint.go Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

heitorPB added 2 commits July 25, 2023 12:28

core:websocket: fix dangling open connection on error

8ff65b1

Move the `defer ws.Close()` to after opening the connection, so the socket is closed also in case of errors.

remove jsonrpc flag

1b7b6a5

heitorPB commented Jul 26, 2023

View reviewed changes

core/websocket.go Outdated Show resolved Hide resolved

core:websocket: fix nil pointer dereference

17c04dc

The connection should only be closed if successfully opened.

TwiN requested changes Aug 3, 2023

View reviewed changes

Merge branch 'master' into websockets-support

b562869

heitorPB added 3 commits August 7, 2023 15:44

Move websocket function to client

4ddd049

update go.mod

f7d9eb2

Fix build errors

d28ddb1

TwiN approved these changes Aug 9, 2023

View reviewed changes

Merge branch 'master' into websockets-support

7ef3efa

TwiN reviewed Aug 9, 2023

View reviewed changes

client/client.go Outdated Show resolved Hide resolved

TwiN reviewed Aug 9, 2023

View reviewed changes

client/client.go Outdated Show resolved Hide resolved

Fix errors

d5fca93

TwiN reviewed Aug 9, 2023

View reviewed changes

client/client.go Outdated Show resolved Hide resolved

Update client/client.go

5a208f2

TwiN merged commit 5f69351 into TwiN:master Aug 9, 2023
1 check passed

TwiN added the feature New feature or request label Aug 9, 2023

heitorPB mentioned this pull request Aug 9, 2023

Websockets support #492

Closed

TwiN changed the title ~~feat: support monitoring WebSocket endpoints~~ feat: support monitoring WebSocket endpoints (wss) Dec 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support monitoring WebSocket endpoints (wss) #511

feat: support monitoring WebSocket endpoints (wss) #511

heitorPB commented Jun 27, 2023 •

edited

Loading

heitorPB commented Jul 23, 2023

TwiN left a comment

TwiN commented Jul 24, 2023 •

edited

Loading

heitorPB commented Jul 25, 2023

heitorPB commented Jul 25, 2023 •

edited

Loading

TwiN Aug 3, 2023

heitorPB Aug 4, 2023

heitorPB Aug 4, 2023

TwiN Aug 4, 2023

heitorPB Aug 7, 2023

heitorPB Aug 8, 2023

codecov-commenter commented Aug 3, 2023

TwiN commented Aug 9, 2023

heitorPB commented Aug 9, 2023

feat: support monitoring WebSocket endpoints (wss) #511

feat: support monitoring WebSocket endpoints (wss) #511

Conversation

heitorPB commented Jun 27, 2023 • edited Loading

Summary

Checklist

heitorPB commented Jul 23, 2023

TwiN left a comment

Choose a reason for hiding this comment

TwiN commented Jul 24, 2023 • edited Loading

heitorPB commented Jul 25, 2023

heitorPB commented Jul 25, 2023 • edited Loading

TwiN Aug 3, 2023

Choose a reason for hiding this comment

heitorPB Aug 4, 2023

Choose a reason for hiding this comment

heitorPB Aug 4, 2023

Choose a reason for hiding this comment

TwiN Aug 4, 2023

Choose a reason for hiding this comment

heitorPB Aug 7, 2023

Choose a reason for hiding this comment

heitorPB Aug 8, 2023

Choose a reason for hiding this comment

codecov-commenter commented Aug 3, 2023

Codecov Report

TwiN commented Aug 9, 2023

heitorPB commented Aug 9, 2023

heitorPB commented Jun 27, 2023 •

edited

Loading

TwiN commented Jul 24, 2023 •

edited

Loading

heitorPB commented Jul 25, 2023 •

edited

Loading