Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ v3 (feature): client refactor #1986

Merged
merged 143 commits into from
Mar 4, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
143 commits
Select commit Hold shift + click to select a range
9f3975f
✨ v3: Move the client module to the client folder and fix the error
wangjq4214 Jul 27, 2022
1af81b8
✨ v3: add xml encoder and decoder
wangjq4214 Jul 29, 2022
894777f
🚧 v3: design plugin and hook mechanism, complete simple get request
wangjq4214 Jul 30, 2022
cf5fa5e
🚧 v3: reset add some field
wangjq4214 Jul 31, 2022
b6c6f2a
🚧 v3: add doc and fix some error
wangjq4214 Jul 31, 2022
f5c8e52
🚧 v3: add header merge
wangjq4214 Jul 31, 2022
4017c76
🚧 v3: add query param
wangjq4214 Aug 1, 2022
3d92f09
🚧 v3: change to fasthttp's header and args
wangjq4214 Aug 2, 2022
3dc9604
✨ v3: add body and ua setting
wangjq4214 Aug 2, 2022
f677738
🚧 v3: add cookie support
wangjq4214 Aug 3, 2022
108ce08
🚧 v3: add path param support
wangjq4214 Aug 3, 2022
51d9780
✅ v3: fix error test case
wangjq4214 Aug 3, 2022
dc0e374
🚧 v3: add formdata and file support
wangjq4214 Aug 4, 2022
c7afe2d
🚧 v3: referer support
wangjq4214 Aug 4, 2022
96f562b
🚧 v3: reponse unmarshal
wangjq4214 Aug 4, 2022
951b6fb
✨ v3: finish API design
wangjq4214 Aug 5, 2022
f479948
🔥 v3: remove plugin mechanism
wangjq4214 Aug 5, 2022
02ddc9b
🚧 v3: add timeout
wangjq4214 Aug 6, 2022
a3d0296
🚧 v3: change path params pattern and add unit test for core
wangjq4214 Aug 7, 2022
5ebb21d
✏️ v3: error spell
wangjq4214 Aug 8, 2022
cdeb94e
✅ v3: improve test coverage
wangjq4214 Aug 9, 2022
89b42d9
✅ perf: change test func name to fit project format
wangjq4214 Aug 12, 2022
2b075be
🚧 v3: handle error
wangjq4214 Aug 12, 2022
b005de7
🚧 v3: add unit test and fix error
wangjq4214 Aug 13, 2022
84f3145
⚡️ chore: change func to improve performance
wangjq4214 Aug 18, 2022
2b785e9
✅ v3: add some unit test
wangjq4214 Aug 18, 2022
9424465
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Aug 19, 2022
c3f40b7
✅ v3: fix error test
wangjq4214 Aug 19, 2022
7706d5a
🐛 fix: add cookie to response
wangjq4214 Aug 19, 2022
03ce5f7
✅ v3: add unit test
wangjq4214 Aug 19, 2022
1abea22
✨ v3: export raw field
wangjq4214 Aug 19, 2022
5570b37
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Aug 19, 2022
6f48694
🐛 fix: fix data race
wangjq4214 Aug 21, 2022
35742a7
🔒️ chore: change package
wangjq4214 Aug 21, 2022
b133103
🐛 fix: data race
wangjq4214 Aug 21, 2022
1a57872
🐛 fix: test fail
wangjq4214 Aug 21, 2022
43b6d28
✨ feat: move core to req
wangjq4214 Aug 21, 2022
beaa309
Merge branch 'v3-client' of github.com:wangjq4214/fiber into v3-client
wangjq4214 Aug 21, 2022
9156dc5
🐛 fix: connection reuse
wangjq4214 Aug 21, 2022
9d59491
🐛 fix: data race
wangjq4214 Aug 21, 2022
3a8f609
🐛 fix: data race
wangjq4214 Aug 21, 2022
4aa0e67
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Aug 22, 2022
22c7d20
🔀 fix: change to testify
wangjq4214 Aug 22, 2022
5d05cce
✅ fix: fail test in windows
wangjq4214 Aug 22, 2022
484be57
✨ feat: response body save to file
wangjq4214 Aug 23, 2022
1198e93
✨ feat: support tls config
wangjq4214 Aug 24, 2022
c14408e
🐛 fix: add err check
wangjq4214 Aug 24, 2022
a336a2a
🎨 perf: fix some static check
wangjq4214 Aug 24, 2022
adcca7d
✨ feat: add proxy support
wangjq4214 Aug 26, 2022
d8fde62
✨ feat: add retry feature
wangjq4214 Aug 26, 2022
a231a92
🐛 fix: static check error
wangjq4214 Aug 26, 2022
88d77cd
🎨 refactor: move som code
wangjq4214 Aug 27, 2022
a77f72a
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Sep 7, 2022
b52b3dc
docs: change readme
wangjq4214 Sep 17, 2022
21881a4
✨ feat: extend axios API
wangjq4214 Sep 17, 2022
2ac0575
perf: change field to export field
wangjq4214 Sep 18, 2022
34f8f2a
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Sep 18, 2022
382cacc
✅ chore: disable startup message
wangjq4214 Sep 18, 2022
d461cc7
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Sep 25, 2022
0dc4093
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Oct 6, 2022
9a49125
🐛 fix: fix test error
wangjq4214 Oct 6, 2022
9d0560b
chore: fix error test
wangjq4214 Oct 6, 2022
fdb2468
chore: fix test case
wangjq4214 Oct 6, 2022
e01a058
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Oct 15, 2022
c5d2df9
feat: add some test to client
wangjq4214 Oct 15, 2022
6a2f0ab
chore: add test case
wangjq4214 Oct 16, 2022
d27665f
chore: add test case
wangjq4214 Oct 16, 2022
af5cd0b
✨ feat: add peek for client
wangjq4214 Oct 16, 2022
37358db
✅ chore: add test case
wangjq4214 Oct 17, 2022
dd51324
⚡️ feat: lazy generate rand string
wangjq4214 Oct 18, 2022
aac6425
🚧 perf: add config test case
wangjq4214 Nov 10, 2022
a020ea6
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Nov 10, 2022
7e9564d
🐛 fix: fix merge error
wangjq4214 Nov 10, 2022
7e5445f
:bug: fix utils error
efectn Nov 12, 2022
bb1ab43
Merge branch 'v3-client' of github.com:wangjq4214/fiber into v3-client
wangjq4214 Nov 12, 2022
4d6d79e
:sparkles: add redirection
efectn Nov 12, 2022
9fa8c94
🔥 chore: delete deps
wangjq4214 Nov 12, 2022
4a73d3b
perf: fix spell error
wangjq4214 Nov 12, 2022
d84da41
🎨 perf: spell error
wangjq4214 Nov 12, 2022
679690b
✨ feat: add logger
wangjq4214 Nov 12, 2022
bddf796
✨ feat: add cookie jar
wangjq4214 Nov 13, 2022
fc9fdb5
✨ feat: logger with level
wangjq4214 Nov 14, 2022
034ec9e
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Nov 14, 2022
80b33d9
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Dec 7, 2022
a2914d8
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Dec 13, 2022
b66fb0d
🎨 perf: change the field name
wangjq4214 Dec 24, 2022
ecf4a8f
perf: add jar test
wangjq4214 Jan 1, 2023
0dd0565
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Feb 2, 2023
270f3d8
Merge remote-tracking branch 'origin/v3-beta' into v3-client
wangjq4214 Mar 8, 2023
d6725cf
Merge branch 'v3-client' of github.com:wangjq4214/fiber into v3-client
wangjq4214 Mar 8, 2023
47fe4be
Merge remote-tracking branch 'origin/v3-beta' into v3-client
efectn Aug 6, 2023
e4a79a6
fix proxy test
efectn Aug 6, 2023
41ae1a5
improve test coverage
efectn Aug 7, 2023
6ea9410
fix proxy tests
efectn Aug 7, 2023
c34321a
add cookiejar support from pending fasthttp PR
efectn Dec 17, 2023
de4b39a
fix some lint errors.
efectn Dec 18, 2023
fa2f858
add benchmark for SetValWithStruct
efectn Dec 18, 2023
2a21c31
optimize
efectn Jan 21, 2024
1b602da
update
efectn Jan 21, 2024
6fec61e
Merge remote-tracking branch 'origin/main' into v3-client
efectn Jan 21, 2024
568152f
fix proxy middleware
efectn Jan 22, 2024
fc4f9d5
use panicf instead of errorf and fix panic on default logger
efectn Jan 22, 2024
b39455a
Merge remote-tracking branch 'origin/main' into v3-client
efectn Jan 22, 2024
7af7e4b
update
efectn Jan 22, 2024
a619175
update
efectn Jan 22, 2024
f371b38
cleanup comments
Feb 5, 2024
e62a8df
cleanup comments
Feb 5, 2024
dae32f5
Merge remote-tracking branch 'upstream/main' into v3-client
Feb 5, 2024
2fbdc89
fix golang-lint errors
Feb 5, 2024
6c413d1
Update helper_test.go
efectn Feb 9, 2024
2e7c85f
Merge branch 'main' into v3-client
Fenny Feb 9, 2024
7a6ca57
add more test cases
Feb 10, 2024
ba78f15
Merge remote-tracking branch 'upstream/main' into v3-client
Feb 16, 2024
de0ccb8
add hostclient pool
efectn Feb 18, 2024
8a8086a
make it more thread safe
Feb 20, 2024
1b42170
fixed some golangci-lint errors
Feb 20, 2024
5b31ffb
fix Test_Request_FormData test
efectn Feb 22, 2024
88d4cfe
create new test suite
efectn Feb 24, 2024
1cc045c
just create client for once
efectn Feb 24, 2024
ebc7d50
use random port instead of 3000
efectn Feb 24, 2024
6264482
remove client pooling and fix test suite
efectn Feb 24, 2024
b978fda
fix data races on logger tests
efectn Feb 24, 2024
be94a3f
fix proxy tests
efectn Feb 24, 2024
ed3c640
fix global tests
efectn Feb 25, 2024
8ae4c35
remove unused code
efectn Feb 25, 2024
eb1961a
Merge remote-tracking branch 'origin/main' into v3-client
efectn Feb 25, 2024
c5901b2
fix logger test
efectn Feb 25, 2024
96cb295
fix proxy tests
efectn Feb 25, 2024
d8ee144
fix linter
efectn Feb 25, 2024
d6d0f17
use lock instead of rlock
efectn Feb 25, 2024
5a9223a
fix cookiejar data-race
efectn Feb 25, 2024
d122be3
fix(client): race conditions
ReneWerner87 Feb 28, 2024
a140863
fix(client): race conditions
ReneWerner87 Feb 28, 2024
6b01572
apply some reviews
efectn Mar 2, 2024
907b8a7
change client property name
efectn Mar 2, 2024
cb33aae
apply review
efectn Mar 2, 2024
9bf3d34
add parallel benchmark for simple request
efectn Mar 2, 2024
f56dfd2
apply review
efectn Mar 2, 2024
96d6068
apply review
efectn Mar 2, 2024
1a6795b
Merge remote-tracking branch 'origin/main' into v3-client
efectn Mar 2, 2024
0b6a4e8
fix log tests
efectn Mar 2, 2024
557ddca
fix linter
efectn Mar 2, 2024
2627b52
fix(client): return error in SetProxyURL instead of panic
ReneWerner87 Mar 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
999 changes: 0 additions & 999 deletions client.go

This file was deleted.

461 changes: 461 additions & 0 deletions client/client.go

Large diffs are not rendered by default.

1,213 changes: 1,213 additions & 0 deletions client/client_test.go

Large diffs are not rendered by default.

232 changes: 232 additions & 0 deletions client/core.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,232 @@
package client

import (
"context"
"encoding/json"
"encoding/xml"
"fmt"
"sync"

"github.com/gofiber/fiber/v3/utils"
"github.com/valyala/fasthttp"
)

// RequestHook is a function that receives Agent and Request,
// it can change the data in Request and Agent.
//
// Called before a request is sent.
type RequestHook func(*Client, *Request) error

// ResponseHook is a function that receives Agent, Respose and Request,
// it can change the data is Respose or deal with some effects.
//
// Called after a respose has been received.
type ResponseHook func(*Client, *Response, *Request) error

// ExecuteFunc will actually execute the request via fasthttp.
type ExecuteFunc func(context.Context, *Client, *Request) (*Response, error)

// Plugin can change the execution flow of requests.
type Plugin interface {
// Return the plugin name and the name should be different.
Name() string

// Determine if the plugin should be executed based on the conditions.
Check() bool

// Modify specific request execution methods,
// such as adding timeouts, cancellations, retries and other operations.
GenerateExecute(ExecuteFunc) (ExecuteFunc, error)
}

// `Core` stores middleware and plugin definitions,
// and defines the execution process
type Core struct {
client *fasthttp.HostClient

// user defined request hooks
userRequestHooks []RequestHook

// client package defined request hooks
buildinRequestHooks []RequestHook

// user defined response hooks
userResponseHooks []ResponseHook

// client package defined respose hooks
buildinResposeHooks []ResponseHook

// store plugins
plugins []Plugin
pluginMap map[string]Plugin

jsonMarshal utils.JSONMarshal
jsonUnmarshal utils.JSONUnmarshal
xmlMarshal utils.XMLMarshal
xmlUnmarshal utils.XMLUnmarshal
}

// execute will exec each hooks and plugins.
func (c *Core) execute(ctx context.Context, agent *Client, req *Request) (*Response, error) {
var execFunc ExecuteFunc = func(ctx context.Context, a *Client, r *Request) (*Response, error) {
resp := AcquireResponse()
resp.setClient(a)
resp.setRequest(r)

// To avoid memory allocation reuse of data structures such as errch.
errCh, reqv, respv := acquireErrChan(), fasthttp.AcquireRequest(), fasthttp.AcquireResponse()
defer func() {
releaseErrChan(errCh)
fasthttp.ReleaseRequest(reqv)
fasthttp.ReleaseResponse(respv)
}()

req.rawRequest.CopyTo(reqv)
go func() {
err := c.client.Do(reqv, respv)
if err != nil {
errCh <- err
return
}
respv.CopyTo(resp.rawResponse)
errCh <- nil
}()

select {
case err := <-errCh:
if err != nil {
// When get error should release Response
ReleaseResponse(resp)
return nil, err
}
return resp, nil
case <-ctx.Done():
return nil, fmt.Errorf("timeout or cancel error")
}
}

// The built-in hooks will be executed only
// after the user-defined hooks are executed。
for _, f := range c.userRequestHooks {
err := f(agent, req)
if err != nil {
return nil, err
}
}

for _, f := range c.buildinRequestHooks {
err := f(agent, req)
if err != nil {
return nil, err
}
}

// Call the plugins to generate the real request function.
for _, p := range c.plugins {
if !p.Check() {
continue
}

var err error
execFunc, err = p.GenerateExecute(execFunc)
if err != nil {
return nil, err
}
}

// Do http request
resp, err := execFunc(ctx, agent, req)
if err != nil {
return nil, err
}

// The built-in hooks will be executed only
// before the user-defined hooks are executed.
for _, f := range c.buildinResposeHooks {
err := f(agent, resp, req)
if err != nil {
return nil, err
}
}

for _, f := range c.userResponseHooks {
err := f(agent, resp, req)
if err != nil {
return nil, err
}
}

return resp, nil
}

// reset clears core object.
// It will not clear buildin hooks.
func (c *Core) reset() {
c.userRequestHooks = c.userRequestHooks[:0]
c.userResponseHooks = c.userResponseHooks[:0]
c.plugins = c.plugins[:0]

for k := range c.pluginMap {
delete(c.pluginMap, k)
}
}

var errChanPool sync.Pool

// acquireErrChan returns an empty error chan from the pool.
//
// The returned error chan may be returned to the pool with releaseErrChan when no longer needed.
// This allows reducing GC load.
func acquireErrChan() (ch chan error) {
chv := errChanPool.Get()
if chv != nil {
ch = chv.(chan error)
return
}
ch = make(chan error, 1)
return
}

// releaseErrChan returns the object acquired via acquireErrChan to the pool.
//
// Do not access the released core object, otherwise data races may occur.
func releaseErrChan(ch chan error) {
errChanPool.Put(ch)
}

var corePool sync.Pool

// AcquireCore returns an empty core object from the pool.
//
// The returned core may be returned to the pool with ReleaseCore when no longer needed.
// This allows reducing GC load.
func AcquireCore() (c *Core) {
cv := corePool.Get()
if cv != nil {
c = cv.(*Core)
return
}
c = &Core{
client: &fasthttp.HostClient{},
userRequestHooks: []RequestHook{},
buildinRequestHooks: []RequestHook{parserRequestURL, parserRequestHeader, parserRequestBody},
userResponseHooks: []ResponseHook{},
buildinResposeHooks: []ResponseHook{parserResponseCookie},
plugins: []Plugin{},
pluginMap: map[string]Plugin{},
jsonMarshal: json.Marshal,
jsonUnmarshal: json.Unmarshal,
xmlMarshal: xml.Marshal,
xmlUnmarshal: xml.Unmarshal,
}

return
}

// ReleaseCore returns the object acquired via AcquireCore to the pool.
//
// Do not access the released core object, otherwise data races may occur.
func ReleaseCore(c *Core) {
c.reset()
corePool.Put(c)
}
Loading