Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KS-15 Add a capabilities library to the core node #11811

Merged
merged 33 commits into from
Feb 2, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
0753776
start working on capability.go
DeividasK Jan 18, 2024
1d0ea04
Combine CapabilityInfoProvider and CapabilityInfo; Add validation tests
cedric-cordenier Jan 18, 2024
bebe6cd
Validate the capability type when adding it
cedric-cordenier Jan 19, 2024
7694b3f
Replace Stringer with string for simplicity
DeividasK Jan 23, 2024
8ab7945
Remove SynchronousCapability interface in favour of using async
DeividasK Jan 23, 2024
332d180
Add comments to public functions and variables
DeividasK Jan 23, 2024
8cca9ce
Add ExecuteSync method. Provide more info on the Executable interface.
DeividasK Jan 23, 2024
43053c6
Add a Test_ExecuteSyncReturnSingleValue
DeividasK Jan 23, 2024
f906ba3
Review comments
cedric-cordenier Jan 25, 2024
3aaa43f
Add CapabilityResponse struct for Execute channel
DeividasK Jan 26, 2024
af8eb3f
Undo chainlink-common change
DeividasK Jan 26, 2024
6a9ea80
Add reasoning behind code
DeividasK Jan 26, 2024
efa3b91
Add missing tests
DeividasK Jan 26, 2024
6274c1e
Create CallbackExecutable. Trim interfaces.
DeividasK Jan 26, 2024
508dddf
Start on an example on demand capability
DeividasK Jan 26, 2024
1924ea8
Add different capability types to registry
cedric-cordenier Jan 26, 2024
a4b07ec
Complete example trigger
cedric-cordenier Jan 26, 2024
0ff16b8
Fix race
cedric-cordenier Jan 26, 2024
5bb25e0
Add TODOs for defaultExecuteTimeout/idMaxLength
cedric-cordenier Jan 31, 2024
bfde8f1
Move capability info to a var; add must handle
cedric-cordenier Jan 31, 2024
5d1fea6
Merge remote-tracking branch 'origin/develop' into KS-15-add-a-capabi…
DeividasK Feb 1, 2024
516b50e
Remove tmockCapability
cedric-cordenier Feb 1, 2024
456a2e4
Udpate broken comment
DeividasK Feb 1, 2024
7b8ac45
Add some tests
cedric-cordenier Feb 1, 2024
42732ec
RegisterUnregisterWorkflow interface
DeividasK Feb 1, 2024
7c4ae33
cmock => mock
DeividasK Feb 2, 2024
2b60ec4
Create a const for test var
DeividasK Feb 2, 2024
6eb8ad1
Use testutils.Context(t)
DeividasK Feb 2, 2024
74ef706
Do not unwrap single value from a list
DeividasK Feb 2, 2024
f79fc5b
Channel naming
DeividasK Feb 2, 2024
3709f1a
MustCapabilityInfo => MustNewCapabilityInfo
DeividasK Feb 2, 2024
a5c8b66
Tidy more
DeividasK Feb 2, 2024
555b087
examples => triggers && downstream
DeividasK Feb 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
246 changes: 246 additions & 0 deletions core/capabilities/capability.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,246 @@
package capabilities

import (
"context"
"errors"
"fmt"
"regexp"
"time"

"golang.org/x/mod/semver"

"github.com/smartcontractkit/chainlink-common/pkg/values"
)

// CapabilityType is an enum for the type of capability.
type CapabilityType int

// CapabilityType enum values.
const (
CapabilityTypeTrigger CapabilityType = iota
CapabilityTypeAction
CapabilityTypeConsensus
CapabilityTypeTarget
)

// String returns a string representation of CapabilityType
func (c CapabilityType) String() string {
switch c {
case CapabilityTypeTrigger:
return "trigger"
case CapabilityTypeAction:
return "action"
case CapabilityTypeConsensus:
return "report"
case CapabilityTypeTarget:
return "target"
}

// Panic as this should be unreachable.
panic("unknown capability type")
}

// IsValid checks if the capability type is valid.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: some comments feel excessive in this file, code is self-explanatory

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

golint

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huh? is golint forcing us to have comments everywhere?

func (c CapabilityType) IsValid() error {
switch c {
case CapabilityTypeTrigger,
CapabilityTypeAction,
CapabilityTypeConsensus,
CapabilityTypeTarget:
return nil
}

return fmt.Errorf("invalid capability type: %s", c)
}

// CapabilityResponse is a struct for the Execute response of a capability.
type CapabilityResponse struct {
Value values.Value
Err error
}

type Metadata struct {
WorkflowID string
}

type CapabilityRequest struct {
Metadata Metadata
Config *values.Map
Inputs *values.Map
}

// CallbackExecutable is an interface for executing a capability.
type CallbackExecutable interface {
// Capability must respect context.Done and cleanup any request specific resources
// when the context is cancelled. When a request has been completed the capability
// is also expected to close the callback channel.
// Request specific configuration is passed in via the inputs parameter.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated comment? I do not see an inputs parameter

// A successful response must always return a value. An error is assumed otherwise.
// The intent is to make the API explicit.
Execute(ctx context.Context, callback chan CapabilityResponse, request CapabilityRequest) error
}

// BaseCapability interface needs to be implemented by all capability types.
// Capability interfaces are intentionally duplicated to allow for an easy change
// or extension in the future.
type BaseCapability interface {
Info() CapabilityInfo
}

// TriggerCapability interface needs to be implemented by all trigger capabilities.
type TriggerCapability interface {
BaseCapability
RegisterTrigger(ctx context.Context, callback chan CapabilityResponse, request CapabilityRequest) error
UnregisterTrigger(ctx context.Context, request CapabilityRequest) error
}

// ActionCapability interface needs to be implemented by all action capabilities.
type ActionCapability interface {
BaseCapability
CallbackExecutable
}

// ConsensusCapability interface needs to be implemented by all consensus capabilities.
type ConsensusCapability interface {
BaseCapability
CallbackExecutable
}

// TargetsCapability interface needs to be implemented by all target capabilities.
type TargetCapability interface {
BaseCapability
CallbackExecutable
}

// CapabilityInfo is a struct for the info of a capability.
type CapabilityInfo struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this return what the strong type is?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmm -- what do you mean by "strong type"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nolag are you talking about CapabilityType?

ID string
CapabilityType CapabilityType
Description string
Version string
}

// Info returns the info of the capability.
func (c CapabilityInfo) Info() CapabilityInfo {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May want to return map[string]CapabilityInfo so we can add name to execute.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CapabilityInfo includes ID, isn't that enough?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a separate conversation that is resolved now - we've parked this for now and @cedric-cordenier will write an exploration doc on whether we want to pass in a name field or not.

return c
}

var idRegex = regexp.MustCompile(`[a-z0-9_\-:]`)

const (
// TODO: this length was largely picked arbitrarily.
// Consider what a realistic/desirable value should be.
// See: https://smartcontract-it.atlassian.net/jira/software/c/projects/KS/boards/182
idMaxLength = 128
)

// NewCapabilityInfo returns a new CapabilityInfo.
func NewCapabilityInfo(
id string,
capabilityType CapabilityType,
description string,
version string,
) (CapabilityInfo, error) {
if len(id) > idMaxLength {
return CapabilityInfo{}, fmt.Errorf("invalid id: %s exceeds max length %d", id, idMaxLength)
}
if !idRegex.MatchString(id) {
cedric-cordenier marked this conversation as resolved.
Show resolved Hide resolved
return CapabilityInfo{}, fmt.Errorf("invalid id: %s. Allowed: %s", id, idRegex)
}

if ok := semver.IsValid(version); !ok {
return CapabilityInfo{}, fmt.Errorf("invalid version: %+v", version)
}

if err := capabilityType.IsValid(); err != nil {
return CapabilityInfo{}, err
}

return CapabilityInfo{
ID: id,
CapabilityType: capabilityType,
Description: description,
Version: version,
}, nil
}

// MustCapabilityInfo returns a new CapabilityInfo,
// panicking if we could not instantiate a CapabilityInfo.
func MustCapabilityInfo(
DeividasK marked this conversation as resolved.
Show resolved Hide resolved
id string,
capabilityType CapabilityType,
description string,
version string,
) CapabilityInfo {
c, err := NewCapabilityInfo(id, capabilityType, description, version)
if err != nil {
panic(err)
}

return c
}

// TODO: this timeout was largely picked arbitrarily.
// Consider what a realistic/desirable value should be.
// See: https://smartcontract-it.atlassian.net/jira/software/c/projects/KS/boards/182
var defaultExecuteTimeout = 10 * time.Second
bolekk marked this conversation as resolved.
Show resolved Hide resolved

// ExecuteSync executes a capability synchronously.
// We are not handling a case where a capability panics and crashes.
// There is default timeout of 10 seconds. If a capability takes longer than
// that then it should be executed asynchronously.
func ExecuteSync(ctx context.Context, c CallbackExecutable, request CapabilityRequest) (values.Value, error) {
ctxWithT, cancel := context.WithTimeout(ctx, defaultExecuteTimeout)
defer cancel()

callback := make(chan CapabilityResponse)
DeividasK marked this conversation as resolved.
Show resolved Hide resolved
sec := make(chan error)

go func(innerCtx context.Context, innerC CallbackExecutable, innerReq CapabilityRequest, innerCallback chan CapabilityResponse, errCh chan error) {
setupErr := innerC.Execute(innerCtx, innerCallback, innerReq)
sec <- setupErr
}(ctxWithT, c, request, callback, sec)

vs := make([]values.Value, 0)
outerLoop:
for {
select {
case response, isOpen := <-callback:
if !isOpen {
break outerLoop
}
// An error means execution has been interrupted.
// We'll return the value discarding values received
// until now.
if response.Err != nil {
return nil, response.Err
}

vs = append(vs, response.Value)

// Timeout when a capability panics, crashes, and does not close the channel.
case <-ctxWithT.Done():
return nil, fmt.Errorf("context timed out. If you did not set a timeout, be aware that the default ExecuteSync timeout is %f seconds", defaultExecuteTimeout.Seconds())
}
}

setupErr := <-sec
// Something went wrong when setting up a capability.
if setupErr != nil {
return nil, setupErr
}

// If the capability did not return any values, we deem it as an error.
// The intent is for the API to be explicit.
if len(vs) == 0 {
return nil, errors.New("capability did not return any values")
}

// If the capability returned only one value,
// let's unwrap it to improve usability.
DeividasK marked this conversation as resolved.
Show resolved Hide resolved
if len(vs) == 1 {
return vs[0], nil
}

return &values.List{Underlying: vs}, nil
}
Loading
Loading