feat: Enforce custom auth userId value

[johnjcsmith]

Require that a cluster's customAuthHandler returns an object with key userId (Other keys are allowed with z.passthrough()).

This value is used as the Run's userId (prefixed with custom:) and validated for run ownership.

This allows for tokens to be refreshed while still providing access to the existing runs by allowing the customAuthHandler to provide the identity.

---

Description

• Enforced userId as the key for custom authentication.

• Updated extractCustomAuthState to use userId consistently.

• Added schema validation for custom authentication results.

• Removed unused getRunCustomAuthToken function.

---

Changes walkthrough 📝

	Relevant files
Tests	auth.test.ts Update tests for custom authentication changes                      control-plane/src/modules/auth/auth.test.ts Updated tests to use userId instead of someAuthValue. Adjusted test cases to align with new entityId logic. +17/-16
Enhancement	auth.ts Refactor custom authentication logic                                          control-plane/src/modules/auth/auth.ts Replaced entityId value with custom:${context.userId}. Updated logic to validate userId for run access. Removed dependency on getRunCustomAuthToken. +10/-10  custom.ts Add schema validation for custom auth results                        control-plane/src/modules/auth/custom.ts Added zod schema for validating custom auth results. Integrated schema validation into the verify function. +17/-2
Cleanup	workflows.ts Remove unused custom auth token function                                  control-plane/src/modules/workflows/workflows.ts Removed unused getRunCustomAuthToken function. +0/-22

---

💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

[qodo-merge-pro]

Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Validation Order The run existence check should happen before checking the clusterId to avoid unnecessary cluster ID validation on non-existent runs if (opts.run && opts.run.clusterId !== clusterId) { throw new AuthenticationError("Custom auth does not have access to this run"); } if (opts.run) {

[qodo-merge-pro]

Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Score
Possible issue	Add validation for potentially undefined external data to prevent runtime errors Add error handling for when context.userId is undefined. The current code assumes it will always be present, but since it comes from an external source, it should be validated. control-plane/src/modules/auth/auth.ts [364] -entityId: `custom:${context.userId}`, +entityId: context.userId ? `custom:${context.userId}` : undefined, Apply this suggestion Suggestion importance[1-10]: 8 Why: This is a critical validation that prevents potential runtime errors when accessing undefined properties, which could lead to system crashes. The suggestion properly handles the case where userId might be undefined.	8
Security	Strengthen input validation to prevent empty identifiers from being accepted Add minimum length validation for userId in the schema to prevent empty strings from being accepted as valid identifiers. control-plane/src/modules/auth/custom.ts [18-20] const customAuthResultSchema = z.object({ - userId: z.string(), + userId: z.string().min(1), }).passthrough(); Apply this suggestion Suggestion importance[1-10]: 7 Why: Adding minimum length validation for userId is important for security and data integrity, preventing potential issues with empty identifiers that could affect authentication and authorization.	7

[qodo-merge-pro]

CI Failure Feedback 🧐

(Checks updated until commit 6c060e3)

Action: test-control-plane

Failed stage: Run tests [❌]

Failed test name: src/modules/auth/auth.test.ts

Failure summary: The action failed due to a test failure in src/modules/auth/auth.test.ts. Specifically: The test expected owner1AuthState to match a certain object structure but the assertion failed at line 339 The test was part of a larger test suite where 5 tests failed out of 172 total tests The failure appears to be related to authentication state validation in the auth module

Relevant error logs: 1: ##[group]Operating System 2: Ubuntu ... 454: at Object.toMatchObject (src/modules/auth/auth.test.ts:339:33) 455: PASS src/modules/workflows/agent/nodes/tool-call.test.ts 456: ● Console 457: console.log 458: {"deployment.version":"unknown","level":"info","message":"Executing tool call","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 459: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 460: at Array.map (<anonymous>) 461: console.log 462: {"deployment.version":"unknown","error":{"message":"Received empty response from tool executor","name":"AgentError","stack":"AgentError: Received empty response from tool executor\n at _handleToolCall (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:174:13)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at async Promise.all (index 0)\n at _handleToolCalls (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:56:23)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.test.ts:75:25)"},"level":"error","message":"Failed to invoke tool","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo","toolName":"console_echo"} 463: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 464: at async Promise.all (index 0) 465: console.log 466: {"cluster.id":"test","deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at _handleToolCall (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:333:5)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at async Promise.all (index 0)\n at _handleToolCalls (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:56:23)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.test.ts:75:25)"},"event":{"clusterId":"test","completedAt":1735949239924,"input":{"input":"hello"},"level":"ERROR","output":{"name":"AgentError"},"runId":"test","startedAt":1735949239904,"toolName":"console_echo","type":"toolCall"},"level":"error","message":"Error sending model call event to SQS","queueUrl":"PLACEHOLDER","run.id":"test","sqs.queue.url":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 467: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 468: at async Promise.all (index 0) 469: console.log 470: {"deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at _handleToolCall (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:333:5)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at async Promise.all (index 0)\n at _handleToolCalls (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:56:23)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.test.ts:75:25)"},"event":{"clusterId":"test","completedAt":1735949239924,"input":{"input":"hello"},"level":"ERROR","output":{"name":"AgentError"},"runId":"test","startedAt":1735949239904,"toolName":"console_echo","type":"toolCall"},"level":"warn","message":"Error sending customer telemetry event to SQS","queueUrl":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} ... 474: {"deployment.version":"unknown","level":"info","message":"Executing tool call","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 475: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 476: at Array.map (<anonymous>) 477: console.log 478: {"deployment.version":"unknown","level":"info","message":"Executing tool call","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 479: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 480: at Array.map (<anonymous>) 481: console.log 482: {"cluster.id":"test","deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at _handleToolCall (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:129:5)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at async Promise.all (index 0)\n at _handleToolCalls (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:56:23)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.test.ts:108:25)"},"event":{"clusterId":"test","completedAt":1735949239945,"input":{"input":"hello"},"level":"ERROR","output":{"name":"NotFoundError","statusCode":404},"runId":"test","startedAt":1735949239944,"toolName":"console_echo","type":"toolCall"},"level":"error","message":"Error sending model call event to SQS","queueUrl":"PLACEHOLDER","run.id":"test","sqs.queue.url":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 483: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 484: at async Promise.all (index 0) 485: console.log 486: {"deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at _handleToolCall (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:129:5)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at async Promise.all (index 0)\n at _handleToolCalls (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:56:23)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.test.ts:108:25)"},"event":{"clusterId":"test","completedAt":1735949239945,"input":{"input":"hello"},"level":"ERROR","output":{"name":"NotFoundError","statusCode":404},"runId":"test","startedAt":1735949239944,"toolName":"console_echo","type":"toolCall"},"level":"warn","message":"Error sending customer telemetry event to SQS","queueUrl":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 487: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 488: at async Promise.all (index 0) 489: console.log 490: {"deployment.version":"unknown","level":"info","message":"Executing tool call","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 491: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 492: at Array.map (<anonymous>) 493: console.log 494: {"deployment.version":"unknown","error":{"message":"instance requires property \"input\"\ninstance is not allowed to have the additional property \"wrongKey\"","name":"Error","stack":"Error: instance requires property \"input\"\ninstance is not allowed to have the additional property \"wrongKey\"\n at AgentTool.execute (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/tool.ts:54:13)\n at execute (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:172:36)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)"},"level":"info","message":"Agent provided invalid tool input","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo","toolName":"console_echo"} 495: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 496: console.log 497: {"cluster.id":"test","deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)"},"event":{"clusterId":"test","completedAt":1735949239957,"input":{"wrongKey":"something"},"level":"ERROR","runId":"test","startedAt":1735949239956,"toolName":"console_echo","type":"toolCall"},"level":"error","message":"Error sending model call event to SQS","queueUrl":"PLACEHOLDER","run.id":"test","sqs.queue.url":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 498: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 499: console.log 500: {"deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)"},"event":{"clusterId":"test","completedAt":1735949239957,"input":{"wrongKey":"something"},"level":"ERROR","runId":"test","startedAt":1735949239956,"toolName":"console_echo","type":"toolCall"},"level":"warn","message":"Error sending customer telemetry event to SQS","queueUrl":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} ... 502: console.log 503: {"deployment.version":"unknown","level":"info","message":"Found invocation-result message, finding last non-invocation message","timestamp":"2025-01-04 00:07:19","toolCallId":"456"} 504: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 505: console.log 506: {"deployment.version":"unknown","level":"info","message":"Executing tool call","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 507: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 508: at Array.map (<anonymous>) 509: console.log 510: {"deployment.version":"unknown","error":{"message":"Received empty response from tool executor","name":"AgentError","stack":"AgentError: Received empty response from tool executor\n at _handleToolCall (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:174:13)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at async Promise.all (index 0)\n at _handleToolCalls (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:56:23)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.test.ts:232:27)"},"level":"error","message":"Failed to invoke tool","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo","toolName":"console_echo"} 511: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 512: at async Promise.all (index 0) 513: console.log 514: {"cluster.id":"test","deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at _handleToolCall (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:333:5)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at async Promise.all (index 0)\n at _handleToolCalls (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:56:23)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.test.ts:232:27)"},"event":{"clusterId":"test","completedAt":1735949239968,"input":{"input":"hello"},"level":"ERROR","output":{"name":"AgentError"},"runId":"test","startedAt":1735949239967,"toolName":"console_echo","type":"toolCall"},"level":"error","message":"Error sending model call event to SQS","queueUrl":"PLACEHOLDER","run.id":"test","sqs.queue.url":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 515: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 516: at async Promise.all (index 0) 517: console.log 518: {"deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at _handleToolCall (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:333:5)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at async Promise.all (index 0)\n at _handleToolCalls (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.ts:56:23)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/workflows/agent/nodes/tool-call.test.ts:232:27)"},"event":{"clusterId":"test","completedAt":1735949239968,"input":{"input":"hello"},"level":"ERROR","output":{"name":"AgentError"},"runId":"test","startedAt":1735949239967,"toolName":"console_echo","type":"toolCall"},"level":"warn","message":"Error sending customer telemetry event to SQS","queueUrl":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} ... 522: {"deployment.version":"unknown","level":"info","message":"Executing tool call","timestamp":"2025-01-04 00:07:19","tool.call.id":"123","tool.name":"console_echo"} 523: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 524: at Array.map (<anonymous>) 525: console.log 526: {"deployment.version":"unknown","level":"info","message":"Executing tool call","timestamp":"2025-01-04 00:07:19","tool.call.id":"456","tool.name":"console_echo"} 527: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 528: at Array.map (<anonymous>) 529: console.log 530: {"cluster.id":"test","deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)"},"event":{"clusterId":"test","completedAt":1735949239975,"input":{"input":"world"},"level":"DEFAULT","output":{"result":{},"resultType":"resolution","status":"success"},"runId":"test","startedAt":1735949239975,"toolName":"console_echo","type":"toolCall"},"level":"error","message":"Error sending model call event to SQS","queueUrl":"PLACEHOLDER","run.id":"test","sqs.queue.url":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"456","tool.name":"console_echo"} 531: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 532: console.log 533: {"deployment.version":"unknown","error":{"message":"Region is missing","name":"Error","stack":"Error: Region is missing\n at default (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:117:11)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/node-config-provider/dist-cjs/index.js:90:104\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:97:33\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at coalesceProvider (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:124:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/property-provider/dist-cjs/index.js:135:20\n at region (/home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/config-resolver/dist-cjs/index.js:142:30)\n at Object.defaultSQSHttpAuthSchemeParametersProvider [as httpAuthSchemeParametersProvider] (/home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/client-sqs/dist-cjs/auth/httpAuthSchemeProvider.js:9:18)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@smithy/core/dist-cjs/index.js:65:5\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-sdk-sqs/dist-cjs/index.js:134:16\n at /home/runner/work/inferable/inferable/control-plane/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22\n at attributes.cluster.id (/home/runner/work/inferable/inferable/control-plane/src/modules/track-customer-telemetry.ts:44:26)\n at handler (/home/runner/work/inferable/inferable/control-plane/src/modules/observability/tracer.ts:49:16)"},"event":{"clusterId":"test","completedAt":1735949239975,"input":{"input":"world"},"level":"DEFAULT","output":{"result":{},"resultType":"resolution","status":"success"},"runId":"test","startedAt":1735949239975,"toolName":"console_echo","type":"toolCall"},"level":"warn","message":"Error sending customer telemetry event to SQS","queueUrl":"PLACEHOLDER","timestamp":"2025-01-04 00:07:19","tool.call.id":"456","tool.name":"console_echo"} ... 565: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 566: console.log 567: {"deployment.version":"unknown","level":"info","message":"Model returned done with no result","timestamp":"2025-01-04 00:07:20"} 568: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 569: console.log 570: {"deployment.version":"unknown","level":"warn","message":"Model context window not set, defaulting to 100_000","timestamp":"2025-01-04 00:07:20"} 571: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 572: console.log 573: {"deployment.version":"unknown","errors":[{"argument":"randomStuff","instance":{"randomStuff":"123"},"message":"is not allowed to have the additional property \"randomStuff\"","name":"additionalProperties","path":[],"property":"instance","schema":{"additionalProperties":false,"properties":{"done":{"description":"Whether the workflow is done. All tasks have been completed or you can not progress further.","type":"boolean"},"invocations":{"description":"Any tool calls you need to make. If multiple are provided, they will be executed in parallel. DO NOT describe previous tool calls.","items":{"additionalProperties":false,"properties":{"input":{"additionalProperties":true,"description":"Arbitrary input parameters for the tool call.","type":"object"},"reasoning":{"description":"Reasoning trace for why this tool call is made.","type":"string"},"toolName":{"enum":["testTool","notify"],"type":"string"}},"required":["toolName","input"],"type":"object"},"type":"array"},"issue":{"description":"Describe any issues you have encountered in this step. Specifically related to the tools you are using. If none, keep this field empty.","type":"string"},"message":{"description":"A message describing the current state or next steps.","type":"string"}},"type":"object"},"stack":"instance is not allowed to have the additional property \"randomStuff\""}],"level":"info","message":"Model provided invalid response object","timestamp":"2025-01-04 00:07:20"} ... 585: {"additional":["notify"],"deployment.version":"unknown","level":"info","message":"Structured output attempted to call additional tools","timestamp":"2025-01-04 00:07:21"} 586: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 587: PASS src/modules/workflows/agent/overflow.test.ts 588: ● Console 589: console.log 590: {"deployment.version":"unknown","inputTokenCount":900,"level":"info","message":"Run history exceeds context window, early messages have been truncated","outputTokenCount":300,"removedMessageIds":[null,null,null],"systemPromptTokenCount":200,"timestamp":"2025-01-04 00:07:21"} 591: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 592: console.log 593: {"deployment.version":"unknown","level":"error","message":"A single message exceeds context window","timestamp":"2025-01-04 00:07:21"} ... 599: {"deployment.version":"unknown","inputTokenCount":1000,"level":"info","message":"Run history exceeds context window, early messages have been truncated","outputTokenCount":600,"removedMessageIds":["456","123"],"systemPromptTokenCount":200,"timestamp":"2025-01-04 00:07:21"} 600: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 601: console.log 602: {"deployment.version":"unknown","inputTokenCount":1000,"level":"info","message":"Run history exceeds context window, early messages have been truncated","outputTokenCount":600,"removedMessageIds":["789","123"],"systemPromptTokenCount":200,"timestamp":"2025-01-04 00:07:21"} 603: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 604: PASS src/modules/service-definitions.test.ts 605: ● Console 606: console.log 607: {"args":[{}],"deployment.version":"unknown","level":"error","message":"Invalid job arguments","timestamp":"2025-01-04 00:07:22"} 608: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 609: console.log 610: {"args":1,"deployment.version":"unknown","level":"error","message":"Invalid job arguments","timestamp":"2025-01-04 00:07:22"} ... 721: at async Promise.all (index 4) 722: console.log 723: {"deployment.version":"unknown","level":"warn","message":"Skipping run resume. NODE_ENV is set to 'test'.","timestamp":"2025-01-04 00:07:31"} 724: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 725: at async Promise.all (index 5) 726: PASS src/modules/models/index.test.ts 727: ● Console 728: console.log 729: {"attempt":1,"deployment.version":"unknown","error":{"message":"","name":"RetryableError","stack":"RetryableError: \n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/models/index.test.ts:33:15)\n at /home/runner/work/inferable/inferable/control-plane/node_modules/jest-mock/build/index.js:397:39\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-mock/build/index.js:404:13)\n at Object.mockConstructor [as create] (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-mock/build/index.js:148:19)\n at create (/home/runner/work/inferable/inferable/control-plane/src/modules/models/index.ts:136:67)\n at RetryOperation.runAttempt [as _fn] (/home/runner/work/inferable/inferable/control-plane/node_modules/async-retry/lib/index.js:42:15)\n at RetryOperation.Object.<anonymous>.RetryOperation.attempt (/home/runner/work/inferable/inferable/control-plane/node_modules/retry/lib/retry_operation.js:116:8)\n at run (/home/runner/work/inferable/inferable/control-plane/node_modules/async-retry/lib/index.js:55:8)\n at new Promise (<anonymous>)\n at retry (/home/runner/work/inferable/inferable/control-plane/node_modules/async-retry/lib/index.js:58:10)\n at Object.call (/home/runner/work/inferable/inferable/control-plane/src/modules/models/index.ts:97:40)\n at Object.call (/home/runner/work/inferable/inferable/control-plane/src/modules/models/index.test.ts:40:19)\n at Promise.then.completed (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/utils.js:298:28)\n at new Promise (<anonymous>)\n at callAsyncCircusFn (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/utils.js:231:10)\n at _callCircusTest (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:316:40)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at _runTest (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:252:3)\n at _runTestsForDescribeBlock (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:126:9)\n at _runTestsForDescribeBlock (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:121:9)\n at _runTestsForDescribeBlock (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:121:9)\n at run (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:71:3)\n at runAndTransformResultsToJestFormat (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)\n at jestAdapter (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:79:19)\n at runTestInternal (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-runner/build/runTest.js:367:16)\n at runTest (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-runner/build/runTest.js:444:34)\n at Object.worker (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-runner/build/testWorker.js:106:12)"},"level":"warn","message":"Model call failed with retryable error","timestamp":"2025-01-04 00:07:32"} 730: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 731: console.log 732: {"deployment.version":"unknown","level":"warn","message":"No cluster id provided, usage tracking will be skipped","timestamp":"2025-01-04 00:07:34"} 733: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 734: console.log 735: {"attempt":1,"deployment.version":"unknown","error":{"message":"","name":"Error","stack":"Error: \n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/models/index.test.ts:59:21)\n at Promise.then.completed (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/utils.js:298:28)\n at new Promise (<anonymous>)\n at callAsyncCircusFn (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/utils.js:231:10)\n at _callCircusTest (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:316:40)\n at _runTest (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:252:3)\n at _runTestsForDescribeBlock (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:126:9)\n at _runTestsForDescribeBlock (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:121:9)\n at _runTestsForDescribeBlock (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:121:9)\n at run (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:71:3)\n at runAndTransformResultsToJestFormat (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)\n at jestAdapter (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:79:19)\n at runTestInternal (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-runner/build/runTest.js:367:16)\n at runTest (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-runner/build/runTest.js:444:34)\n at Object.worker (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-runner/build/testWorker.js:106:12)"},"level":"error","message":"Model call failed with non-retryable error","timestamp":"2025-01-04 00:07:34"} 736: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 737: PASS src/modules/integrations/valtown.test.ts 738: PASS src/modules/clusters.test.ts 739: PASS src/modules/auth/clerk.test.ts 740: ● Console 741: console.log 742: {"deployment.version":"unknown","error":{"message":"jwt malformed","name":"JsonWebTokenError","stack":"JsonWebTokenError: jwt malformed\n at Object.<anonymous>.module.exports [as verify] (/home/runner/work/inferable/inferable/control-plane/node_modules/jsonwebtoken/verify.js:70:17)\n at verify (/home/runner/work/inferable/inferable/control-plane/src/modules/auth/clerk.ts:41:9)\n at new Promise (<anonymous>)\n at Object.verify (/home/runner/work/inferable/inferable/control-plane/src/modules/auth/clerk.ts:40:10)\n at Object.verify (/home/runner/work/inferable/inferable/control-plane/src/modules/auth/clerk.test.ts:32:28)\n at Promise.then.completed (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/utils.js:298:28)\n at new Promise (<anonymous>)\n at callAsyncCircusFn (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/utils.js:231:10)\n at _callCircusTest (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:316:40)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at _runTest (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:252:3)\n at _runTestsForDescribeBlock (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:126:9)\n at _runTestsForDescribeBlock (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:121:9)\n at run (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/run.js:71:3)\n at runAndTransformResultsToJestFormat (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapterInit.js:122:21)\n at jestAdapter (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:79:19)\n at runTestInternal (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-runner/build/runTest.js:367:16)\n at runTest (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-runner/build/runTest.js:444:34)\n at Object.worker (/home/runner/work/inferable/inferable/control-plane/node_modules/jest-runner/build/testWorker.js:106:12)"},"level":"info","message":"Error verifying JWT token","timestamp":"2025-01-04 00:07:36"} ... 759: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 760: console.log 761: {"deployment.version":"unknown","level":"warn","message":"Skipping run resume. NODE_ENV is set to 'test'.","timestamp":"2025-01-04 00:07:42"} 762: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 763: console.log 764: {"deployment.version":"unknown","jobId":"01JGQA7766ZP0K4HMWHESECZE5","level":"warn","message":"Job result was not persisted","timestamp":"2025-01-04 00:07:42"} 765: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 766: PASS src/modules/workflows/metadata.test.ts 767: PASS src/utilities/errors.test.ts ... 769: PASS src/modules/router.test.ts 770: PASS src/modules/auth/custom.test.ts (21.531 s) 771: ● Console 772: console.log 773: {"deployment.version":"unknown","level":"warn","message":"Skipping run resume. NODE_ENV is set to 'test'.","timestamp":"2025-01-04 00:07:26"} 774: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 775: at async Promise.allSettled (index 0) 776: console.log 777: {"deployment.version":"unknown","error":{"message":"The client is closed","name":"Error","stack":"Error: The client is closed\n at Commander._RedisClient_sendCommand (/home/runner/work/inferable/inferable/control-plane/node_modules/@redis/client/dist/lib/client/index.js:520:31)\n at Commander.commandsExecutor (/home/runner/work/inferable/inferable/control-plane/node_modules/@redis/client/dist/lib/client/index.js:190:154)\n at Commander.BaseClass.<computed> [as set] (/home/runner/work/inferable/inferable/control-plane/node_modules/@redis/client/dist/lib/commander.js:8:29)\n at Object.set (/home/runner/work/inferable/inferable/control-plane/src/utilities/cache.ts:38:11)\n at set (/home/runner/work/inferable/inferable/control-plane/src/modules/auth/custom.ts:120:34)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at Object.<anonymous> (/home/runner/work/inferable/inferable/control-plane/src/modules/auth/custom.test.ts:87:20)"},"key":"439e65049498ca8c8956b64a74b36a4b7a1147bef099057f11d6fffbda1dff3a","level":"error","message":"Error setting cache","timestamp":"2025-01-04 00:07:26","value":{"result":"{\"value\":{\"userId\":\"test-user\",\"roles\":[\"admin\"]}}","resultType":"resolution","service":"authService","status":"success"}} 778: at Console.log (node_modules/winston/lib/winston/transports/console.js:87:23) 779: A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --detectOpenHandles to find leaks. Active timers can also cause this, ensure that .unref() was called on them. 780: Summary of all failing tests ... 879: �[0m �[90m 337 |�[39m 880: �[90m 338 |�[39m owner1AuthState �[33m=�[39m �[36mawait�[39m extractAuthState(�[32m""�[39m)�[33m;�[39m 881: �[31m�[1m>�[22m�[39m�[90m 339 |�[39m expect(owner1AuthState)�[33m.�[39mtoMatchObject({ 882: �[90m |�[39m �[31m�[1m^�[22m�[39m 883: �[90m 340 |�[39m entityId�[33m:�[39m owner1�[33m.�[39muserId�[33m,�[39m 884: �[90m 341 |�[39m organizationId�[33m:�[39m owner1�[33m.�[39morganizationId�[33m,�[39m 885: �[90m 342 |�[39m organizationRole�[33m:�[39m �[32m"org:member"�[39m�[33m,�[39m�[0m 886: at Object.toMatchObject (src/modules/auth/auth.test.ts:339:33) 887: Test Suites: 1 failed, 31 passed, 32 total 888: Tests: 5 failed, 1 skipped, 166 passed, 172 total 889: Snapshots: 0 total 890: Time: 35.989 s 891: Ran all test suites matching /src/i. 892: Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished? 893: ##[error]Process completed with exit code 1. ... 900: [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 901: [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 902: [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 903: http.https://github.com/.extraheader 904: [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 905: [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 906: Print service container logs: a815d8f1010646e3a609bf59c82fd6ea_redis_479337 907: ##[command]/usr/bin/docker logs --details c537eb96713dc87b0e648f27c0e0d841699d6b1edeb82b87f5b01f6797a4165a 908: 1:C 04 Jan 2025 00:06:16.676 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

---

✨ CI feedback usage guide:

---

The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
The tool analyzes the failed checks and provides several feedbacks:

• Failed stage
• Failed test name
• Failure summary
• Relevant error logs

In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

/checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"

where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed check.

Configuration options

• enable_auto_checks_feedback - if set to true, the tool will automatically provide feedback when a check is failed. Default is true.
• excluded_checks_list - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list.
• enable_help_text - if set to true, the tool will provide a help message with the feedback. Default is true.
• persistent_comment - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true.
• final_update_message - if persistent_comment is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true.

See more information about the checks tool in the docs.