diff --git a/text/0300-programmatic-toolkit.md b/text/0300-programmatic-toolkit.md index 957285a83..a7db4dd4b 100644 --- a/text/0300-programmatic-toolkit.md +++ b/text/0300-programmatic-toolkit.md @@ -173,7 +173,8 @@ The most flexible way to obtain a Cloud Executable is from a function returning Providing a function allows your implementation to integrate with the [Programmatic Toolkit]'s automatic context fetching system. The function will be called one or more times with the known context values of the current iteration. It must then use these values to synthesize and return a `CloudAssembly`. -If the Cloud Assembly contains metadata that indicates that context information is missing, the toolkit will perform lookup AWS API calls using the appropriate credentials, and invoke the cloud executable again. +If the Cloud Assembly contains metadata that indicates that context information is missing, +the toolkit will perform lookup AWS API calls using the appropriate credentials, and invoke the cloud executable again. For all features (e.g. lookups) to work correctly, the `App()` must be instantiated with the received context values. Since it is not possible to update the context of a `cdk.App`, it must be created inside the function scope. @@ -238,7 +239,8 @@ interface ToolkitError extends IoMessage { The toolkit might throw other exceptions. These are bugs and you should report them by [raising an issue](https://github.com/aws/aws-cdk/issues/new?assignees=&labels=bug%2Cneeds-triage&projects=&template=bug-report.yml&title=%28toolkit%29%3A+Untyped+Exception+%28short+description%29). To assist integrators with detecting the type of error, the following helper methods are available. -Even though errors are typed, you should not rely on `instanceof` checks because it can behave unexpectedly when working with multiple copies of the same package. [TODO: insert reference link] +Even though errors are typed, you should not rely on `instanceof` checks +because it can behave unexpectedly when working with multiple copies of the same package. [TODO: insert reference link] ```ts try { @@ -258,14 +260,14 @@ try { ##### Synth-time Errors -There is an important semantic difference between errors that originate from the [Programmatic Toolkit] and from a Cloud Executable, i.e. from a user's app: +There is an important semantic difference between errors that originate from the [Programmatic Toolkit] and from a Cloud Executable, +i.e. from a user's app: Errors from the [Programmatic Toolkit] are typically misconfigurations and maybe fixable by an integrator, while errors from a Cloud Executable are usually code problems and only fixable by the user. The [Programmatic Toolkit] treats and throws synth-time errors like any errors. When implementing a custom Cloud Executable in code, an integrator may choose to handle synth-time errors in the toolkit code or in the Cloud Executable. - ```ts const cx: ICloudExecutable = CloudExecutable.fromCloudAssemblyProducer( async (context: Record): cxapi.CloudAssembly => { @@ -288,10 +290,12 @@ try { #### Dispose -The `Toolkit` class implements an `AsyncDisposable` Resource according to the [Explicit Resource Management](https://github.com/tc39/proposal-explicit-resource-management) feature in ECMAScript. +The `Toolkit` class implements an `AsyncDisposable` Resource according to the [Explicit Resource Management][] feature in ECMAScript. This means that any resources used by the [Programmatic Toolkit] are automatically cleaned-up. `AsyncDisposable` guarantees this clean-up even in case of an uncaught exception. +[Explicit Resource Management]: https://github.com/tc39/proposal-explicit-resource-management + You may also call the `dispose()` method directly. We recommend to do this inside a `finally` statement to guarantee execution. @@ -333,14 +337,16 @@ ACME develops a custom cli tool to abstract the hard parts of creating REST APIs ACME's data scientists are able to place a `app-config.yml` file in a repository. They then write a FastAPI app compatible with Lambda to serve their models. Then they run `acme-cli deploy serverless-rest-api --config app-config.yml`. -This is great, because the data scientists don't have to know anything about CloudFormation, the AWS CDK, or other IaC tools, but they can easily deploy apps. +This is great, because the data scientists don't have to know anything about CloudFormation, +the AWS CDK, or other IaC tools, but they can easily deploy apps. With [Programmatic Toolkit] ACME can automatically handle bootstrapping for its users and it allows ACME to provided targeted output to theirs Real world examples: * [Amplify](https://github.com/aws-amplify/amplify-cli) uses CDK apps to enable their users to build cloud-enabled frontend and mobile apps. Amplify is wrapping the CDK in a custom CLI and adds additional features like sandbox environments. -* [The Guardian](https://github.com/guardian/cdk) is providing their engineers with a construct library and cli tool to create new projects and migrate from existing CloudFormation YAML based stacks. +* [The Guardian](https://github.com/guardian/cdk) is providing their engineers with a construct library and cli tool to create new projects + and migrate from existing CloudFormation YAML based stacks. * [Pulumi CDK Adapter](https://github.com/pulumi/pulumi-cdk) is enabling Pulumi users to use CDK constructs in their applications. The tool uses `cli-lib-alpha` to synthesize user's apps and integrate with other Pulumi tooling. @@ -349,16 +355,19 @@ Real world examples: Build an app, e.g. a serverless REST API and then run functional tests against it. That could mean sending several requests to a REST API and watching for expected responses. On completion the stack is torn down and a report provided to the user. -This can also be used to automatically create test environments from a Pull Request and keep them up-to-date until the change is merged, at which point the environment is destroyed again. +This can also be used to automatically create test environments from a Pull Request and keep them up-to-date until the change is merged, +at which point the environment is destroyed again. The [Programmatic Toolkit] enables execution of integration test cases as soon as the relevant resources are deployed (as opposed to the whole stack). It can be run from within a Lambda function and handle complex error scenarios. Real world examples: -* [integ-runner](https://github.com/aws/aws-cdk/tree/v2.165.0/packages/%40aws-cdk/integ-runner) is orchestrating lifecycle actions on behalf of the users to provide end-to-end integration testing. +* [integ-runner](https://github.com/aws/aws-cdk/tree/v2.165.0/packages/%40aws-cdk/integ-runner) + is orchestrating lifecycle actions on behalf of the users to provide end-to-end integration testing. It uses a wrapper library to call the AWS CDK CLI via subshell. -* [@aws-cdk-testing/cli-integ](https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk-testing/cli-integ/lib/with-cdk-app.ts) executes complex test scenarios for features like authentication, bootstrapping, garbage collection as well as thrid-party integrations. +* [@aws-cdk-testing/cli-integ](https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk-testing/cli-integ/lib/with-cdk-app.ts) + executes complex test scenarios for features like authentication, bootstrapping, garbage collection as well as thrid-party integrations. It runs tests in specially prepared environments. To script the setup and the test execution, it uses a custom subshell wrapper to execute CLI commands. @@ -370,7 +379,8 @@ The EMR cluster and tasks can be defined and deployed from within the Notebook. API calls are then used to execute the tasks, download results and remove the infrastructure. Today users are limited to make imperative SDK calls or to provide a separate companion app with additional instructions. -With the [Programmatic Toolkit], CDK apps can be deployed directly from within a Notebook and resources can be referenced directly as the result of CDK actions. +With the [Programmatic Toolkit], CDK apps can be deployed directly from within a Notebook +and resources can be referenced directly as the result of CDK actions. Real world examples: @@ -407,13 +417,14 @@ and these might potentially be satisfied by smaller, more targeted improvements. ### What is the technical solution (design) of this feature? The [Programmatic Toolkit] is building on the existing design of the `cli-lib-alpha` package. -User apps are defined as a Cloud Executable and can be created in multiple ways like reading existing on-disk CDK apps or adding code directly by implementing an interface. +User apps are defined as a Cloud Executable and can be created in multiple ways like reading existing on-disk CDK apps +or adding code directly by implementing an interface. A new [Programmatic Toolkit] package is introduced to contain the implementation for all actions. A new [Toolkit] class provides access to the actions and offers control over configuration and certain features. To start with, only **interactive controls** (`IoHost`) will become part of the public contract. -Today Users call the CLI and Integrators either call the CLI or use cli-lib-alpha which is a wrapper around the CLI. +![Today Users call the CLI and Integrators either call the CLI or use cli-lib-alpha which is a wrapper around the CLI.](../images/0300/tomorrow.png) To keep the solution maintainable, the codebase will restructured into three main packages: @@ -434,9 +445,11 @@ This is important because programmatic use is inherently different to interactiv A good example for this is `cdk deploy` and the `--rollback` / `--no-rollback` options. Typically `--no-rollback` is used to increase iteration speed during development and a production deployment would run with `--rollback`. -However there are situations in which deploying in rollback mode is necessary, or even an explicit rollback needs to be performed before a deployment can continue. +However there are situations in which deploying in rollback mode is necessary, +or even an explicit rollback needs to be performed before a deployment can continue. For example if the stack is in a failed state or if the update contains a replacement which must always be run with rollbacks enabled. -In an interactive CLI it makes sense to prompt the user for their input if these situations are detected, even if the user originally had `--no-rollback` specified. +In an interactive CLI it makes sense to prompt the user for their input if these situations are detected, +even if the user originally had `--no-rollback` specified. Now think of a different interactive mode: An app with a UI. All the same applies as before, but the implementation of the interaction will be different. @@ -446,7 +459,7 @@ Although this is the easiest because we simply respect the value of the flag. `IoHost` formalizes how we deal with interaction. In our code base, we codify where we have decision points that potentially warrant a (different) response from the integrator and in extension the user. -Timeline of the communication between Toolkit and Integrator: First the Toolkit sends three messages. Then it sends a Request to which the integrator resonds. Finally two more messages are send from the Toolkit. +![Timeline of the communication between Toolkit and Integrator: First the Toolkit sends three messages. Then it sends a Request to which the integrator resonds. Finally two more messages are send from the Toolkit](../images/0300/iohost.png) We have an other requirement, that is in a similar space: Full control over all output. Think different interaction modes (none vs console vs app), but also rewriting the message text messages and re-prioritization w.r.t to log levels. @@ -543,7 +556,8 @@ The toolkit reads the error information from the Cloud Assembly and converts the > If we end up implementing a lot of transactional retries, we can revisit this decision and add recoverable errors as a generic reusable way. While most errors are terminal, it's possible to recover from some. -For example if an authentication token is expired, an integrator may prompt the user to re-authenticate via a browser login, before continuing with the action. +For example if an authentication token is expired, an integrator may prompt the user to re-authenticate via a browser login, +before continuing with the action. With recoverable errors, the integrator can return to normal program flow after an issue has been addressed. In the [Programmatic Toolkit], recoverable errors are implemented as a special type of request. @@ -582,7 +596,8 @@ The solution includes a paradigm shift to consider programmatic access first and This shift is codified by the introduction of a new package that contains all actions and the CLI calling them. With this, we now have more work to do: Any feature needs to first consider its programmatic usage and then derive the CLI usage from it. -We mitigate this by providing tooling that helps contributors to define different inputs and defaults declaratively and generate (parts of) the code for it. +We mitigate this by providing tooling that helps contributors to define different inputs and defaults declaratively +and generate (parts of) the code for it. Additional test tooling will help writing the same test cases onces for different input modes. #### A lot of refactoring @@ -772,5 +787,5 @@ Users and integrators that are using a jsii-supported language (Python, Java, .N ### E. Usage of @aws-cdk/cli-lib-alpha -![Graph of downloads of the @aws-cdk/cli-lib-alpha package per month from Nov 2022 to Oct 2024. The line starts at 0 and climbs to 16k downloads/month in a wave pattern.](../images/0300/cli-lib-alpha-usage.png) +![Graph of downloads of the @aws-cdk/cli-lib-alpha package](../images/0300/cli-lib-alpha-usage.png) [Source](https://npm-stat.com/charts.html?package=%40aws-cdk%2Fcli-lib-alpha&from=2022-11-01&to=2024-10-2)