aws-cdk-lib > codepipeline: Setting cliVersion can cause an invalid version of cdk-assets to attempt to be installed

[sethmiller]

Describe the bug

The property cliVersion in CodePipeline is used to infer the version of cdk-assets to install in the publishAssetsAction.

Since #31119 was released, this causes the install to fail and the pipeline to break.

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

2.153.0

Expected Behavior

The pipeline buildspec will be generated with a command to install a version of cdk-assets that is appropriate for the given version of aws-cdk.

Current Behavior

The pipeline buildspec will be generated with a command to install cdk-assets with an invalid version string. This causes the build to fail with the following error.

[Container] 2024/08/29 14:43:38.596103 Running command npm install -g [email protected]
npm ERR! code ETARGET
npm ERR! notarget No matching version found for [email protected].
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

Reproduction Steps

Building this CDK Stack will generate a buildspec with the invalid npm install command. Note that CrossAccountZoneDelegationRecord is what triggers the behavior.

The full repo can be found here.

import { App, Stack, StackProps, Stage, StageProps } from 'aws-cdk-lib';
import { Role } from 'aws-cdk-lib/aws-iam';
import { CrossAccountZoneDelegationRecord, HostedZone } from 'aws-cdk-lib/aws-route53';
import { ShellStep, CodePipeline, CodePipelineSource } from 'aws-cdk-lib/pipelines';
import { Construct } from 'constructs';

class MyServiceStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    // This Construct adds a node with the type `publish-assets` which triggers the behavior.
    new CrossAccountZoneDelegationRecord(this, 'cross-account-zone', {
      delegatedZone: HostedZone.fromHostedZoneAttributes(this, 'my-hosted-zone', {
        hostedZoneId: 'Z01201272X6Y0ABCDE0FG',
        zoneName: 'zone.name',
      }),
      delegationRole: Role.fromRoleName(this, 'my-role', 'my-role-name'),
      parentHostedZoneId: 'parent-hosted-zone',
    });
  }
}

class MyStage extends Stage {
  constructor(scope: Construct, id: string, props?: StageProps) {
    super(scope, id, props);

    new MyServiceStack(this, 'my-stack');
  }
}

export class MyStack extends Stack {
  constructor(scope: Construct, id: string, props: StackProps = {}) {
    super(scope, id, props);

    const pipeline = new CodePipeline(this, 'my-pipeline', {
      synth: new ShellStep('my-build', {
        input: CodePipelineSource.gitHub(
          'my-org/my-repo',
          'main',
        ),
        commands: [
          "echo 'hello world'",
        ],
      }),
      cliVersion: '2.154.1',
    });

    const wave = pipeline.addWave('my-wave');
    wave.addStage(new MyStage(this, 'my-stage'));
  }
}

const devEnv = {
  account: process.env.CDK_DEFAULT_ACCOUNT,
  region: process.env.CDK_DEFAULT_REGION,
};

const app = new App();

new MyStack(app, 'cdk-assets-bug-dev', { env: devEnv });

app.synth();

Possible Solution

Either don't try to infer the version of cdk-assets at all or allow it to be set explicitly if cliVersion is set.

Additional Information/Context

This commit was the source of this issue (PR #31119).

CDK CLI Version

2.154.1

Framework Version

No response

Node.js Version

v20.15.1

OS

Linux

Language

TypeScript

Language Version

5.5.4

Other information

No response

[ashishdhingra]

Could be an issue since running cdk synth generates the below resource in CFN template which has command to run npm install -g [email protected]:

...
  mypipelineAssetsFileAsset18008F02D:
    Type: AWS::CodeBuild::Project
    Properties:
      Artifacts:
        Type: CODEPIPELINE
      Cache:
        Type: NO_CACHE
      Description: Pipeline step cdk-assets-bug-dev/Pipeline/Assets/FileAsset1
      EncryptionKey: alias/aws/s3
      Environment:
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/standard:7.0
        ImagePullCredentialsType: CODEBUILD
        PrivilegedMode: false
        Type: LINUX_CONTAINER
      ServiceRole:
        Fn::GetAtt:
          - mypipelineAssetsFileRole7EA0DD74
          - Arn
      Source:
        BuildSpec: |-
          {
            "version": "0.2",
            "phases": {
              "install": {
                "commands": [
                  "npm install -g [email protected]"
                ]
              },
              "build": {
                "commands": [
                  "cdk-assets --path \"assembly-cdk-assets-bug-dev-my-codepipeline-stage/cdkassetsbugdevmycodepipelinestagemycodepipelinestackCAF97708.assets.json\" --verbose publish \"4cfc7e5d073ec23fbedf66061a6290221275ac87a2477bf6b9aae76b4ca74ddf:current_account-current_region\""
                ]
              }
            }
          }
        Type: CODEPIPELINE
    Metadata:
      aws:cdk:path: cdk-assets-bug-dev/my-pipeline/Assets/FileAsset1/Resource
...

Perhaps we should try to avoid using cliVersion for installing cdk-assets. Needs input from team.

[TheRealAmazonKendra]

Thanks for raising this issue. We are working on a fix for it, however, we do not recommend use of cliVersion in general. Leaving it empty will pull in the latest version and will not attempt to pull in the exact same version of cdk-assets.

See:

aws-cdk/packages/aws-cdk-lib/pipelines/lib/codepipeline/codepipeline.ts

Lines 73 to 74 in 2a27711

	* We recommend you do not specify this value, as not specifying it always
	* uses the latest CLI version which is backwards compatible with old versions.

[sethmiller]

... however, we do not recommend use of cliVersion in general. Leaving it empty will pull in the latest version and will not attempt to pull in the exact same version of cdk-assets.

Yup, we 100% get that (In fact I pasted that same comment to my team in Slack as we were investigating this 😄). We started using this during a period where minor releases were impacting the stability of our deploy pipeline. Since it is been a while since this has been the case, we are going to stop setting cliVersion. That said, it is a nice escape hatch when we have issues with the latest release of aws-cdk.

Thanks for following up so quickly! We'll keep an eye out for updates.

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.