-
Notifications
You must be signed in to change notification settings - Fork 3.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(kinesisfirehose-alpha): refactor sourceStream property to suppor…
…t multiple types of sources (#31723) ### Reason for this change The previous API for `source` was designed under the assumption that a Source would either be a `Stream` or `Direct Put` if not. Since the alpha module was written, support on the service side for MSK as a Source has been added so we should update the `source` property to accept an `ISource` which can then be implemented by different types of Sources. ### Description of changes Replaced the `sourceStream` property with `source`. Changed the `source` property from `IStream` to `ISource`. Added an `ISource` interface which is implemented by classes which represent the different Source types. Currently implemented by the `KinesisStreamSource` class. The `MSKSource` class can be added in a separate PR. Added a `SourceConfig` which contains the property configs for each respective source (as the fields within these property configs are different across each source). In `delivery-stream.ts` we call the `_bind` method which will populate and return the correct property config for the Source and that gets directly injected where the L1 `CFNDeliveryStream` is created. This pattern is also used for Destinations: ```ts const destinationConfig = props.destination.bind(this, {}); const sourceConfig = props.source?._bind(this, this._role?.roleArn); const resource = new CfnDeliveryStream(this, 'Resource', { deliveryStreamEncryptionConfigurationInput: encryptionConfig, deliveryStreamName: props.deliveryStreamName, deliveryStreamType: props.source ? 'KinesisStreamAsSource' : 'DirectPut', ...sourceConfig, ...destinationConfig, }); ``` ### Description of how you validated changes no behavioural changes. the updated integ tests and unit tests still pass existing tests. exempting integ tests because we don't want the generated CFN to change. ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* --- BREAKING CHANGE: Replaced the `sourceStream` property with `source`. Changed the `source` property type from `IStream` to `ISource`. Instead of passing in the source Stream directly, it will be passed in by calling the appropriate class like so: `source: new source.KinesisStreamSource(sourceStream)`.
- Loading branch information
Showing
6 changed files
with
103 additions
and
25 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| import { Construct } from 'constructs'; | ||
| import { CfnDeliveryStream } from 'aws-cdk-lib/aws-kinesisfirehose'; | ||
| import * as iam from 'aws-cdk-lib/aws-iam'; | ||
| import * as kinesis from 'aws-cdk-lib/aws-kinesis'; | ||
|
|
||
| /** | ||
| * A Kinesis Data Firehose delivery stream source configuration. | ||
| */ | ||
| interface SourceConfig { | ||
| /** | ||
| * Configuration for using a Kinesis Data Stream as a source for the delivery stream. | ||
| * | ||
| * This will be returned by the _bind method depending on what type of Source class is specified. | ||
| * | ||
| * @default - Kinesis Data Stream Source configuration property is not provided. | ||
| */ | ||
| readonly kinesisStreamSourceConfiguration?: CfnDeliveryStream.KinesisStreamSourceConfigurationProperty; | ||
|
|
||
| /** | ||
| * Configuration for using an MSK (Managed Streaming for Kafka) cluster as a source for the delivery stream. | ||
| * | ||
| * This will be returned by the _bind method depending on what type of Source class is specified. | ||
| * | ||
| * @default - MSK Source configuration property is not provided. | ||
| */ | ||
| readonly mskSourceConfiguration?: CfnDeliveryStream.MSKSourceConfigurationProperty; | ||
| } | ||
|
|
||
| /** | ||
| * An interface for defining a source that can be used in a Kinesis Data Firehose delivery stream. | ||
| */ | ||
| export interface ISource { | ||
| /** | ||
| * Binds this source to the Kinesis Data Firehose delivery stream. | ||
| * | ||
| * @internal | ||
| */ | ||
| _bind(scope: Construct, roleArn?: string): SourceConfig; | ||
|
|
||
| /** | ||
| * Grant read permissions for this source resource and its contents to an IAM | ||
| * principal (the delivery stream). | ||
| * | ||
| * If an encryption key is used, permission to use the key to decrypt the | ||
| * contents of the stream will also be granted. | ||
| */ | ||
| grantRead(grantee: iam.IGrantable): iam.Grant; | ||
| } | ||
|
|
||
| /** | ||
| * A Kinesis Data Firehose delivery stream source. | ||
| */ | ||
| export class KinesisStreamSource implements ISource { | ||
|
|
||
| /** | ||
| * Creates a new KinesisStreamSource. | ||
| */ | ||
| constructor(private readonly stream: kinesis.IStream) {} | ||
|
|
||
| grantRead(grantee: iam.IGrantable): iam.Grant { | ||
| return this.stream.grantRead(grantee); | ||
| } | ||
|
|
||
| /** | ||
| * Binds the Kinesis stream as a source for the Kinesis Data Firehose delivery stream. | ||
| * | ||
| * @returns The configuration needed to use this Kinesis stream as the delivery stream source. | ||
| * @internal | ||
| */ | ||
| _bind(_scope: Construct, roleArn: string): SourceConfig { | ||
| return { | ||
| kinesisStreamSourceConfiguration: { | ||
| kinesisStreamArn: this.stream.streamArn, | ||
| roleArn: roleArn, | ||
| }, | ||
| }; | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters