Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[revive] Use the configured AWS partition when constructing ARNs #1554

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
77 changes: 55 additions & 22 deletions config/externalname.go
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the most risky changes in this PR are the conversions of the hardcoded ARNs in the external names to the calls to fullArnTemplate. But the conversion is rather straightforward and we can trigger uptest on the resources with external-name implementation changes, where applicable.

Not probably for this PR, but we could also have a unit test that asserts the expected external-name syntax for the configured resources by invoking the implemented external-name configuration (and probably other aspects of the configuration). cc. @turkenf

Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// Uses the ID of workspace, workspace_id parameter.
"aws_prometheus_alert_manager_definition": config.IdentifierFromProvider,
//
"aws_prometheus_rule_group_namespace": config.TemplatedStringAsIdentifier("name", "arn:aws:aps:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:rulegroupsnamespace/{{ .parameters.workspace_id }}/{{ .external_name }}"),
"aws_prometheus_rule_group_namespace": config.TemplatedStringAsIdentifier("name", fullArnTemplate("aps", "rulegroupsnamespace/{{ .parameters.workspace_id }}/{{ .external_name }}")),
// ID is a random UUID.
"aws_prometheus_workspace": config.IdentifierFromProvider,

Expand Down Expand Up @@ -282,7 +282,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// appflow
//
// arn:aws:appflow:us-west-2:123456789012:flow/example-flow
"aws_appflow_flow": config.TemplatedStringAsIdentifier("name", "arn:aws:appflow:{{ .setup.configuration.region }}:{{ .client_metadata.account_id }}:flow/{{ .external_name }}"),
"aws_appflow_flow": config.TemplatedStringAsIdentifier("name", fullArnTemplate("appflow", "flow/{{ .external_name }}")),

// appintegrations
//
Expand Down Expand Up @@ -422,7 +422,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// arn:aws:batch:us-east-1:123456789012:job-definition/sample:1
"aws_batch_job_definition": config.IdentifierFromProvider,
// Batch Scheduling Policy can be imported using the arn: arn:aws:batch:us-east-1:123456789012:scheduling-policy/sample
"aws_batch_scheduling_policy": config.TemplatedStringAsIdentifier("name", "arn:aws:batch:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:scheduling-policy/{{ .external_name }}"),
"aws_batch_scheduling_policy": config.TemplatedStringAsIdentifier("name", fullArnTemplate("batch", "scheduling-policy/{{ .external_name }}")),

// budgets
//
Expand Down Expand Up @@ -469,7 +469,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
//
// config.NameAsIdentifier did not work, the identifier for the resource turned out to be an ARN
// arn:aws:cloudformation:us-west-1:123456789123:stack/networking-stack/1e691240-6f2c-11ed-8f91-06094dc221f3
"aws_cloudformation_stack": TemplatedStringAsIdentifierWithNoName("arn:aws:cloudformation:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:stack/{{ .parameters.name }}/{{ .external_name }}"),
"aws_cloudformation_stack": TemplatedStringAsIdentifierWithNoName(fullArnTemplate("cloudformation", "stack/{{ .parameters.name }}/{{ .external_name }}")),
// CloudFormation StackSets can be imported using the name
"aws_cloudformation_stack_set": config.NameAsIdentifier,
// Cloudformation Stacks Instances imported using the StackSet name, target
Expand Down Expand Up @@ -517,7 +517,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// cloudtrail
//
// Cloudtrails can be imported using the name arn:aws:cloudtrail:us-west-1:153891904029:trail/foobar
"aws_cloudtrail": config.TemplatedStringAsIdentifier("name", "arn:aws:cloudtrail:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:trail/{{ .external_name }}"),
"aws_cloudtrail": config.TemplatedStringAsIdentifier("name", fullArnTemplate("cloudtrail", "trail/{{ .external_name }}")),
// Event data stores can be imported using their arn
"aws_cloudtrail_event_data_store": config.IdentifierFromProvider,

Expand Down Expand Up @@ -599,7 +599,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// CodeDeploy CustomActionType can be imported using the id
"aws_codepipeline_custom_action_type": config.IdentifierFromProvider,
// CodePipeline Webhooks can be imported by their ARN: arn:aws:codepipeline:us-west-2:123456789012:webhook:example
"aws_codepipeline_webhook": config.TemplatedStringAsIdentifier("name", "arn:aws:codepipeline:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:webhook:{{ .external_name }}"),
"aws_codepipeline_webhook": config.TemplatedStringAsIdentifier("name", fullArnTemplate("codepipeline", "webhook:{{ .external_name }}")),

// codestarconnections
//
Expand Down Expand Up @@ -1112,7 +1112,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// ECS Account Setting defaults can be imported using the name
"aws_ecs_account_setting_default": config.IdentifierFromProvider,
//
"aws_ecs_capacity_provider": config.TemplatedStringAsIdentifier("name", "arn:aws:ecs:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:capacity-provider/{{ .external_name }}"),
"aws_ecs_capacity_provider": config.TemplatedStringAsIdentifier("name", fullArnTemplate("ecs", "capacity-provider/{{ .external_name }}")),
//
"aws_ecs_cluster": config.TemplatedStringAsIdentifier("name", "arn:aws:ecs:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:cluster/{{ .external_name }}"),
// ECS cluster capacity providers can be imported using the cluster_name attribute
Expand Down Expand Up @@ -1259,7 +1259,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
"aws_evidently_project": config.IdentifierFromProvider,
// CloudWatch Evidently Segment can be imported using the arn
// Example: arn:aws:evidently:us-west-2:123456789012:segment/example
"aws_evidently_segment": config.TemplatedStringAsIdentifier("name", "arn:aws:evidently:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:segment/{{ .external_name }}"),
"aws_evidently_segment": config.TemplatedStringAsIdentifier("name", fullArnTemplate("evidently", "segment/{{ .external_name }}")),

// firehose
//
Expand Down Expand Up @@ -1330,7 +1330,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
//
"aws_glue_job": config.NameAsIdentifier,
// Imported using ARN: arn:aws:glue:us-west-2:123456789012:registry/example
"aws_glue_registry": config.TemplatedStringAsIdentifier("registry_name", "arn:aws:glue:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:registry/{{ .external_name }}"),
"aws_glue_registry": config.TemplatedStringAsIdentifier("registry_name", fullArnTemplate("glue", "registry/{{ .external_name }}")),
// Imported using the account ID: 12356789012
"aws_glue_resource_policy": config.IdentifierFromProvider,
// Glue Registries can be imported using arn
Expand Down Expand Up @@ -1399,7 +1399,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// test-role/arn:aws:iam::xxxxxxxxxxxx:policy/test-policy
"aws_iam_role_policy_attachment": config.IdentifierFromProvider,
// IAM SAML Providers can be imported using the arn
"aws_iam_saml_provider": config.TemplatedStringAsIdentifier("name", "arn:aws:iam::{{ .setup.client_metadata.account_id }}:saml-provider/{{ .external_name }}"),
"aws_iam_saml_provider": config.TemplatedStringAsIdentifier("name", regionlessArnTemplate("iam", "saml-provider/{{ .external_name }}")),
// IAM Server Certificates can be imported using the name
"aws_iam_server_certificate": config.NameAsIdentifier,
// IAM service-linked roles can be imported using role ARN that contains the
Expand Down Expand Up @@ -1518,7 +1518,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
"aws_iot_topic_rule": config.NameAsIdentifier,
// IoT topic rule destinations can be imported using the arn
// arn:aws:iot:us-west-2:123456789012:ruledestination/vpc/2ce781c8-68a6-4c52-9c62-63fe489ecc60
"aws_iot_topic_rule_destination": TemplatedStringAsProviderDefinedIdentifier("arn:aws:iot:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:ruledestination/vpc/{{ .external_name }}"),
"aws_iot_topic_rule_destination": TemplatedStringAsProviderDefinedIdentifier(fullArnTemplate("iot", "ruledestination/vpc/{{ .external_name }}")),

// ivs
//
Expand Down Expand Up @@ -1583,18 +1583,18 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// kinesis
//
// Even though the documentation says the ID is name, it uses ARN..
"aws_kinesis_stream": config.TemplatedStringAsIdentifier("name", "arn:aws:kinesis:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:stream/{{ .external_name }}"),
"aws_kinesis_stream": config.TemplatedStringAsIdentifier("name", fullArnTemplate("kinesis", "stream/{{ .external_name }}")),
// Kinesis Stream Consumers can be imported using the Amazon Resource Name (ARN)
// that has a random substring.
"aws_kinesis_stream_consumer": config.IdentifierFromProvider,

// kinesisanalytics
//
"aws_kinesis_analytics_application": config.TemplatedStringAsIdentifier("name", "arn:aws:kinesisanalytics:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:application/{{ .external_name }}"),
"aws_kinesis_analytics_application": config.TemplatedStringAsIdentifier("name", fullArnTemplate("kinesisanalytics", "application/{{ .external_name }}")),

// kinesisanalyticsv2
//
"aws_kinesisanalyticsv2_application": config.TemplatedStringAsIdentifier("name", "arn:aws:kinesisanalytics:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:application/{{ .external_name }}"),
"aws_kinesisanalyticsv2_application": config.TemplatedStringAsIdentifier("name", fullArnTemplate("kinesisanalytics", "application/{{ .external_name }}")),
// aws_kinesisanalyticsv2_application can be imported by using application_name together with snapshot_name
// e.g. example-application/example-snapshot
"aws_kinesisanalyticsv2_application_snapshot": FormattedIdentifierUserDefinedNameLast("snapshot_name", "/", "application_name"),
Expand Down Expand Up @@ -1635,7 +1635,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// lambda
//
// Lambda Function Aliases are identified by their ARN, like arn:aws:lambda:eu-west-1:123456789012:function:lambda-function:alias
"aws_lambda_alias": config.TemplatedStringAsIdentifier("name", "arn:aws:lambda:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:function:{{ .parameters.function_name }}:{{ .external_name }}"),
"aws_lambda_alias": config.TemplatedStringAsIdentifier("name", fullArnTemplate("lambda", "function:{{ .parameters.function_name }}:{{ .external_name }}")),
// Code Signing Configs can be imported using their ARN that has a random
// substring in the end.
// arn:aws:lambda:us-west-2:123456789012:code-signing-config:csc-0f6c334abcdea4d8b
Expand Down Expand Up @@ -1830,13 +1830,13 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
"aws_networkfirewall_firewall": config.IdentifierFromProvider,
// Network Firewall Policies can be imported using their ARN
// Example: arn:aws:network-firewall:us-west-1:123456789012:firewall-policy/example
"aws_networkfirewall_firewall_policy": config.TemplatedStringAsIdentifier("name", "arn:aws:network-firewall:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:firewall-policy/{{ .external_name }}"),
"aws_networkfirewall_firewall_policy": config.TemplatedStringAsIdentifier("name", fullArnTemplate("network-firewall", "firewall-policy/{{ .external_name }}")),
// Network Firewall Logging Configurations can be imported using the firewall_arn
// Example: arn:aws:network-firewall:us-west-1:123456789012:firewall/example
"aws_networkfirewall_logging_configuration": config.IdentifierFromProvider,
// Network Firewall Rule Groups can be imported using their ARN
// Example: arn:aws:network-firewall:us-west-1:123456789012:stateful-rulegroup/example
"aws_networkfirewall_rule_group": config.TemplatedStringAsIdentifier("", "arn:aws:network-firewall:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:{{ .parameters.type | ToLower }}-rulegroup/{{ .external_name }}"),
"aws_networkfirewall_rule_group": config.TemplatedStringAsIdentifier("", fullArnTemplate("network-firewall", "{{ .parameters.type | ToLower }}-rulegroup/{{ .external_name }}")),

// networkmanager
//
Expand Down Expand Up @@ -2343,7 +2343,7 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
// arn:aws:securityhub:eu-west-1:312940875350:action/custom/a
// TODO: following configuration assumes the `a` in the above ARN
// is the security hub custom action identifier
"aws_securityhub_action_target": config.TemplatedStringAsIdentifier("identifier", "arn:aws:securityhub:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:action/custom/{{ .external_name }}"),
"aws_securityhub_action_target": config.TemplatedStringAsIdentifier("identifier", fullArnTemplate("securityhub", "action/custom/{{ .external_name }}")),
// imported using the arn that has a random substring:
// arn:aws:securityhub:eu-west-1:123456789098:finding-aggregator/abcd1234-abcd-1234-1234-abcdef123456
"aws_securityhub_finding_aggregator": config.IdentifierFromProvider,
Expand Down Expand Up @@ -2510,9 +2510,9 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{

// sfn
//
"aws_sfn_activity": config.TemplatedStringAsIdentifier("name", "arn:aws:states:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:activity/{{ .external_name }}"),
"aws_sfn_activity": config.TemplatedStringAsIdentifier("name", fullArnTemplate("states", "activity/{{ .external_name }}")),
//
"aws_sfn_state_machine": config.TemplatedStringAsIdentifier("name", "arn:aws:states:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:stateMachine:{{ .external_name }}"),
"aws_sfn_state_machine": config.TemplatedStringAsIdentifier("name", fullArnTemplate("states", "stateMachine:{{ .external_name }}")),

// signer
//
Expand All @@ -2528,12 +2528,12 @@ var TerraformPluginSDKExternalNameConfigs = map[string]config.ExternalName{
//
// SNS platform applications can be imported using the ARN:
// arn:aws:sns:us-west-2:0123456789012:app/GCM/gcm_application
"aws_sns_platform_application": config.TemplatedStringAsIdentifier("name", "arn:aws:sns:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:app/GCM/{{ .external_name }}"),
"aws_sns_platform_application": config.TemplatedStringAsIdentifier("name", fullArnTemplate("sns", "app/GCM/{{ .external_name }}")),
// no import documentation is provided
// TODO: we will need to check if normalization is possible
"aws_sns_sms_preferences": config.IdentifierFromProvider,
// SNS Topics can be imported using the topic arn
"aws_sns_topic": config.TemplatedStringAsIdentifier("name", "arn:aws:sns:{{ .setup.configuration.region }}:{{ .setup.client_metadata.account_id }}:{{ .external_name }}"),
"aws_sns_topic": config.TemplatedStringAsIdentifier("name", fullArnTemplate("sns", "{{ .external_name }}")),
// SNS Topic Policy can be imported using the topic ARN:
// arn:aws:sns:us-west-2:0123456789012:my-topic
"aws_sns_topic_policy": FormattedIdentifierFromProvider("", "arn"),
Expand Down Expand Up @@ -3228,3 +3228,36 @@ func eksPodIdentityAssociation() config.ExternalName {
}
return e
}

// fullArnTemplate builds a templated string for constructing a terraform id component which is an ARN, which includes
// the aws partition, service, region, account id, and resource. This is by far the most common form of ARN.
// e.g. arn:aws:ec2:ap-south-1:123456789012:instance/i-1234567890ab
func fullArnTemplate(service string, resource string) string {
erhancagirici marked this conversation as resolved.
Show resolved Hide resolved
return genericArnTemplate(service, resource, false, false)

}

// regionlessArnTemplate builds a templated string for constructing a terraform id component which is an ARN of a
// resource which is regionless, but specific to your account id. It includes the partition, service, account id, and
// resource.
// e.g. arn:aws:iam::123456789012:role/example
func regionlessArnTemplate(service string, resource string) string {
return genericArnTemplate(service, resource, false, true)
}

// genericArnTemplate builds a templated string for constructing a terraform id component which is an ARN of any format.
// It always includes the aws partition, service, and resource. Unless you specify to elide them, it will also include
// templates which resolve to the region (from the spec.forProvider) and the account id (calculated from the provider
// config).
func genericArnTemplate(service string, resource string, elideAccountId bool, elideRegion bool) string {
erhancagirici marked this conversation as resolved.
Show resolved Hide resolved
erhancagirici marked this conversation as resolved.
Show resolved Hide resolved
region := "{{ .setup.configuration.region }}"
if elideRegion {
region = ""
}
accountId := "{{ .setup.client_metadata.account_id }}"
if elideAccountId {
accountId = ""
}
partition := "{{ .setup.configuration.partition }}"
return fmt.Sprintf("arn:%s:%s:%s:%s:%s", partition, service, region, accountId, resource)
erhancagirici marked this conversation as resolved.
Show resolved Hide resolved
}
30 changes: 29 additions & 1 deletion internal/clients/aws.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import (
const (
keyAccountID = "account_id"
keyRegion = "region"
keyPartition = "partition"
localstackAccountID = "000000000"
)

Expand All @@ -38,6 +39,16 @@ type SetupConfig struct {
Logger logging.Logger
}

// iamRegions holds the region used for signing IAM credentials for each AWS partition.
var iamRegions = map[string]string{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the source of truth for this table?

Following is a more robust implementation for the signing region lookup keyed by the partition ID:

	// map from a partition to its IAM signing region
	signingRegionMap := make(map[string]string)
	for _, p := range endpoints.DefaultResolver().(endpoints.EnumPartitions).Partitions() {
		for _, r := range p.Regions() {
			re, err := p.EndpointFor("iam", r.ID(), func(opt *endpoints.Options) {
				opt.ResolveUnknownService = true
			})
			if err != nil {
				panic(err)
			}
			signingRegionMap[p.ID()] = re.SigningRegion
			break
		}
	}

It currently produces the following map:

map[aws:us-east-1 aws-cn:cn-north-1 aws-iso:us-iso-east-1 aws-iso-b:us-isob-east-1 aws-iso-e:eu-isoe-west-1 aws-us-gov:us-gov-west-1]

Please note the difference for the partition aws-cn, don't know if this really matters.

Please also see the discussion below regarding partition lookups using the region as a key, which are also applicable here.

Also please note that we have spec.endpoint.signingRegion in the ProviderConfig API, which we should factor into this implementation. But I suggest we do it with a follow-up PR.

"aws": "us-east-1",
"aws-us-gov": "us-gov-west-1",
"aws-cn": "cn-northeast-1",
"aws-iso": "us-iso-east-1",
"aws-iso-b": "us-isob-east-1",
"aws-iso-e": "eu-isoe-west-1",
}

func SelectTerraformSetup(config *SetupConfig) terraform.SetupFn { // nolint:gocyclo
credsCache := NewAWSCredentialsProviderCache(WithCacheLogger(config.Logger))
return func(ctx context.Context, c client.Client, mg resource.Managed) (terraform.Setup, error) {
Expand Down Expand Up @@ -93,7 +104,13 @@ func SelectTerraformSetup(config *SetupConfig) terraform.SetupFn { // nolint:goc
}
ps.ClientMetadata = map[string]string{
keyAccountID: credCache.accountID,
keyPartition: "aws",
}

if pc.Spec.Endpoint != nil && pc.Spec.Endpoint.PartitionID != nil {
ps.ClientMetadata[keyPartition] = *pc.Spec.Endpoint.PartitionID
}
Comment on lines +110 to +112
Copy link
Collaborator

@ulucinar ulucinar Nov 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:
Looks like we have a robust & low maintenance cost way of mapping a region to a partition:

  • Here, it's stated that every region is in exactly one partition.
  • aws-sdk-go maintains the set of available regions for each partition.

So, we could set up a map from regions identifiers to parition identifiers using the github.com/aws/aws-sdk-go/aws/endpoints as follows:

	// map from a region to its partition
	regionMap := make(map[string]string)
	for _, p := range endpoints.DefaultResolver().(endpoints.EnumPartitions).Partitions() {
		for id, _ := range p.Regions() {
			// not necessary, as a sanity check in case we've misunderstood something
			if pn := regionMap[id]; pn != "" {
				panic(errors.Errorf("region %q is already in partition %q", id, pn))
			}
			regionMap[id] = p.ID()
		}
	}

Then if the resource has spec.forProvider.region specified, we can deduce the partition to be used from the regionMap. Let's initialize the map only once in init.

Because spec.endpoint.partitionId is already part of our ProviderConfig API, let's also keep it so that if a user wants to override the partition we compute, we still honor it (so we had better lookup the partition first and override it if ProviderConfig specifies one).

In case new regions are added before we bump the aws-go-sdk version (which depends on upstream Terraform), ProviderConfig overrides should be a workaround (so if the map does not contain an entry for the specified region, let's not immediately error but instead, check the ProviderConfig's spec.endpoint.partitionId override.

We can always defer to a default parition, i.e., aws if no partition resolution mechanism yields one.

Not mandatory but I believe it will yield a better UX while the implementation should be robust. Also if at some point, we need to register a new region with the region map but cannot update the aws-go-sdk to a version that already contains the region, we can extend the map with a manually added key until the SDK update.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was also in my mind. I was a bit hesitant to source partitions from github.com/aws/aws-sdk-go as it has a deprecation notice here, stating

Per that announcement, as of 7/31/2024, the SDK has entered maintenance mode. Going forward, we will limit releases to address critical bug fixes and security issues only. The SDK will not receive API updates for new or existing services, or be updated to support new regions.
Maintenance mode will last for 1 year. After 7/31/2025, the SDK will enter end-of-support, and no updates at all will be made. We recommend that you migrate to AWS SDK for Go v2. For additional details as well as information on how to migrate, please refer to the linked announcement.

Searched for an equivalent in aws-sdk-go-v2 but relevant parts were moved to internal packages, (if I did not miss anything) I could not find anything exported for this.

probably for short term this won't be an issue, but I thought that we can implement something for now to enable the feature, and iterate for cheaper maintenance code.


// several external name configs depend on the setup.Configuration for templating region
ps.Configuration = map[string]any{
keyRegion: awsCfg.Region,
Expand Down Expand Up @@ -127,11 +144,22 @@ func getAWSConfigWithDefaultRegion(ctx context.Context, c client.Client, obj run
return nil, err
}
if cfg.Region == "" && obj.GetObjectKind().GroupVersionKind().Group == "iam.aws.upbound.io" {
cfg.Region = "us-east-1"
cfg.Region = getIAMRegion(pc)
}
return cfg, nil
}

func getIAMRegion(pc *v1beta1.ProviderConfig) string {
defaultRegion := "us-east-1"
if pc == nil || pc.Spec.Endpoint == nil || pc.Spec.Endpoint.PartitionID == nil {
return defaultRegion
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see the comment regarding the iamRegions map above. We should not be using us-east-1 for the partitions other than aws and this implementation leaves the burden of configuring the right partition for an IAM resource to the user by specifying a proper ProviderConfig. While we should allow for overrides via the ProviderConfig (as it's already part of the ProviderConfig API, and for cases where the SDK has not updated its known partitions), we can provide a better UX with a robust & low maintenance cost implementation that relies on the aws-go-sdk.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ulucinar while I understand the comment about iamRegions map, I don't think it is directly relevant for the particular case of IAM resources. Note that this code part is only called for IAM MRs.

this implementation leaves the burden of configuring the right partition for an IAM resource to the user by specifying a proper ProviderConfig

IIUC, unlike other resources, we anyways need some "partition-like" input at the ProviderConfig level as IAM resources are "regionless", so there is nothing else from which we can "infer" the partition, by only looking at the MR. Configuring a partition is actually a requirement rather than a config burden.

Alternative can be introducing an optional "partition" input for IAM MR specs, but I don't think this has a big benefit. Because the associated credentials of the provider config are "partition-bound" as described here, i.e. they will be only valid for a single particular partition. The credentials & ProviderConfig actually "dictates" the partition already.

Unlike handling of regions, where it is possible to use the same provider config and switch regions at MR, this does not make sense for partitions.

Copy link
Collaborator

@ulucinar ulucinar Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I don't like about this implementation is that we do not inject the region parameter for IAM resources and we rely on spec.endpoint.partitionID to choose the correct region. For resources with an explicit region parameter, we can in fact infer the correct region and the signing region from that parameter. The current implementation provides a more uniform UX in this regard. You always need to specify the required region parameter (for non-IAM resources) and if the specified region is not in the aws partition, then specify the correct partition in spec.endpoint.partitionID, which can be improved. But then we have slightly different semantics for IAM resources.

Looks like to me it would have been better to have region as spec.region in the ProviderConfig API.

}
if region, ok := iamRegions[*pc.Spec.Endpoint.PartitionID]; ok {
return region
}
return defaultRegion
}

type metaOnlyPrimary struct {
meta any
}
Expand Down
Loading