Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support a list of new AWS Resources for Python SDK #261

Closed
wants to merge 0 commits into from

Conversation

yiyuan-he
Copy link
Contributor

Description of changes:

Adding support for SNS AWS resource for Python SDK with auto-instrumentation.

Specifically, this change populates aws.sns.topic.arn in the Span by extracting the TopicArn field from SnsRequest.

  • CFNIdentifier: "primaryIdentifier": [ "/properties/TopicArn" ]

Testing Plan:

Set up a simple client-server locally and verified that the new data is added to the span.

Screenshot 2024-09-20 at 12 16 59 PM

Ran Code Style Checks:

codespell . --write-changes
python scripts/eachdist.py lint

Added unit tests for:

  • AWS Metric Attribute Generator
  • Instrumentation Patches

Screenshot 2024-09-20 at 12 23 46 PM

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@yiyuan-he yiyuan-he requested a review from a team as a code owner September 20, 2024 19:26
@yiyuan-he yiyuan-he requested a review from mxiamxia September 20, 2024 19:26
Comment on lines 42 to 43
There exists SpanAttributes.MESSAGING_DESTINATION_NAME in the upstream logic that we could re-purpose here. However,
we are not using it here to maintain consistent naming patterns with other AWS resources.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Synced with @mxiamxia about this. Mentioned we should not use the upstream logic to maintain consistent naming patterns for other AWS resources

@@ -407,6 +409,9 @@ def _set_remote_type_and_identifier(span: ReadableSpan, attributes: BoundedAttri
elif is_key_present(span, GEN_AI_REQUEST_MODEL):
remote_resource_type = _NORMALIZED_BEDROCK_SERVICE_NAME + "::Model"
remote_resource_identifier = _escape_delimiters(span.attributes.get(GEN_AI_REQUEST_MODEL))
elif is_key_present(span, AWS_SNS_TOPIC_ARN):
remote_resource_type = _NORMALIZED_SNS_SERVICE_NAME + "::Topic"
remote_resource_identifier = _escape_delimiters(span.attributes.get(AWS_SNS_TOPIC_ARN))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add TODO here that we'll need to parse the actual resource name from the long ARN string by following the rules defined in https://quip-amazon.com/ShMUAP1okoxs/Support-AWS-Resources-with-Different-CFN-Identifier-Formats#temp:C:IHL175110d2baee4144bbe0f54cc

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can create a new attribute named aws.cloudformation.primary.identifier to store the orignal CFN prmiray identifier value from OTel instrumentation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use aws.remote.resource.cfn.primary.identifier and the new attribute name in EMF will be RemoteResourceCfnPrimaryIdentifier?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good 👍

@yiyuan-he yiyuan-he changed the title feat: Add SNS Topic Arn to Span Data feat: Support a list of new AWS Resources for Python SDK Sep 24, 2024
@yiyuan-he yiyuan-he closed this Sep 24, 2024
@yiyuan-he
Copy link
Contributor Author

I reset the main branch of my forked repo so that I can split my future work into multiple branches. It was getting confusing having everything on just my main branch. I am hoping this will enable me to break up my future PRs in a more organized and atomic manner for a better reviewer experience.

yiyuan-he added a commit that referenced this pull request Sep 25, 2024
### *Description of changes:*

These changes support AWS Resources with different CFN Identifier
formats. This new attribute will have the key
`aws.remote.resource.cfn.primary.identifier`.

More context can be found in the comments of this closed PR:
#261

In most cases, this new attribute should have the same value as the
existing attribute `aws.remote.resource.identifier`.

However, there are some edge cases we must handle as seen in this PR:
- `AWS::SQS::Queue`
-
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sqs-queue.html
  - The CFN Id should be the URL of the Queue.
- `AWS::Bedrock::DataSource`
-
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-bedrock-datasource.html
- The CFN Id should be a compound value where the knowledge base Id and
the data source Id are separated by `|`

Also updated Bedrock Agent request parameters to also include Knowledge
Base ID:
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock-agent/client/get_data_source.html

### *Test Plan:*
Set up a client-server with auto-instrumentation to verify that the
correct span data is being generated.


![bedrock_cfn_primary_id_verification](https://github.com/user-attachments/assets/9a375751-a70c-41ab-896a-00e26a5fcc76)

![sqs_cfn_primary_id_verification](https://github.com/user-attachments/assets/a8ee28b0-4344-4f48-82b7-a100da939860)

Unit tests for instrumentation

![metric_generator_unit_test_verification](https://github.com/user-attachments/assets/52c26651-068d-4d40-a020-69714feab17c)

![instrumentation_patch_unit_test_verification](https://github.com/user-attachments/assets/3c23031d-52c2-49e6-9285-da679351ae0d)


By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants