Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Change log level to warning when temporarily partial EC2 info #609

Conversation

lukeseawalker
Copy link
Contributor

@lukeseawalker lukeseawalker commented Dec 29, 2023

Description of changes

  • Change log level to warning when temporarily unable to retrieve EC2 instance info from DescribeInstances call.

The from_describe_instance_data logs a warning and throws an exception. The from_describe_instance_data is called by

  1. fleet_manager.launch_ec2_instances, but here the instances info are already populated in the _launch_instances, which calls the _get_instances_info that does have the retries, by calling _retrieve_instances_info_from_ec2
  2. fleet_manager._retrieve_instances_info_from_ec2 that catch the exception

If at the end of the retries there are still instances info with partial data, in the _launch_instances these instances are finally logged as error.

Tests

  • n/a

References

  • n/a

Checklist

  • Make sure you are pointing to the right branch.
  • If you're creating a patch for a branch other than develop add the branch name as prefix in the PR title (e.g. [release-3.6]).
  • Check all commits' messages are clear, describing what and why vs how.
  • Make sure to have added unit tests or integration tests to cover the new/modified code.
  • Check if documentation is impacted by this change.

Please review the guidelines for contributing and Pull Request Instructions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Change log level to warning when temporarily unable to retrieve EC2 instance info from DescribeInstances call.

The from_describe_instance_data log a warning and throw an exception.
The from_describe_instance_data is called by
1. fleet_manager.launch_ec2_instances, but here the instance info are already populated in the _launch_instances, which calls the _get_instances_info that does have the retries, by calling _retrieve_instances_info_from_ec2
2. fleet_manager._retrieve_instances_info_from_ec2 that catch the exception

If at the end of the retires there are still instances info with partial data, in the _launch_instances these instances are finally logged as error

Signed-off-by: Luca Carrogu <[email protected]>
Copy link

codecov bot commented Dec 29, 2023

Codecov Report

Attention: 23 lines in your changes are missing coverage. Please review.

Comparison is base (2d2b6d6) 90.86% compared to head (5285ed3) 90.87%.
Report is 63 commits behind head on develop.

Files Patch % Lines
src/aws/common.py 77.90% 19 Missing ⚠️
src/common/utils.py 91.17% 3 Missing ⚠️
...rc/common/schedulers/slurm_reservation_commands.py 98.85% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #609      +/-   ##
===========================================
+ Coverage    90.86%   90.87%   +0.01%     
===========================================
  Files           20       20              
  Lines         3152     3156       +4     
===========================================
+ Hits          2864     2868       +4     
  Misses         288      288              
Flag Coverage Δ
unittests 90.87% <95.38%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lukeseawalker lukeseawalker enabled auto-merge (rebase) January 2, 2024 15:27
@lukeseawalker lukeseawalker merged commit d8e8b25 into aws:develop Jan 2, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants