Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support autoscaling managed services wrapping ASGs #281

Open
lenucksi opened this issue Jul 20, 2018 · 1 comment
Open

Support autoscaling managed services wrapping ASGs #281

lenucksi opened this issue Jul 20, 2018 · 1 comment

Comments

@lenucksi
Copy link
Contributor

Support autoscaling managed services wrapping ASGs

Rationale

Currently, Autospotting manages autoscaling groups (ASGs) using standard EC2 instances without any knowledge of the services running in them.
Autospotting expects services to react well to sudden infrastructure failure (i.e. termination of an on-demand instance and raising of a new spot instance) which is acceptable for a lot of services that would be put into an autoscaling group.

There are a number of other services, such as e.g. AWS EMR (i.e. Amazon managed variants of Flink, Spark, Hadoop, etc.) or AWS Batch that run clusters of EC2 instances with managed services on them which might profit from using spot-instances.

Proposal

Thus, this ticket proposes to handle autospotting groups of EC2 instances attached to AWS managed services.

Requirements

There are a number of requirements limiting the feasibility and value of this approach, some of which are listed below.

  • A service needs to be realized using an ASG to be handled
    • The ASG needs to be accessible
  • A service needs to be able to handle sudden infrastructure failure and replacement.
    • The service needs to run long enough such that an instance replacement is worthwhile.
    • The cost of instance replacement in terms of potentially needed reprocessing of data needs to outweigh what is gained by replacing it.

Goals

The goal of this ticket is to identify which hosted services use an ASG as a base unit and evaluate which of them fulfill the requirements to apply Autospotting as stated above.
Also, if a service does fulfill the requirements, collect possible usage experiences around it.
Collect everything and possible change requests around it in a wiki page.

Wikipage with results

https://github.com/cristim/autospotting/wiki/AutospotManagedServices

If you know of any additional use cases with managed services, feel free to add a comment or a reference.

@cristim
Copy link
Member

cristim commented Mar 6, 2023

As far as I know we support anything backed by ASGs:

  • I know of users using AutoSpotting with Beanstalk and EKS managed groups for a long time.
  • EMR doesn't use AutoScaling groups
  • Batch I haven't seen but it may work, may just need a way to pass the required tags

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants