[Enhancement]: Improve error message for: deleting Security Group (sg-0ec09a4fb869084b2): DependencyViolation: resource sg-0ec09a4fb869084b2 has a dependent object

[EugenKon]

Description

https://www.youtube.com/watch?v=ZRhKcnEgAtc

Here is how error message shown by terraform:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  ~ update in-place
  - destroy

Terraform will perform the following actions:

  # module.client-vpn[0].aws_security_group.client_vpn_access (deposed object 999c8886) will be destroyed
  # (left over from a partially-failed replacement of this instance)
  - resource "aws_security_group" "client_vpn_access" {
      - arn                    = "arn:aws:ec2:us-west-2:315400321086:security-group/sg-058ef7f94e6a0a695" -> null
      - description            = "Managed by Terraform" -> null
      - egress                 = [
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - description      = ""
              - from_port        = 0
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "-1"
              - security_groups  = []
              - self             = false
              - to_port          = 0
            },
        ] -> null
      - id                     = "sg-058ef7f94e6a0a695" -> null
      - ingress                = [
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - description      = ""
              - from_port        = 0
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "-1"
              - security_groups  = []
              - self             = false
              - to_port          = 0
            },
        ] -> null
      - name                   = "security group for client vpn endpoint" -> null
      - owner_id               = "315400321086" -> null
      - revoke_rules_on_delete = false -> null
      - tags                   = {} -> null
      - tags_all               = {
          - "Project" = "platform-exp"
        } -> null
      - vpc_id                 = "vpc-0d639d3e779362f6b" -> null
    }

  # module.private-cloud.aws_instance.www will be updated in-place
  ~ resource "aws_instance" "www" {
        id                                   = "i-06fc0b6b06afc9cdd"
        tags                                 = {
            "Name" = "www"
        }
      ~ vpc_security_group_ids               = [
          - "sg-07070d44c7de49423",
          - "sg-0711bfadcad25bc88",
          - "sg-0ec09a4fb869084b2",
          + "sg-08daafae770c045e1",
          + "sg-091c1fa7f5ce25ba8",
          + "sg-09eed4a303c063c01",
        ]
        # (33 unchanged attributes hidden)

        # (9 unchanged blocks hidden)
    }

  # module.private-cloud.aws_security_group.egress_for_all (deposed object ae85722e) will be destroyed
  # (left over from a partially-failed replacement of this instance)
  - resource "aws_security_group" "egress_for_all" {
      - arn                    = "arn:aws:ec2:us-west-2:315400321086:security-group/sg-0711bfadcad25bc88" -> null
      - description            = "Allow all outbound traffic" -> null
      - egress                 = [
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - description      = ""
              - from_port        = 0
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "-1"
              - security_groups  = []
              - self             = false
              - to_port          = 0
            },
        ] -> null
      - id                     = "sg-0711bfadcad25bc88" -> null
      - ingress                = [] -> null
      - name                   = "platform-exp_allow_all_egress" -> null
      - owner_id               = "315400321086" -> null
      - revoke_rules_on_delete = false -> null
      - tags                   = {} -> null
      - tags_all               = {
          - "Project" = "platform-exp"
        } -> null
      - vpc_id                 = "vpc-0d639d3e779362f6b" -> null
    }

  # module.private-cloud.aws_security_group.ssh_for_all[0] will be destroyed
  # (because index [0] is out of range for count)
  - resource "aws_security_group" "ssh_for_all" {
      - arn                    = "arn:aws:ec2:us-west-2:315400321086:security-group/sg-0ec09a4fb869084b2" -> null
      - description            = "Allow SSH traffic from anywhere" -> null
      - egress                 = [] -> null
      - id                     = "sg-0ec09a4fb869084b2" -> null
      - ingress                = [
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - description      = ""
              - from_port        = 22
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "tcp"
              - security_groups  = []
              - self             = false
              - to_port          = 22
            },
        ] -> null
      - name                   = "platform-exp_ssh_for_all" -> null
      - owner_id               = "315400321086" -> null
      - revoke_rules_on_delete = false -> null
      - tags                   = {} -> null
      - tags_all               = {
          - "Project" = "platform-exp"
        } -> null
      - vpc_id                 = "vpc-0d639d3e779362f6b" -> null
    }

  # module.private-cloud.aws_security_group.www[0] will be destroyed
  # (because index [0] is out of range for count)
  - resource "aws_security_group" "www" {
      - arn                    = "arn:aws:ec2:us-west-2:315400321086:security-group/sg-07070d44c7de49423" -> null
      - description            = "Allow HTTP/HTTPS traffic from anywhere" -> null
      - egress                 = [] -> null
      - id                     = "sg-07070d44c7de49423" -> null
      - ingress                = [
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - description      = ""
              - from_port        = 443
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "tcp"
              - security_groups  = []
              - self             = false
              - to_port          = 443
            },
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - description      = ""
              - from_port        = 80
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "tcp"
              - security_groups  = []
              - self             = false
              - to_port          = 80
            },
        ] -> null
      - name                   = "platform-exp_www" -> null
      - owner_id               = "315400321086" -> null
      - revoke_rules_on_delete = false -> null
      - tags                   = {} -> null
      - tags_all               = {
          - "Project" = "platform-exp"
        } -> null
      - vpc_id                 = "vpc-0d639d3e779362f6b" -> null
    }

  # module.private-cloud.aws_security_group.www_ssh_only_vpn[0] (deposed object 631393b1) will be destroyed
  # (left over from a partially-failed replacement of this instance)
  - resource "aws_security_group" "www_ssh_only_vpn" {
      - arn                    = "arn:aws:ec2:us-west-2:315400321086:security-group/sg-07bf1a6ca28dbd8ab" -> null
      - description            = "Allow HTTP/HTTPS traffic from anywhere" -> null
      - egress                 = [] -> null
      - id                     = "sg-07bf1a6ca28dbd8ab" -> null
      - ingress                = [
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - description      = ""
              - from_port        = 443
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "tcp"
              - security_groups  = []
              - self             = false
              - to_port          = 443
            },
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - description      = ""
              - from_port        = 80
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "tcp"
              - security_groups  = []
              - self             = false
              - to_port          = 80
            },
        ] -> null
      - name                   = "platform-exp_www_ssh_only" -> null
      - owner_id               = "315400321086" -> null
      - revoke_rules_on_delete = false -> null
      - tags                   = {} -> null
      - tags_all               = {
          - "Project" = "platform-exp"
        } -> null
      - vpc_id                 = "vpc-0d639d3e779362f6b" -> null
    }

Plan: 0 to add, 1 to change, 5 to destroy.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: ../../state.d/terraform.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "../../state.d/terraform.plan"

[prtl] OK    "terraform plan -state ../../state.d/terraform.tfstate -var project_name=platform-exp -var aws_region=us-west-2 -var domain_name=platform-exp.plntr.ca -var use_dns=true -var vpn_setup=client-ssh-only -var use_elastic_ip=false -out ../../state.d/terraform.plan"

kes@Eugens-MacBook-Pro platform-exp $ ../prtl aws apply

[prtl] Run   "terraform apply -state ../../state.d/terraform.tfstate ../../state.d/terraform.plan"

module.private-cloud.aws_security_group.ssh_for_all[0]: Destroying... [id=sg-0ec09a4fb869084b2]
module.private-cloud.aws_security_group.www[0]: Destroying... [id=sg-07070d44c7de49423]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 10s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 10s elapsed]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 20s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 20s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 30s elapsed]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 30s elapsed]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 40s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 40s elapsed]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 50s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 50s elapsed]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 1m0s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 1m0s elapsed]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 1m10s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 1m10s elapsed]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 1m20s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 1m20s elapsed]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 1m31s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 1m31s elapsed]
module.private-cloud.aws_security_group.www[0]: Still destroying... [id=sg-07070d44c7de49423, 1m41s elapsed]
module.private-cloud.aws_security_group.ssh_for_all[0]: Still destroying... [id=sg-0ec09a4fb869084b2, 1m41s elapsed]
^CStopping operation...

Interrupt received.
Please wait for Terraform to exit or data loss may occur.
Gracefully shutting down...


kes@Eugens-MacBook-Pro platform-exp $ ╷
│ Error: deleting Security Group (sg-0ec09a4fb869084b2): DependencyViolation: resource sg-0ec09a4fb869084b2 has a dependent object
│ 	status code: 400, request id: 8e8c19b8-ece3-46a6-a849-e8fd9317335b
│
│
╵
╷
│ Error: execution halted
│
│
╵
╷
│ Error: execution halted
│
│
╵
╷
│ Error: deleting Security Group (sg-07070d44c7de49423): DependencyViolation: resource sg-07070d44c7de49423 has a dependent object
│ 	status code: 400, request id: 1bcea930-ead8-48ec-91a9-cec6a5f9e896
│
│
╵
╷
│ Error: execution halted
│
│
╵
╷
│ Error: execution halted
│
│
╵

Here the message how it was diplayed by AWS:

It would be nice, if terraform show meaning full message instead of just status code: 400, request id: 1bcea930-ead8-48ec-91a9-cec6a5f9e896

Affected Resource(s) and/or Data Source(s)

• aws_security_group

Potential Terraform Configuration

resource "aws_security_group" "ssh_for_all" {
  count = var.vpn_setup == "" ? 1 : 0

  name_prefix = "${var.project_name}_ssh_for_all"
  description = "Allow SSH traffic from anywhere"

  vpc_id = aws_vpc.main.id

  ingress {
    protocol    = "tcp"
    from_port   = 22
    to_port     = 22
    cidr_blocks = ["0.0.0.0/0"]
  }

  lifecycle {
    create_before_destroy = true
  }
}

References

Probably related to: #1671

https://repost.aws/knowledge-center/troubleshoot-delete-vpc-sg (scroll to The security group is associated with a network interface )

Would you like to implement a fix?

No

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[EugenKon]

Manualy removing security groups from interface helps.

[EugenKon]

same for another security group:

[EugenKon]

The tool which could help: https://github.com/mingbowan/sgdeps

[justinretzolk]

Hey @EugenKon 👋 Thank you for taking the time to raise this! Can you verify what version of Terraform and the AWS Provider you're using when experiencing this behavior?

[EugenKon]

@justinretzolk I just installed terraform few days ago:

$ terraform --version
Terraform v1.5.4
on darwin_amd64

terraform version -json
{
  "terraform_version": "1.5.4",
  "platform": "darwin_amd64",
  "provider_selections": {
    "registry.terraform.io/hashicorp/aws": "4.67.0",
    "registry.terraform.io/hashicorp/external": "2.3.1",
    "registry.terraform.io/hashicorp/local": "2.3.0",
    "registry.terraform.io/hashicorp/null": "3.2.1"
  },
  "terraform_outdated": false
}

UPD
I also upgraded provider to 5.10.0 version. The problem still exists.

[EugenKon]

Another not meaningful error messages:

module.dns.aws_route53_zone.private-cloud[0]: Modifying... [id=Z0803079GS4S3AQWFY1V]
╷
│ Error: associating Route53 Hosted Zone (Z0803079GS4S3AQWFY1V) to VPC (vpc-0d639d3e779362f6b): PublicZoneVPCAssociation: Attempting to associate public zone: Z0803079GS4S3AQWFY1V with vpc: vpc-0d639d3e779362f6b
│ 	status code: 400, request id: ba65e3de-ce13-4e36-9b0a-da8bc58de3fb
│
│   with module.dns.aws_route53_zone.private-cloud[0],
│   on modules/dns/route53.tf line 1, in resource "aws_route53_zone" "private-cloud":
│    1: resource "aws_route53_zone" "private-cloud" {
│
╵

╷
│ Warning: Argument is deprecated
│
│   with module.private-cloud.aws_eip.www,
│   on modules/private-cloud/ec2.tf line 93, in resource "aws_eip" "www":
│   93:   vpc = true
│
│ use domain attribute instead

Should I here use vpn = domain?. It would be nice if you print message like:

use domain attribute instead, eg. `vpn = domain`

[justinretzolk]

Hey @EugenKon 👋 Thank you for the additional information. I've got a few bits of information that might be helpful here, though unfortunately I'm not certain they'll solve everything you're looking for here.

As far as the main issue with security group deletion, this is enough of an issue that we have a large section in the resource documentation covering some of the intricacies of this particular resource. I took a moment to re-read it, and found this particular line relevant:

In addition, when the aws_security_group resource attempts to recreate, it receives a dependent object error, which does not provide information on whether the dependent object is a security group rule or, for example, an associated EC2 instance. Within Terraform, the associated resource (e.g., aws_instance) does not receive an error when the aws_security_group is trying to recreate even though that is where changes to the associated resource would need to take place (e.g., removing the security group association).

This is an important callout due to the first sentence; we don't get a more descriptive error back, so surfacing a more helpful error is likely not possible at this time. Similarly, referencing the API documentation for DeleteSecurityGroup, there are no references to more specific error messages, rather, only common client errors are returned, which is where the DependencyViolation error comes from.

As far as the warning on the deprecated vpc argument of the aws_eip resource, my perspective on the intent is to guide you towards referencing the resource documentation for assistance with what to do about deprecated arguments. In this case, you'd want to use domain = "vpc" to indicate that the EIP is for us in a VPC.

[EugenKon]

@justinretzolk: Thank you for links

So we know the source of problem. I could suggest to change a strategy a bit. Instead of a relay on AWS errors when terraform does API request to delete security group, it could make request to describe resources, eg.

$ aws ec2 describe-network-interfaces --filters Name=group-id,Values=sg-07070d44c7de49423 platform-exp_www --region us-west-2 --output json^C
$ aws ec2 describe-security-groups --query "SecurityGroups[*].{ID:GroupId,Name:GroupName,dependentOnSGs:IpPermissions[].UserIdGroupPairs[].GroupId}

And if there are dependency it could return meaningful error message or, even, delete/disassociate this sg itself.

This couple of additional requests are significantly faster then a waiting 8-15mins.

[justinretzolk]

@EugenKon I definitely understand where you're coming from, but there's a couple of things that would prevent us from doing so.

One of the tenants of our Provider Design Principles is that resources should represent a single API object. Doing so makes lowers the burden of maintainership by limiting the resource to a single underlying component, which in turn helps to keep the resource behavior consistent with what users would expect from the underlying API (and the related CLI functionality).

Related, adding the calls that you mentioned could cause unexpected permissions errors that could lead to additional confusion. If the IAM role used to authenticate the AWS Provider in a given configuration was not scoped to allow describing network interfaces (or other potentially dependent resources), an error indicating such would be thrown. For some users, adding those additional permissions might not be acceptable, which would leave them unable to use the resource.

If I've overlooked something, got something wrong, or if you have additional input, I'm more than happy to continue chatting about this! I absolutely agree that I'd love to see a more useful error output here; I'm just not sure that it's possible at the moment given what we've discussed so far.