google_compute_network data source now fails instantly when referring to a network that is also about to be created in 5.x

[gygitlab]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

v1.5.5

Affected Resource(s)

• google_compute_network (data)

Terraform Configuration Files

locals {
  create_test_network = true

  test_vpc_name = local.create_test_network ? google_compute_network.test_vpc[0].name : "default"
}

data "google_compute_network" "test_network" {
  name = local.test_vpc_name
}

resource "google_compute_network" "test_vpc" {
  count = local.create_test_network ? 1 : 0

  name                    = "test-vpc"
  auto_create_subnetworks = false
}

Expected Behavior

In the 4.x Terraform provider the google_compute_network data source would wait gracefully if it's passed a name of a network to be created.

Actual Behavior

In 5.x this behaviour no longer happens and the data source immediately attempts to find the network and fails as a race condition:

│ Error: projects/test-d0c40c/global/networks/test-vpc not found
│
│   with module.test_network,
│   on terraform/modules/test/network.tf line 10, in data "google_compute_network" "test_network":
│   10: data "google_compute_network" "test_network" {
│

Steps to Reproduce

Have a google_compute_network data source referring to the name parameter of a google_compute_network resource (typically used in conditionals when working with multiple network types) and attempt to apply with the 4.x and 5.x GCP provider.

Note this works fine with 4.x gracefully but fails immediately on 5.x

[edwardmedia]

@gygitlab using your code above, I can't repro the issue. I do not hit the error in both v4.80.0 and v5.0.0.

Can you share your debug logs for both 4.x and 5.x and detail the steps to repro?

[gygitlab]

@gygitlab using your code above, I can't repro the issue. I do not hit the error in both v4.80.0 and v5.0.0.

Can you share your debug logs for both 4.x and 5.x and detail the steps to repro?

Huh how interesting, well like on the other issue our code is here but I'll see if I can whip up a more specific example for you later this week.

[edwardmedia]

@gygitlab I guess you have followed the v5.0.0 upgrade guide when you moved to v5.x, right?

[gygitlab]

@gygitlab I guess you have followed the v5.0.0 upgrade guide when you moved to v5.x, right?

I did indeed as well as do a clean build with 5.0.0 yeah (well it failed instantly since the data source attempts to find the database instantly before it's created)

[edwardmedia]

@gygitlab waiting for your config and repro steps

[gygitlab]

Ok had another go at this and indeed it's certainly a bit weird but I am able to recreate it with the following config:

locals {
  create_test_network = true

  test_vpc_name = local.create_test_network ? google_compute_network.test_vpc[0].name : "default"
}

data "google_compute_network" "test_network" {
  name = local.test_vpc_name
}

resource "google_compute_network" "test_vpc" {
  count = local.create_test_network ? 1 : 0

  name                    = "test-vpc"
  auto_create_subnetworks = false
}

Effectively for some reason with 5.x the Terraform behavior rules around reading data resurce are being triggered differently with the above and it's attempting to read the data during the planning phase.

To put that plainly with the above config Terraform thinks local.test_vpc_name is a string when it's actually a deferred reference to google_compute_network.test_vpc[0].name.

While this feels like it could be a Terraform issue this exact same code passes absolutely fine with v4.84.0. Very weird.

Debug outputs of terraform plan for both versions - https://gist.github.com/gygitlab/fd248a54bb71bf93e5fa7f62ed5b3ff5

[edwardmedia]

Yeah, I can repro the issue now. Thanks @gygitlab

@roaks3 I see HandleDataSourceNotFoundError was introduced in 5.x. Should it be used for this data source?

[NickElliot]

This is working as intended -- the reason it worked before was purely because the data source in question did not return errors on 404s, which was inconsistent across the provider and was intended to function as it does now (this prior mistaken functionality giving the illusion of "waiting" the way a deferred plan does). But to elaborate why this isn't a deferred data source using the rules on the page you linked:

At least one of the given arguments is a managed resource attribute or other value that Terraform cannot predict until the apply step.

google_compute_network.test_vpc[0].name is an assumed unchanging non-computed value that is statically provided by the user - therefore it exists as a predicted attribute.

The data resource depends directly on a managed resource that itself has planned changes in the current plan.

because of the above, (data) google_compute_network.test_network is not actually dependent on (resource) google_compute_network.test_vpc because no computed attribute is needed to create the data source

The data resource has custom conditions and it depends directly or indirectly on a managed resource that itself has planned changes in the current plan.

no custom conditions are used in this config, and this would a potential solution to your config, updating the data source block to resemble the following

  provider=google-beta
  name = local.test_vpc_name
  lifecycle {
    postcondition {
      condition     = google_compute_network.test_vpc[0].id != ""
      error_message = "google_compute_network.test_vpc[0] must be created."
    }
  }
}

or doing some other change within your locals{} values to make test_vpc_name depend on google_compute_network.test_vpc, and process after its portion of the plan is applied. Or otherwise, there are plenty of possibilities to create the dependency desired here.

Most simply being adding a depends_on = [google_compute_network.test_vpc] to the data source. 🙂

[rileykarson]

Closed based on #16273 (comment)

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.