Plan fails when using google_project_service data source with disabled APIs - API not found

[d-costa]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.6.3
on linux_amd64
+ provider registry.terraform.io/hashicorp/google v5.5.0

Affected Resource(s)

• google_project_service (Data Source)

Terraform Configuration Files

terraform {
  required_version = "1.6.3"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "5.5.0"
    }
  }
}

provider "google" {
  project = var.project_id
}

variable "project_id" {
  type = string
}

data "google_project_service" "compute" {
  service = "compute.googleapis.com"
}

data "google_project_service" "dataproc" {
  service = "dataproc.googleapis.com"
}

Debug Output

Panic Output

Expected Behavior

data.google_project_service.dataproc: Reading...
data.google_project_service.compute: Reading...
data.google_project_service.compute: Read complete after 5s [id=PROJECT_ID/compute.googleapis.com]
data.google_project_service.dataproc: Read complete after 5s

Changes to Outputs:
  + compute_enabled  = true
  + dataproc_enabled = false

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

compute_enabled = true
dataproc_enabled = false

Actual Behavior

data.google_project_service.compute: Reading...
data.google_project_service.dataproc: Reading...
data.google_project_service.compute: Read complete after 5s [id=PROJECT_ID/compute.googleapis.com]

Changes to Outputs:
  + compute_enabled = true

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
╷
│ Error: PROJECT_ID/dataproc.googleapis.com not found
│ 
│   with data.google_project_service.dataproc,
│   on main.tf line 24, in data "google_project_service" "dataproc":
│   24: data "google_project_service" "dataproc" {
│ 
╵

Steps to Reproduce

1. terraform apply

Important Factoids

• This configuration works with google provider version 4.84.0
• For context, we need this data source to conditionally enable APIs based on the status of others

References

• Data source created here Adding google_project_service datasource #13434
• Modified here: 3e00cfb

I believe this change was not intended, because it prevents the requested use case.

[edwardmedia]

@d-costa you are right. The behavior has been updated on most data sources by this PR in v5.0.0

The original issue was #12873

Please go through the links above to see if that make sense.

[d-costa]

Thank you for your response @edwardmedia

I understand the objective of the PR you mentioned, and I am not contesting its benefits, but this prevents the use case that even the documentation describes. We cannot check if the service is enabled if we can only use it with enabled services:

Verify the API service for the Google Cloud Platform project to see if it is enabled or not.
(https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/google_project_service)

[edwardmedia]

@trodge what do you think about this?

[trodge]

It looks like this was an intended change for all data sources in 5.0.0 based on #12873.

That being said, it's possible the behavior is not appropriate for this data source in particular, since the main use case for data_source_google_project_service would be determining whether a service is enabled.

@NickElliot might have more context having made the change in GoogleCloudPlatform/magic-modules#8858.

[NickElliot]

I think it would make sense to remove it for this data source as an exception, as it is the intended usage case for this specific data source.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.