Create better errors when calling Cloud Service APIs incorrectly

[lazarillo]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Description

There are several resource and data objects that have elements with confusing names. This is inevitable simply due to the scale of the challenge. Therefore, I find myself accidentally calling <my-resource>.id when I need to be calling <my-resource>.project_id, or something similar. (There are examples below.)

The problem is that the error is simply a response from the cloud provider REST API which is, nearly always, uninformative.

It would be great if the errors could be improved in the cases that are known. This might be too hard to do everywhere, but I think some help is better than none.

Some specific examples I've come across using GCP (Google):

1. google_project object:

data "google_project" "self" {
  project_id = var.project
}

Most every resource that will then depend upon project should reference the above object as data.google_project.self.project_id, and not as data.google_project.self.id (note project_id vs id).

I think it is entirely rational to understand how / why a user would make that mistake. However, the impact of that mistake is that a path such as projects/<my-project-number>/secrets/<my-secret-name> becomes projects/projects/<my-project-name>/secrets/<my-secret-name>. This is because project_id is the number of the project, whereas id is projects/<my-project-name>.

This gives a completely unintelligible Google REST API error similar to the one found here. That error was for a different reason, but what I'm describing gives the same nearly useless answer. (In short, it just gives an HTTP 404 return without any useful information except for that mis-constructed URI.)

This seems somewhat easy to catch, because we can just look for a set of keywords in the URI path and ensure that they only occur once. For instance, a regex to ensure that any one of ["projects", "instances", "secrets", ...] only occurs once in the returned URI.

2. google_secret_manager_secret object:

data "google_secret_manager_secret" "oauth" {
  project   = var.project
  secret_id = var.oauth_secret_name
}

In this case, if you reference this secret with either data.google_secret_manager_secret.oauth.id or data.google_secret_manager_secret.oauth.name instead of the correct data.google_secret_manager_secret.oauth.secret_id, then you will get a similar problem. Again, here id and name are of the format projects/{{project}}/secrets/{{secret_id}}, whereas all you really want is the name, the {{secret_id}}.

New or Affected Resource(s)

• google_*

I haven't verified that all google resources are affected, but I would suspect that nearly all of them are. Any time a resource needs to reference another reason and that resource has multiple name-like attributes on its object, this problem will occur.

Potential Terraform Configuration

It's not a TF config, but the error could say something like:

(First give the error from the cloud provider.) Then:
The URL provided, "projects/projects/<project-number>/..." appears to be misconfigured. Did you perhaps provide a full URI when only the resource name was needed?

References

No response