TERRA-96 ⁃ astra_private_link fails with json: cannot unmarshal string into Go value of type astra.Errors

[oscar-b]

Hi there,

Please provide the following details with your issue report.

Terraform Version

terraform --version
Terraform v1.3.3
on darwin_arm64
+ provider registry.terraform.io/datastax/astra v2.1.10
+ provider registry.terraform.io/elastic/ec v0.5.0
+ provider registry.terraform.io/elastic/elasticstack v0.4.0
+ provider registry.terraform.io/hashicorp/google v4.47.0
+ provider registry.terraform.io/hashicorp/random v3.4.3

Affected Resource(s)

Please list the resources as a list, for example:

• astra_private_link

Terraform Configuration Files

resource "astra_private_link" "main" {
  allowed_principals = [var.project_id]
  database_id        = astra_database.main.id
  datacenter_id      = local.datacenter_id
}

Debug Output

2023-01-06T09:53:16.935+0100 [ERROR] provider.terraform-provider-astra_v2.1.10: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary="json: cannot unmarshal string into Go value of type astra.Errors" tf_provider_addr=provider @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail= tf_req_id=e6da3053-edf2-6004-b525-54bd6fbf2d0a tf_resource_type=astra_private_link tf_rpc=ApplyResourceChange tf_proto_version=5.3 timestamp=2023-01-06T09:53:16.934+0100

<snip>

Error: json: cannot unmarshal string into Go value of type astra.Errors
│ 
│   with astra_private_link.main,
│   on main.tf line 32, in resource "astra_private_link" "main":
│   32: resource "astra_private_link" "main" {

Expected Behavior

Created a private link.

Actual Behavior

It failed.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

terraform {
  required_version = "= 1.3.3"

  required_providers {
    astra = {
      source  = "datastax/astra"
      version = "2.1.10"
    }
  }
}

variable "astra_token" {
  description = "Astra token (define in terraform.tfvars)"
  type        = string
}

provider "astra" {
  token = var.astra_token
}

resource "astra_database" "main" {
  name           = "bugtest1"
  keyspace       = "bugtest1"
  cloud_provider = "gcp"
  regions        = ["europe-west1"]
}

locals {
  datacenter_id = astra_database.main.datacenters["${astra_database.main.cloud_provider}.${astra_database.main.regions[0]}"]
}

resource "astra_private_link" "main" {
  allowed_principals = ["bugtest1"]
  database_id        = astra_database.main.id
  datacenter_id      = local.datacenter_id
}

Important Factoids

Not that I know of.

References

Maybe #197

┆Issue is synchronized with this Jira Task by Unito
┆Issue Number: TERRA-96
┆Priority: Major

[oscar-b]

Please email/Slack me if the full debug log is needed, but couldn't see anything more interesting there.

[oscar-b]

@phact I don't know if hashicorp/terraform-plugin-go#237 could be related, possibly solving this? Seems like it might be an error in the response not being handled properly?

[oscar-b]

Re-creating the http rest call manually, with an invalid gcp project id, returns error 400 with "invalid allowed principal".

My underlying issue might be that TF doesn't have the GCP id ready when this code runs, trying to create the private link with an empty ID, but the issue remains that the endpoint returns a simple string and I assume that the provider expects a json object.

Update: The GCP project ID should be there when the module runs, it has to be some other error message being returned that I can't see :|

[oscar-b]

I tried running this rest call on the actual prod db and prod project now and got an error 500 back:

"error occurred trying to add principals in Gcp"

I'm getting the same error when I try to add it through the AstraDB admin UI.

[emerkle826]

@oscar-b the error you get

Error: json: cannot unmarshal string into Go value of type astra.Errors

is because the Go client used by this provider to interact with the Astra DevOps API has an incorrect response mapping, as you have painfully figured out. Unfortunately, when this happens, I have to do basically what you did, to figure out which DevOps API call triggered the response, then fix the Go client to have the correct response definition. It usually means the response was not successful, and was expected to be a properly formatted JSON object, but the response is actually just plain text, hence the failure to unmarshal.

So there are generally 2 issues when you see this. The first is that the DevOps API response doesn't unmarshal into JSON, which hides the real issue. Once I fix the unmarhsaling, then I can see what the real issue is and I can figure out how to resolve that.

I'm hoping to get some time today to take a look at this one.

[oscar-b]

@emerkle826 just fyi, I opened a support ticket for this as well as the same thing happens in the Astra DB UI, I can't add the GCP project in the private endpoint configuration.

[oscar-b]

@emerkle826 Sorry, didn't update this ticket, but the underlying issue was solved (something with the Google side of things, support opened a ticket with GCP to solve it). But I guess the issue here still stands, that the provider fails on certain error conditions from the API that isn't proper JSON.

[emerkle826]

@oscar-b Thanks for the update. Sorry I haven't replied much lately. I've been swamped with other things. I'll leave this ticket open to address the Error issue as you suggest.

[emerkle826]

@oscar-b

I tried running this rest call on the actual prod db and prod project now and got an error 500 back:

"error occurred trying to add principals in Gcp"

I'm getting the same error when I try to add it through the AstraDB admin UI.

Was the REST endpoint you used

v2/organizations/clusters/<uuid>/datacenters/<uuid>-1/allowed-principals

or

v2/organizations/clusters/<uuid>/datacenters/<uuid>-1/private-link

Both appear to do the same thing in DevOps API, but the Terraform code currently only uses the private-link version, not the allowed-principals version

[emerkle826]

The marshaling error should be fixed in v2.1.11 as I fixed both places where private-link allowed principals are handled in the client.