Bridge gap between Terraform and API for AlloyDB

[GauravJain21]

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Generated Terraform providers, and ran make test and make lint in the generated providers to ensure it passes unit and linter tests.
• Ran relevant acceptance tests using my own Google Cloud project and credentials (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read Write release notes before writing my release note below.

This PR contains multiple changes for AlloyDB cluster, instance, and backup. It adds all the attributes which were missing in Terraform but were present in REST API for these resources and the corresponding tests.

Release Note Template for Downstream PRs (will be copied)

alloydb: added additional fields to `google_alloydb_instance` and `google_alloydb_backup`

[modular-magician]

Hello! I am a robot. It looks like you are a: Community Contributor Googler Core Contributor. Tests will run automatically.

@slevenick, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 36 insertions(+))
Terraform Beta: Diff ( 2 files changed, 36 insertions(+))
TF Conversion: Diff ( 1 file changed, 10 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_alloydb_backup (13 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_backup" "primary" {
  display_name = # value needed
}

[modular-magician]

Tests analytics

Total tests: 2990
Passed tests 2693
Skipped tests: 296
Affected tests: 1

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataprocClusterIamPolicy

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDataprocClusterIamPolicy[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[slevenick]

Can you add a test that sets this field? You can just add the field to an existing test

[GauravJain21]

sure.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 36 insertions(+))
Terraform Beta: Diff ( 2 files changed, 36 insertions(+))
TF Conversion: Diff ( 1 file changed, 10 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_alloydb_backup (13 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_backup" "primary" {
  display_name = # value needed
}

[modular-magician]

Tests analytics

Total tests: 3008
Passed tests 2711
Skipped tests: 296
Affected tests: 1

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccSpannerDatabaseIamPolicy

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccSpannerDatabaseIamPolicy[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 36 insertions(+))
Terraform Beta: Diff ( 2 files changed, 36 insertions(+))
TF Conversion: Diff ( 1 file changed, 10 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_alloydb_backup (13 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_backup" "primary" {
  display_name = # value needed
}

[modular-magician]

Tests analytics

Total tests: 3011
Passed tests 2714
Skipped tests: 296
Affected tests: 1

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataSourceGoogleServiceAccountIdToken_impersonation

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDataSourceGoogleServiceAccountIdToken_impersonation[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 36 insertions(+))
Terraform Beta: Diff ( 2 files changed, 36 insertions(+))
TF Conversion: Diff ( 1 file changed, 10 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_alloydb_backup (13 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_backup" "primary" {
  display_name = # value needed
}

[modular-magician]

Tests analytics

Total tests: 3018
Passed tests 2721
Skipped tests: 297
Affected tests: 0

$\textcolor{green}{\textsf{All tests passed in REPLAYING mode.}}$
View the build log

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 322 insertions(+), 35 deletions(-))
Terraform Beta: Diff ( 2 files changed, 322 insertions(+), 35 deletions(-))
TF Conversion: Diff ( 1 file changed, 41 insertions(+), 4 deletions(-))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_alloydb_backup (13 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_backup" "primary" {
  annotations  = # value needed
  display_name = # value needed
  type         = # value needed
}

[modular-magician]

Tests analytics

Total tests: 3023
Passed tests 2720
Skipped tests: 297
Affected tests: 6

Action taken

Found 6 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbBackup_alloydbBackupFullExample|TestAccAlloydbBackup_createBackupWithMandatoryFields|TestAccAlloydbBackup_alloydbBackupBasicExample|TestAccAlloydbBackup_update|TestAccAlloydbCluster_restore|TestAccAlloydbBackup_usingCMEK

Get to know how VCR tests work

[modular-magician]

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbBackup_alloydbBackupFullExample[Error message] [Debug log]
TestAccAlloydbBackup_createBackupWithMandatoryFields[Error message] [Debug log]
TestAccAlloydbBackup_alloydbBackupBasicExample[Error message] [Debug log]
TestAccAlloydbBackup_update[Error message] [Debug log]
TestAccAlloydbCluster_restore[Error message] [Debug log]
TestAccAlloydbBackup_usingCMEK[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 322 insertions(+), 35 deletions(-))
Terraform Beta: Diff ( 2 files changed, 322 insertions(+), 35 deletions(-))
TF Conversion: Diff ( 1 file changed, 41 insertions(+), 4 deletions(-))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_alloydb_backup (13 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_backup" "primary" {
  annotations  = # value needed
  display_name = # value needed
  type         = # value needed
}

[slevenick]

Hey Sam, These tests are working fine on my machine. When I look at the debug logs, I am seeing transient error for the networking service (not AlloyDB). Any idea what to do?

Error waiting for Create Service Networking Connection: Error code 9, message: encountered some transient internal error, please try again later

I'm seeing some recent service networking changes that might be causing this. I'll ask around

[slevenick]

I think this is being tracked here and is unrelated to your changes: #8904

[GauravJain21]

Thank you so much for the update, Sam! :D

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 8 files changed, 637 insertions(+), 35 deletions(-))
Terraform Beta: Diff ( 8 files changed, 637 insertions(+), 35 deletions(-))
TF Conversion: Diff ( 3 files changed, 130 insertions(+), 4 deletions(-))
TF OiCS: Diff ( 1 file changed, 1 insertion(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_alloydb_backup (14 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_backup" "primary" {
  annotations  = # value needed
  display_name = # value needed
}

Resource: google_alloydb_cluster (62 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_cluster" "primary" {
  annotations = # value needed
  etag        = # value needed
}

[modular-magician]

Tests analytics

Total tests: 3075
Passed tests 2774
Skipped tests: 299
Affected tests: 2

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAlloydbInstance_createInstanceWithMaximumFields|TestAccAlloydbInstance_updateDatabaseFlagInPrimaryInstance

Get to know how VCR tests work

[modular-magician]

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccAlloydbInstance_createInstanceWithMaximumFields[Error message] [Debug log]
TestAccAlloydbInstance_updateDatabaseFlagInPrimaryInstance[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

[GauravJain21]

Hey Sam,

As these two are passing on my local machine and the debug log suggest that it is an issue with the service networking, I have commented out these two tests for now. I will re-enable them once this issue from service networking team is fixed.

I have created b/301367801 to track it.

Can you please approve and merge the PR?

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 8 files changed, 641 insertions(+), 39 deletions(-))
Terraform Beta: Diff ( 8 files changed, 641 insertions(+), 39 deletions(-))
TF Conversion: Diff ( 3 files changed, 130 insertions(+), 4 deletions(-))
TF OiCS: Diff ( 1 file changed, 1 insertion(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_alloydb_backup (14 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_backup" "primary" {
  annotations  = # value needed
  display_name = # value needed
}

Resource: google_alloydb_cluster (59 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_cluster" "primary" {
  annotations = # value needed
  etag        = # value needed
}

Resource: google_alloydb_instance (20 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_alloydb_instance" "primary" {
  query_insights_config {
    query_plans_per_minute  = # value needed
    query_string_length     = # value needed
    record_application_tags = # value needed
    record_client_address   = # value needed
  }
}

[modular-magician]

Tests analytics

Total tests: 3073
Passed tests 2770
Skipped tests: 299
Affected tests: 4

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccBigQueryDataTable_bigtable|TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample|TestAccBigtableAppProfile_bigtableAppProfileSingleclusterExample|TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccBigQueryDataTable_bigtable[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileSingleclusterExample[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[slevenick]

One small request, can you change the changelog to be formatted in our release notes format: https://googlecloudplatform.github.io/magic-modules/contribute/release-notes/

Then I'll approve and merge

[GauravJain21]

done.

[slevenick]

done.

See https://googlecloudplatform.github.io/magic-modules/contribute/release-notes/#type-specific-guidelines-and-examples

And existing release notes: https://github.com/hashicorp/terraform-provider-google/blob/main/CHANGELOG.md

It should look like:

alloydb: added `x` field(s) to `y` resource

[GauravJain21]

updated as per suggested. please check and let me know if it looks good.

[slevenick]

Can you add these fields to tests as suggested by #8781 (comment)

[GauravJain21]

These fields are part of the tests which I have commented out. I will enable them once service networking issue is resolved and they will cover these fields.

[GauravJain21]

Hey Sam,
Can you please merge it? I don't have the permission to merge.