Fix hashicorp/terraform-provider-google#17388 (only run tests in beta)

[rainshen49]

Roll-forward #10111 with handwritten update tests also limited to beta.

Release Note Template for Downstream PRs (will be copied)

test only

[rainshen49]

Please assign to @ScottSuarez per #10111. It looks like CI is still broken. I'm okay to wait till the diff generator is functional before merging.

[github-actions]

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@trodge, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[ScottSuarez]

/gcbrun

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 15 files changed, 30 insertions(+), 908 deletions(-))
Terraform Beta: Diff ( 15 files changed, 76 insertions(+), 16 deletions(-))

[modular-magician]

Tests analytics

Total tests: 16
Passed tests: 16
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

• firebaseappcheck

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log

[ScottSuarez]

@rainshen49 is there more context for this change. Why are we moving these tests to beta only. I presumed they were in GA by mistake but am interested in more context.

[rainshen49]

Right, I should've explained. The API for google_firebase_app_check_* has a GA version and a beta version https://firebase.google.com/docs/reference/appcheck/rest, but google_firebase_*_app only has a beta API https://firebase.google.com/docs/reference/firebase-management/rest.

1. If the user wants to use the complete flow to create an app and use App Check, they have to use the google-beta provider at least for the google_firebase_*_app.
2. If the user creates their app elsewhere (e.g. console, CLI), they can absolutely use App Check with the GA provider
3. In the examples, we want to show the complete picture, so it can only be run in the beta provider.

[rainshen49]

Friendly ping @ScottSuarez

[ScottSuarez]

Friendly ping @ScottSuarez

Sorry for sleeping on this PR. I've been so busy. Let me do a pass.

[ScottSuarez]

Could you message me on google chat. Lets have a short call on this topic. I think we need to talk this out a bit.

[ScottSuarez]

/gcbrun

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 15 files changed, 30 insertions(+), 908 deletions(-))
google-beta provider: Diff ( 15 files changed, 76 insertions(+), 16 deletions(-))

[modular-magician]

Tests analytics

Total tests: 18
Passed tests: 17
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages

• firebaseappcheck

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccFirebaseAppCheckServiceConfig_firebaseAppCheckServiceConfigUpdate

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccFirebaseAppCheckServiceConfig_firebaseAppCheckServiceConfigUpdate[Debug log]

$\textcolor{red}{\textsf{Tests failed when rerunning REPLAYING mode:}}$
TestAccFirebaseAppCheckServiceConfig_firebaseAppCheckServiceConfigUpdate[Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[rainshen49]

The failure TestAccFirebaseAppCheckServiceConfig_firebaseAppCheckServiceConfigUpdate is unrelated to this PR. It looks like #10206 is flakey. I'll investigate separately.

[ScottSuarez]

Could we investigate or disable this testcase before checkin. Unfortunately if we check in a PR with a failing test that recording will be brought into our CI database.

[rainshen49]

Ack. I will wait for #10281 then.

[rainshen49]

Looks like it's fixed by #10258

[ScottSuarez]

/gcbrun

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 15 files changed, 30 insertions(+), 908 deletions(-))
google-beta provider: Diff ( 15 files changed, 76 insertions(+), 16 deletions(-))

[modular-magician]

Tests analytics

Total tests: 18
Passed tests: 17
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages

• firebaseappcheck

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccFirebaseAppCheckServiceConfig_firebaseAppCheckServiceConfigUpdate

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccFirebaseAppCheckServiceConfig_firebaseAppCheckServiceConfigUpdate[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[ScottSuarez]

Thanks for your patience here :)