Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add condition and accessPolicyVersion to BQ dataset access #12475

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
0edec7b
Add condition field to bigquery dataset access
obada-ab Nov 27, 2024
3677133
Merge branch 'GoogleCloudPlatform:main' into dataset-acl-condition
obada-ab Dec 2, 2024
1daccab
Merge branch 'GoogleCloudPlatform:main' into dataset-acl-condition
obada-ab Dec 2, 2024
1f52ebe
Fix bq dataset access with conditions
obada-ab Dec 3, 2024
2ef7a14
Merge branch 'GoogleCloudPlatform:main' into dataset-acl-condition
obada-ab Dec 3, 2024
0a7fcd3
Merge branch 'dataset-acl-condition' of https://github.com/obada-ab/m…
obada-ab Dec 4, 2024
6903e71
Merge branch 'GoogleCloudPlatform:main' into dataset-acl-condition
obada-ab Dec 4, 2024
d91c15f
Add debug message for BQ dataset access entries
obada-ab Dec 4, 2024
3547901
Merge branch 'dataset-acl-condition' of https://github.com/obada-ab/m…
obada-ab Dec 4, 2024
3c53b5e
Remove unused bq dataset encoder
obada-ab Dec 4, 2024
1ac1ac4
Fix testAccCheckBigQueryDatasetAccess
obada-ab Dec 4, 2024
40a7e55
Merge branch 'GoogleCloudPlatform:main' into dataset-acl-condition
obada-ab Dec 6, 2024
0334398
Remove accessPolicyVersion field
obada-ab Dec 6, 2024
bee7e9d
Remove accessPolicyVersion from BQ dataset URLs
obada-ab Dec 6, 2024
2bad050
Merge branch 'GoogleCloudPlatform:main' into dataset-acl-condition
obada-ab Dec 10, 2024
e3f8799
Fix BQ Dataset/DatasetAccess http URLs
obada-ab Dec 10, 2024
0ff8ccc
Merge branch 'GoogleCloudPlatform:main' into dataset-acl-condition
obada-ab Dec 12, 2024
3e2cf62
Add condition to BQ Dataset
obada-ab Dec 12, 2024
dbdc91f
Fix create url for BQ dataset
obada-ab Dec 13, 2024
a7af1b3
Merge branch 'GoogleCloudPlatform:main' into dataset-acl-condition
obada-ab Dec 13, 2024
82b6996
Fix BQ dataset access test
obada-ab Dec 13, 2024
169be9d
Move BQ external AWS dataset test to resource_bigquery_dataset_test
obada-ab Dec 13, 2024
3705c08
Fix BQ dataset tests
obada-ab Dec 13, 2024
bf98483
Merge branch 'GoogleCloudPlatform:main' into dataset-acl-condition
obada-ab Dec 13, 2024
af61577
Add cai_base_url to BQ dataset
obada-ab Dec 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion mmv1/products/bigquery/Dataset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ docs:
The API does accept both formats but it will always return the legacy format which results in Terraform
showing permanent diff on each plan and apply operation.
base_url: 'projects/{{project}}/datasets'
self_link: 'projects/{{project}}/datasets/{{dataset_id}}'
id_format: 'projects/{{project}}/datasets/{{dataset_id}}'
self_link: 'projects/{{project}}/datasets/{{dataset_id}}?accessPolicyVersion=3'
zli82016 marked this conversation as resolved.
Show resolved Hide resolved
has_self_link: true
delete_url: 'projects/{{project}}/datasets/{{dataset_id}}?deleteContents={{delete_contents_on_destroy}}'
zli82016 marked this conversation as resolved.
Show resolved Hide resolved
import_format:
Expand Down
28 changes: 27 additions & 1 deletion mmv1/products/bigquery/DatasetAccess.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ docs:
The API does accept both formats but it will always return the legacy format which results in Terraform
showing permanent diff on each plan and apply operation.
base_url: 'projects/{{project}}/datasets/{{dataset_id}}'
self_link: 'projects/{{project}}/datasets/{{dataset_id}}'
self_link: 'projects/{{project}}/datasets/{{dataset_id}}?accessPolicyVersion=3'
zli82016 marked this conversation as resolved.
Show resolved Hide resolved
create_verb: 'PATCH'
delete_verb: 'PATCH'
immutable: true
Expand Down Expand Up @@ -301,3 +301,29 @@ properties:
A-Z), numbers (0-9), or underscores (_). The maximum length
is 256 characters.
required: true
- name: 'condition'
type: NestedObject
description: |
Condition for the binding. If CEL expression in this field is true, this
access binding will be considered.
properties:
- name: expression
type: String
required: true
description: |
Textual representation of an expression in Common Expression Language syntax.
- name: title
type: String
description: |
Title for the expression, i.e. a short string describing its purpose.
This can be used e.g. in UIs which allow to enter the expression.
- name: description
type: String
description: |
Description of the expression. This is a longer text which describes the expression,
e.g. when hovered over it in a UI.
- name: location
type: String
description: |
String indicating the location of the expression for error reporting, e.g. a file
name and a position in the file.
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,40 @@ func TestAccBigQueryDatasetAccess_userByEmailWithMixedCase(t *testing.T) {
})
}

func TestAccBigQueryDatasetAccess_withCondition(t *testing.T) {
t.Parallel()

datasetID := fmt.Sprintf("tf_test_%s", acctest.RandString(t, 10))
saID := fmt.Sprintf("tf-test-%s", acctest.RandString(t, 10))

expected := map[string]interface{}{
"condition": map[string]interface{}{
"description": "Request after midnight of 2019-12-31",
"expression": "request.time > timestamp(\"2020-01-01T00:00:00Z\")",
"location": "any.file.anywhere",
"title": "test-condition",
},
"role": "OWNER",
"userByEmail": fmt.Sprintf("%s@%s.iam.gserviceaccount.com", saID, envvar.GetTestProjectFromEnv()),
}

acctest.VcrTest(t, resource.TestCase{
PreCheck: func() { acctest.AccTestPreCheck(t) },
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories(t),
Steps: []resource.TestStep{
{
Config: testAccBigQueryDatasetAccess_withCondition(datasetID, saID),
Check: testAccCheckBigQueryDatasetAccessPresent(t, "google_bigquery_dataset.dataset", expected),
},
{
// Destroy step instead of CheckDestroy so we can check the access is removed without deleting the dataset
Config: testAccBigQueryDatasetAccess_destroy(datasetID, "dataset"),
Check: testAccCheckBigQueryDatasetAccessAbsent(t, "google_bigquery_dataset.dataset", expected),
},
},
})
}

func TestAccBigQueryDatasetAccess_groupByEmailWithMixedCase(t *testing.T) {
t.Parallel()

Expand Down Expand Up @@ -575,3 +609,27 @@ resource "google_bigquery_dataset" "dataset" {
}
`, accessType, email, datasetID)
}

func testAccBigQueryDatasetAccess_withCondition(datasetID, saID string) string {
return fmt.Sprintf(`
resource "google_bigquery_dataset_access" "withCondition" {
dataset_id = google_bigquery_dataset.dataset.dataset_id
role = "OWNER"
user_by_email = google_service_account.bqowner.email
condition {
title = "test-condition"
description = "Request after midnight of 2019-12-31"
expression = "request.time > timestamp(\"2020-01-01T00:00:00Z\")"
location = "any.file.anywhere"
}
}

resource "google_bigquery_dataset" "dataset" {
dataset_id = "%s"
}

resource "google_service_account" "bqowner" {
account_id = "%s"
}
`, datasetID, saID)
}
Loading