feat: Upgrade tag SDK #3126

sfc-gh-jmichalak · 2024-10-10T13:53:29Z

add missing object types (snowflake_tag_association does not fully support objects #1496, Snowflake Tag Resource Should Support all Object Types #1862, Add support for tagging Snowflake Alert #2598)
adjust show options
add ifExists to tag queries
add missing masking policy note about v1 rc
add integration tests for setting, getting, and unsetting tags, except:
- failover group - business critical edition needed
- alert, password policy, network policy - missing tag handling in SDK
- git repo, iceberg table, budget, classification, replication group - objects not implemented in the sdk

Test Plan

integration tests

unit tests

References

https://docs.snowflake.com/en/sql-reference/sql/create-tag
https://docs.snowflake.com/en/user-guide/object-tagging
https://docs.snowflake.com/en/sql-reference/functions/system_get_tag

TODO

change GetTag to not fail on null values
move tag assignments integration tests to a common place
rework tag, tag_association resource
remove tag_masking_policy_association_resource (tag_association will be used instead)

github-actions · 2024-10-10T14:42:53Z

Integration tests failure for 424b47af067b90540fd2c61a1d0ae3c058ca4fea

github-actions · 2024-10-11T14:03:58Z

Integration tests cancelled for 651c4e36cd88317fcbd19030f713ef50522c4ad0

github-actions · 2024-10-11T14:03:59Z

Integration tests cancelled for acb0981ae15561392c73f553800c16f1a293794b

github-actions · 2024-10-11T16:16:17Z

Integration tests failure for 47b9ac5dec645cea701af0885eb5b790ea5a0117

sfc-gh-jcieslak · 2024-10-14T09:36:54Z

pkg/resources/tag.go

@@ -47,6 +47,7 @@ var tagSchema = map[string]*schema.Schema{
 FullyQualifiedNameAttributeName: schemas.FullyQualifiedNameSchema,
 }

+// TODO: remove after rework of external table, materialized view, stage and table


Maybe let's create a ticket for it, assign it to this todo, and set the dependencies of listed reworks on this ticket. Otherwise there's a big chance we'll just forget about it.

Or just add this point to each resource dedicated task (so link here three issues)

sfc-gh-jcieslak · 2024-10-14T09:41:41Z

pkg/sdk/tag_association_validations.go

 ObjectTypeAlert,
+ ObjectTypeBudget,
+ ObjectTypeClassification,
 ObjectTypeExternalFunction,


We could create a test that would go through all the object types to see which ones are valid ones, I just checked that, e.g. dynamic tables can also be used (with table data type ofc).

+1 for such a test

It's related to #3126 (comment) - I can do this in the next PR.

sfc-gh-asawicki · 2024-10-14T10:10:33Z

pkg/sdk/object_types.go

@@ -25,6 +25,7 @@ const (
 ObjectTypeNetworkPolicy ObjectType = "NETWORK POLICY"
 ObjectTypePasswordPolicy ObjectType = "PASSWORD POLICY"
 ObjectTypeSessionPolicy ObjectType = "SESSION POLICY"
+ ObjectTypePrivacyPolicy ObjectType = "PRIVACY POLICY"


sdk.ObjectType is used in more places than just tagging, are we sure, that we do not break other resources by adding it (e.g. grants)?

sfc-gh-asawicki · 2024-10-14T10:13:42Z

pkg/sdk/system_functions.go

+// SQL compilation error: Invalid value VIEW for argument OBJECT_TYPE. Please use object type TABLE for all kinds of table-like objects.
+// TODO [SNOW-1022645]: discuss how we handle situation like this in the SDK
+func normalizeGetTagObjectType(objectType ObjectType) ObjectType {
+ if slices.Contains([]ObjectType{ObjectTypeView, ObjectTypeMaterializedView, ObjectTypeExternalTable}, objectType) {


what about dynamic table?

Initially, I missed dynamic tables because they are missing from https://docs.snowflake.com/en/user-guide/object-tagging. Managing dynamic tables is not implemented in the SDK. I just checked it manually, and it turns out you can use GetTag with dynamic table as an argument.

sfc-gh-asawicki · 2024-10-14T10:16:56Z

pkg/sdk/testint/applications_integration_test.go

- // tv, err := client.SystemFunctions.GetTag(ctx, tagTest.ID(), id, sdk.ObjectTypeApplication)
- // require.NoError(t, err)
- // assert.Equal(t, "v1", tv)
+ // TODO: adjust after this is fixed on Snowflake side


do we have any trackable issue? let's create one and like it here

sfc-gh-asawicki · 2024-10-14T10:17:57Z

pkg/sdk/testint/databases_integration_test.go

+ tagTest.ID(),
+ },
+ })
+ require.NoError(t, err)


we can check if the GetTag return no tag after.

sfc-gh-asawicki · 2024-10-14T10:20:39Z

pkg/sdk/testint/external_functions_integration_test.go

@@ -194,6 +195,33 @@ func TestInt_ExternalFunctions(t *testing.T) {
 assertExternalFunction(t, externalFunction.ID(), true)
 })

+ t.Run("alter external function: set and unset tags", func(t *testing.T) {


Maybe instead adding such a func to each object, we could extract one integration test with parametrized setup and run it for all objects instead? (I know that currently we had setting/unsetting tags in each object but with the rework, we can pick more centralized approach)

I can do it in the next PR.

sfc-gh-asawicki · 2024-10-14T10:21:23Z

pkg/sdk/testint/materialized_views_gen_integration_test.go

@@ -326,7 +326,7 @@ func TestInt_MaterializedViews(t *testing.T) {
 err := client.Views.Alter(ctx, alterRequestSetTags)
 require.NoError(t, err)

- returnedTagValue, err := client.SystemFunctions.GetTag(ctx, tag.ID(), id, sdk.ObjectTypeTable)
+ returnedTagValue, err := client.SystemFunctions.GetTag(ctx, tag.ID(), id, sdk.ObjectTypeMaterializedView)


why in some tests we check for err nil and in some we also validate the tag's value?

That's because when a tag is not set, GetTag returns an error. I'd like to change this, so that nil is returned as a tag value in the next PR.

sfc-gh-asawicki · 2024-10-14T10:22:02Z

pkg/sdk/testint/schemas_integration_test.go

-
- s, err := client.Schemas.ShowByID(ctx, schemaID)
- require.NoError(t, err)
+ schema, cleanupSchema := testClientHelper().Schema.CreateSchema(t)


+1 za posprzatanie setupu

sfc-gh-jmichalak added 2 commits October 10, 2024 15:48

Tag sdk

41c3ab4

Improve tag sdk

424b47a

sfc-gh-jmichalak added 3 commits October 11, 2024 15:48

Cleanup

acb0981

Cleanup

651c4e3

Cleanup

47b9ac5

sfc-gh-jmichalak requested review from sfc-gh-asawicki and sfc-gh-jcieslak October 11, 2024 14:03

sfc-gh-jmichalak marked this pull request as ready for review October 11, 2024 14:11

sfc-gh-jcieslak reviewed Oct 14, 2024

View reviewed changes

sfc-gh-asawicki requested changes Oct 14, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Upgrade tag SDK #3126

feat: Upgrade tag SDK #3126

sfc-gh-jmichalak commented Oct 10, 2024 •

edited

Loading

github-actions bot commented Oct 10, 2024

github-actions bot commented Oct 11, 2024

github-actions bot commented Oct 11, 2024

github-actions bot commented Oct 11, 2024

sfc-gh-jcieslak Oct 14, 2024

sfc-gh-asawicki Oct 14, 2024

sfc-gh-jcieslak Oct 14, 2024

sfc-gh-asawicki Oct 14, 2024

sfc-gh-jmichalak Oct 15, 2024

sfc-gh-asawicki Oct 14, 2024

sfc-gh-asawicki Oct 14, 2024

sfc-gh-jmichalak Oct 15, 2024

sfc-gh-asawicki Oct 14, 2024

sfc-gh-asawicki Oct 14, 2024

sfc-gh-asawicki Oct 14, 2024

sfc-gh-jmichalak Oct 15, 2024

sfc-gh-asawicki Oct 14, 2024

sfc-gh-jmichalak Oct 15, 2024

sfc-gh-asawicki Oct 14, 2024

feat: Upgrade tag SDK #3126

Are you sure you want to change the base?

feat: Upgrade tag SDK #3126

Conversation

sfc-gh-jmichalak commented Oct 10, 2024 • edited Loading

Test Plan

References

TODO

github-actions bot commented Oct 10, 2024

github-actions bot commented Oct 11, 2024

github-actions bot commented Oct 11, 2024

github-actions bot commented Oct 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfc-gh-jmichalak commented Oct 10, 2024 •

edited

Loading