Bug/null customer tags

[fivetran-catfritz]

PR Overview

This PR will address the following Issue/Feature:

• Internal ticket

This PR will result in the following new package version:

• 0.13.2 non-breaking since no schema changes.

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Bug Fixes

• Fixed an issue where the shopify__customers model incorrectly displayed NULL values for the customer_tags field for customers without orders. Updated the logic to ensure customer tags are retrieved even if no orders have been placed for that customer.

Under the Hood

• Updated seed data to include customers without orders, verifying that their tags are correctly pulled through.
• Added consistency and integrity tests for the shopify__customers model to ensure accurate handling of customer tags for all customers.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

• dbt run –full-refresh && dbt test
• dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

• The appropriate issue has been linked, tagged, and properly assigned
• All necessary documentation and version upgrades have been applied
• docs were regenerated (unless this PR does not include any code or yml updates)
• BuildKite integration tests are passing
• Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

• Simulated issue with seed data and confirm the update corrects this. See prod vs dev for customer_id '3584045351010'.
  

• Newly added validation tests are:

  • integrity_customers: passes
  • consistency_customers: expected not to pass since this update corrects an issue in prod
    

If you had to summarize this PR in an emoji, which would it be?

💃

[fivetran-joemarkiewicz]

@fivetran-catfritz this looks good to me! I do have one request to update the integrity test. Other than that, this code update looks good to go from my point of view! Thanks!

[fivetran-joemarkiewicz]

Thank you for remembering to make this change!

[fivetran-joemarkiewicz]

This integrity test produces inaccurate results. This will count the number of tags in the source (accurate); however, it's only counting the number of rows for a given customer in the transform (inaccurate since this should always be 1). Therefore, this will result in a failed test if a given customer has multiple tags associated with them.

Instead we can adjust the transformation component of this test to check the items in the array and then count those. This way if a customer has 3 tags in the source, then there should be 3 items in the customer_tags array in the transform.

Suggested change

	select
	customer_id,
	count(*) as transform_customer_tag_count
	from {{ target.schema }}_shopify_dev.shopify__customers
	where customer_tags is not null
	group by 1
	select
	customer_id,
	array_length(split(customer_tags, ',')) as transform_customer_tag_count -- Only BigQuery compatible for the time being
	from zz_dbt_joe_shopify_dev.shopify__customers
	where customer_tags is not null
	group by customer_id, customer_tags

[fivetran-catfritz]

Thank you. I have updated.

[fivetran-catfritz]

Thanks for the review @fivetran-joemarkiewicz. I updated the test, so moving to release review.

[fivetran-catfritz]

Thank you. I have updated.

[fivetran-avinash]

@fivetran-catfritz lgtm!