-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] Quotes missing around column names and aliases when hubspot__pass_through_all_columns is enabled #128
Comments
Hi @aryan-wisdom , thanks for reporting this issue to us. We were able to reproduce an error locally that when columns were unquoted, the model would not run. We reconfigured our logic to utilize a new macro that handles quoting columns, and were able to resolve this error so that columns were quoted and the model did compile. However, we did have to rename the columns so that dashes were now underscores (because that was the source of the problem). Is this acceptable for your data? Just want to check before we proceed forward with this solution. |
Thanks for taking a look @fivetran-avinash ! That sounds good to me. |
Any update on this @fivetran-avinash ? |
Apologies for the delay in responding @aryan-wisdom ! Yes, if you are okay with this approach, we will move this task to accepted and aim to address it in one of the coming sprints. |
Thanks @fivetran-avinash |
Hi @aryan-wisdom I just wanted to reach out and provide an update that we are planning to address this in the last sprint in the month of October. You can expect us to roll out an update to the HubSpot dbt package then which will address this issue. Thanks for your patience! |
Is there an existing issue for this?
Describe the issue
The
models/stg_hubspot__company.sql
,models/stg_hubspot__contact.sql
andmodels/stg_hubspot__deal.sql
files are not compiling because quotes are not present around column names whenhubspot__pass_through_all_columns
is true.Relevant error log or model output
.... as company_annual_revenue ( -- just pass everything through if extra columns are present, but ensure required columns are present. property_hs_all-deprecated-85213efe-a820-40c1-815f-4c19fed96ad7 as hs_all-deprecated-85213efe-a820-40c1-815f-4c19fed96ad7, from base )
Expected behavior
....
as company_annual_revenue
(
-- just pass everything through if extra columns are present, but ensure required columns are present.
"property_hs_all-deprecated-85213efe-a820-40c1-815f-4c19fed96ad7" as "hs_all-deprecated-85213efe-a820-40c1-815f-4c19fed96ad7",
from base
)
Possible solution
A
for
loop around the code:which applies
fivetran_utils.quote_column
on each column name and alias would help.dbt Project configurations
vars:
hubspot__pass_through_all_columns: true
Package versions
packages:
package: fivetran/fivetran_utils
version: [">=0.4.0", "<0.5.0"]
package: dbt-labs/spark_utils
version: [">=0.3.0", "<0.4.0"]
What database are you using dbt with?
redshift
How are you running this dbt package?
dbt Core™
dbt Version
1.7.7
Additional Context
No response
Are you willing to open a PR to help address this issue?
The text was updated successfully, but these errors were encountered: