PR #109 includes the following updates:
- As announced in the v0.9.2 August 2023 release, the
full_statement_version=falseparameter has been fully deprecated from thesfdc_formula_viewmacro. Please remove any references to this parameter to avoid errors.- Additionally, the
reserved_table_nameandfields_to_includeparameters have also been deprecated and should be removed.
- Additionally, the
- The following macros have also been removed:
sfdc_current_formula_valuessfdc_formula_pivotsfdc_formula_refactorsfdc_formula_view_fieldssfdc_formula_view_sqlsfdc_get_formula_column_valuessfdc_star_exact
- Included auto-releaser GitHub Actions workflow to automate future releases.
- Updated maintainer pull request template.
PR #101 includes the following updates:
- Fivetran has deprecated support for the
full_statement_version=false. We've removed mention of this obsolete method in the README.
- Users will not see functional differences from v0.9.2. However, due to changes in the Fivetran Salesforce connector, users still utilizing the
full_statement_version=falseshould expect formula field values to be null.
PR #96 includes the following updates:
- Fivetran will be deprecating support for the
full_statement_version=falseconfig by October 1, 2023. We've highlighted this change to users of the package in the README.
- Users still utilizing the
full_statement_version=falseconfig will receive a log message in theirdbt runindicating deprecation of support for those options.
- Databricks users faced a syntax error resulting from the
sfdc_formula_view_sqlmacro not correctly compiling. This update adds the Databricks warehouse to a conditional in the macro that allows the sql to correctly compile. (PR #94) (PR #92) - Postgres users faced a syntax error resulting from the
sfdc_formula_view_sqlmacro not correctly compiling. This update adds the Postgres warehouse to a conditional in the macro that allows the sql to correctly compile. (PR #94)
- Databricks compatibility! (#89)
- Clarify Step 5 of the README that
sfdc_exclude_formulasworks only with Options 2 and 3. (#88)
- Renamed macro filename from
sfdc_current_formula_fieldstosfdc_current_formula_valuesto be consistent with its macro name. (#86) - Incorporated the new
fivetran_utils.drop_schemas_automationmacro into the end of each Buildkite integration test job. (#82) - Updated the pull request templates. (#82)
- Updated the README for easier use and navigation of the package (PR #78).
(PR #77) includes the following changes:
- In v0.8.0, the
using_quoted_identifiersargument in thesfdc_formula_view()macro was erroneously removed, making its default value ofFalseimmutable. It has been reintroduced. using_quoted_identifiersnow uses warehouse-specific quoting syntax.
PR #64 includes the following breaking changes:
- The default behavior of the
sfdc_formula_viewmacro now has thefull_statement_versionequal totrue. This means, by default the macro will search for thefivetran_formula_modelsource table (full SQL statement for each object) opposed to thefivetran_formulasource table (individual formulas).- Please be sure to update your macros that intend to reference the
fivetran_formulasource table accordingly.
- Please be sure to update your macros that intend to reference the
- Dispatch update for dbt-utils to dbt-core cross-db macros migration. Specifically
{{ dbt_utils.<macro> }}have been updated to{{ dbt.<macro> }}for the below macros:any_valuebool_orcast_bool_to_textconcatdate_truncdateadddatediffescape_single_quotesexcepthashintersectlast_daylengthlistaggpositionreplacerightsafe_castsplit_partstring_literaltype_biginttype_floattype_inttype_numerictype_stringtype_timestamparray_appendarray_concatarray_construct
- For
current_timestampandcurrent_timestamp_in_utcmacros, the dispatch AND the macro names have been updated to the below, respectively:dbt.current_timestamp_backcompatdbt.current_timestamp_in_utc_backcompat
- Dependencies on
fivetran/fivetran_utilshave been upgraded, previously[">=0.3.0", "<0.4.0"]now[">=0.4.0", "<0.5.0"].
- Incorporated a new boolean argument (
using_quoted_identifiers) to thesfdc_formula_viewmacro. This argument allows users to quote the compiled sql generated by the macro. This is especially necessary when a users warehouse has case sensitivity enabled. If a user has a warehouse with case sensitivity, they will likely see the macro fail by default. If this is the case, you will want to set theusing_quoted_identifiersargument totrue(the argument isfalseby default). (#72) - Updated documentation within the README for easier understanding of the macros and how to use them in a dbt project. (#70)
- Added BuildKite integration tests. (#72)
- Added the
docsfolder for dbt docs hosting on GitHub Pages. (#72) - Upgraded dbt-expectations dependency within the package integration_tests to the
[">=0.8.0", "<0.9.0"]range. (#72)
- README edits for easier navigation and understanding of how to use the solution. (#61)
- The
dbt_utils.get_column_valuesmacro for the full_statement_version of the package was not properly filtering for only the relevant source table records within thefivetran_formula_modelsource table. A where clause has been added to remedy this issue. (#61)
- Addition of the
full_statement_versionargument within thesfdc_formula_viewmacro. This argument is a boolean and will allow users to either leverage the default (false) version of the macro to pull individual formulas from thefivetran_formulatable to be produced in the materialized view, or (true) leverage thefivetran_formula_modelsource to generate the entire sql statement that will be materialized in warehouse. (#55)
- Inclusion of the
materializationargument to thesfdc_formula_viewmacro. The argument isviewby default. However, if a user wishes to override the materialization then they may do so with this argument.
- The string
__tableis now appended to the table alias to match with the data provided within the connector. To ensure consistency across the board, thereserved_table_nameargument is now ignored in place of the{{source_table}}__tableformat. (#51)- The
reserved_table_nameargument is still accepted, but it will now be ignored for the time being. This will be removed in the next breaking release.
- The
- Adjusted the conditional within the
sfdc_formula_viewsmacro to reference the properly namedcurrent_formula_fieldsvariable opposed to the incorrectold_formula_fieldsvariable that is not longer referenced. (#46)
- Resolving dependency issues for use of
fields_to_includefeature ofsfdc_formula_viewmacro. Previously if the list offields_to_includeincluded formulas that referenced other formulas excluded fromfields_to_include, an error was served for the excluded formulas. (#42) - All formula references will now be included until the final output model where
fields_to_includeis used to filter returned columns. - Unused
*inclusion_fieldsarguments have been removed from macros.
- Modified the
sfdc_formula_view_sqlmacro to remove extranneous double quotes on Redshift databases (#36) - Formulas were previously being lower cased within the macro to be consistent across the board. However, this resulted in some translations to be incorrect if they required specific casing. A fix has been implemented to rely on the casing within the
fivetran_formulatable instead of casing within the macro. ((#37)[#37])
- Removal of the
main_tablealiasing within the translatedview_sqlformulas. This would result in errors when self joins were being conducted. Therefore, this logic is not handled natively within the Salesforce connector. ((#38)[#38])
🎉 dbt v1.0.0 Compatibility 🎉
- Adjusts the
require-dbt-versionto now be within the range [">=1.0.0", "<2.0.0"]. Additionally, the package has been updated for dbt v1.0.0 compatibility. If you are using a dbt version <1.0.0, you will need to upgrade in order to leverage the latest version of the package.- For help upgrading your package, I recommend reviewing this GitHub repo's Release Notes on what changes have been implemented since your last upgrade.
- For help upgrading your dbt project to dbt v1.0.0, I recommend reviewing dbt-labs upgrading to 1.0.0 docs for more details on what changes must be made.
- Upgrades the package dependency to refer to the latest
dbt_fivetran_utils. The latestdbt_fivetran_utilspackage also has a dependency ondbt_utils[">=0.8.0", "<0.9.0"].- Please note, if you are installing a version of
dbt_utilsin yourpackages.ymlthat is not in the range above then you will encounter a package dependency error.
- Please note, if you are installing a version of
- Modified the
sfdc_formula_viewmacro to wrap the arguments within an additional parenthesis. This is required for Snowflake warehouses with mixed casing enabled. (#32) - Updated the
sfdc_old_formula_fieldsmacro to properly work with mixed-quoting styles in Snowflake. This was fixed by setting the target relation in the macro to explicitly be set to a table relation. (#32)
- @morgankrey-amplitude (#32)
- Update to the
fields_to_includeargument within thesfdc_formula_viewmacro. The macro now MUST be passed through as an array (using brackets[]) rather than a string (using parenthesis()). (#28)- This fixes a bug where more than one field to include would not work and you would be required to add two or more fields (#25). Additionally, passing the argument in as an array aligns more with dbt best practices.
- Addition of the
reserved_tableargument to thesfdc_formula_view_sqlmacro which resolves a bug identified where a duplicate alias error would occur due to aliasing not properly being accounted for in the final compile query. (#30)
- Addition of the
sfdc_old_formula_fieldsmacro which changes the nature of theold_formula_fieldsvariable within thesfdc_formula_viewmacro to exclude only formula fields that are related to the base table. (29)- Previously, if a formula field existed as a base field for another table then it would be erroneously excluded. With this update those fields will not be excluded.