Fix column declaration and enhance regexp_like support for Snowflake #107
Conversation
There was a problem hiding this comment.
@katy-sadowski this may not be desired but given we've done a lot of cleaning up recently I thought it would be silly to not try and add the finishing touches to enable snowflake support.
Currently missing documentation (thought I'd await people's views first)!
If we decide against it then at the very least take 1a2e6cd changes (took me too long to figure out why it was causing problems 💀)!
| , CASE | ||
| WHEN o.observation_value ~ '^[-+]?[0-9]+\.?[0-9]*$' | ||
| WHEN {{ regexp_like("o.observation_value", "^[-+]?[0-9]+\.?[0-9]*$") }} | ||
| THEN {{ dbt.cast("o.observation_value", api.Column.translate_type("decimal")) }} | ||
| ELSE {{ dbt.cast("null", api.Column.translate_type("decimal")) }} | ||
| END AS value_as_number |
There was a problem hiding this comment.
the new macro is useful and we should keep it for future use (thanks!!!) - but looking at this gross regex, maybe we could use safe_cast here instead? https://docs.getdbt.com/reference/dbt-jinja-functions/cross-database-macros#safe_cast
There was a problem hiding this comment.
@katy-sadowski Had another look through the cross-DB macros and fairly sure there's no regular expression oriented ones sadly - I don't think there is a way around this without using custom dispatchers / macros like this
It raises the question of do we event want to add ❄️ support! I think it is nice to have - just wish they used the ~ operator...
I looked into transpiring with SQLGlot, but I think thats a lot of work for relatively low yield - pulling out the DAG from dbt etc as per here #39 (comment) is quite a lot of work which isn't in the scope of the project (IMHO!)
There was a problem hiding this comment.
ahhh sorry i forgot about what you already said in #68 about safe_cast not being supported on all platforms. let's carry on with the regex approach! agree re: sqlglot.
|
Honestly hair pulling stupidness! I'm not sure safe_cast is functionally equivalent but will give it ago! Sadly dbt has no native binding for this function ... |
This PR adds support for snowflake as a backend through various steps.
Idcolumn across multiple seed filesregexp_likemacro for Snowflake, and...