Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

View ending in "WHERE <col> IN (<val1>, <val2>);" not parsed correctly by snowddl-convert #149

Open
JohnL4 opened this issue Nov 19, 2024 · 4 comments

Comments

@JohnL4
Copy link

JohnL4 commented Nov 19, 2024

Describe the bug
We have a view whose definition ends as in the title. It appears that the yaml generated by snowddl-convert has the trailing ")" stripped off. Looks like a recent addition of .rstrip(";") in the following line removes the trailing ";" that was previously protecting us from this.

https://github.com/littleK0i/SnowDDL/blame/c7ca3c48a533340bc88ac690b6664840005093ec/snowddl/converter/view.py#L86C9-L86C85

Expected behavior
Expected the output of snowddl-convert (in the "text" field of the view yaml) to have the full text of the view without truncations.

Attach log
No log (for now), but: Snowflake version = 8.43.0 (BUSINESS_CRITICAL), SnowDDL version = 0.35.1

Attach YAML config (if applicable)
Not applicable, I think.

@JohnL4
Copy link
Author

JohnL4 commented Nov 19, 2024

No idea what a good fix is here, apart from removing the .rstrip(";") (which is probably there for a reason) or the () characters from the .strip() expression (which are probably there for an even older reason :) ).

I'll remove the .rstrip(";") from our local version and see what happens.

@JohnL4
Copy link
Author

JohnL4 commented Nov 19, 2024

Ok, that last step did eliminate this issue for us. (Removing the .rstrip(";").)

@littleK0i
Copy link
Owner

littleK0i commented Nov 21, 2024

If you have time, you may try to adapt pyparsing module instead of using raw regexp:
https://pypi.org/project/pyparsing/

Here is the example:
https://github.com/pyparsing/pyparsing/blob/master/examples/simpleSQL.py

It should be something like:

  • Starts with CREATE
  • Followed by optional keywords
  • Followed by VIEW
  • Followed by optional anything inside parenthesis (column names with comments, not important)
  • Followed by optional anything
  • Followed by AS keyword outside of single quotes and parenthesis
  • Followed by everything else

Capture the final part. It should be raw VIEW SQL. At least until Snowflake changes something in future.

I'll do it myself as well, but a little bit later. Including pyparsing in dependencies would bump minimum Python version to 3.9.

@JohnL4
Copy link
Author

JohnL4 commented Nov 22, 2024

Interesting, thanks. Like you, I have some higher priorities, but I'll bear this in mind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants