-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add duckdb build/concepts and use SQLGlot to convert BigQuery SQL into other dialects #1689
Conversation
…)...committing before gutting dead code.
…es without a large amount of RAM, and this way it can be skipped with `--skip-indexes`
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, looks good to me, with the caveat that I haven't run the code! (and that breaking this into smaller PRs would be nicer). I added a couple of notes about typos but all very minor, so feel free to go ahead and merge. The sqlglot
package looks very cool.
How much manual tweaking of the outputs is needed?
Ideally I think we would maintain a single source format (psql/bigquery) and then create other dialects in a GitHub workflow. My preference would be to have dialects on separate branches, rather than trying to maintain them all on main.
Thanks for the review! Fixed the typos; regarding some unaddressed points:
|
A few things in this PR, mainly to get the concepts to work with duckdb. This builds off #1529 / @SphtKr. However #1529 has the conversion for MIMIC-III, whereas here I've converted MIMIC-IV. It should be straightforward to adapt for MIMIC-III, but I can't explain the parsing errors I'm getting when running it over that folder, so I've left it for a future PR.
I also overhauled the READMEs a bit for clarity.