Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(source-facebook-marketing): added fivetran dbt converter #55

Merged
merged 29 commits into from
Aug 31, 2024
Merged

feat(source-facebook-marketing): added fivetran dbt converter #55

merged 29 commits into from
Aug 31, 2024

Conversation

portalhacker
Copy link
Contributor

@portalhacker portalhacker commented Aug 22, 2024 •
edited
Loading

resolves #10

✅ I moved the logic as much upstream as I could as seen in other issues, reducing the complexity.

⚠️ I likely have rare Postgresql syntax to wrap around a new macro to extract sublevel json text for BQ and SF.

Just want to make sure I am in the right direction and almost ready to merge.

@portalhacker
chore: complete phase 1
f735e56
@portalhacker
feat: add unionned accounts
6286239
@portalhacker
feat: add ads table
98657f6
@portalhacker
feat: add ad_set, campaign and creative (wip)
4489b4b
@portalhacker
feat: add basic ad staging
e4cf9ef
@portalhacker
chore: remove fivetran macros
a1da42b
@portalhacker
fix: add missing creative fields
e045223
@portalhacker
fix: use cross-db cast and json_extract macros, except for nested jso…
99bdb93 
…n objects
@portalhacker
chore: update status
7e00b3d
@portalhacker
fix: use the sources created by Airbyte
c3708ec
@portalhacker
fix: renamme source relation column from the start
d59385a
@portalhacker
fix: rename ad_set like in fivetran analyticsl package
3d74421
@portalhacker
fix: parse url tags
9224309
@portalhacker
fix: source relation as empty string for joins
c589d6c
@portalhacker
fix: parse url_tags
ddb30cf
@portalhacker
chore: update status
2bad000
@portalhacker
fix: clean output schemas and materializations + comment unused varia…
51eb5a5 
…bles
@portalhacker
feat: move all transformations upstream as tmp models
55f640a
@portalhacker
fix: add ad_history
fb0bfd3
@portalhacker
fix: add _dbt_source_relation columns in case of union
2a0d045
@portalhacker
chore: clean default vars
db2598c
@portalhacker
chore: clean dependencies
0b5a410
@portalhacker
chore: ignore local profiles.yml from versionning
52e0817
@portalhacker
chore: add README documentation
13c60d5
@portalhacker
chore: add integration tests
b953afa
@portalhacker
Merge branch 'airbytehq:main' into main
55031d2
@portalhacker
chore: remove unused documentation and macros
ef7bf04
@portalhacker
Merge branch 'main' of github.com:FloDevelops/airbyte-dbt-models
3c9c58e
@portalhacker portalhacker mentioned this pull request Aug 22, 2024
Closed
@marcosmarxm
Copy link
Member

Thanks @FloDevelops I'm gonna review tomorrow and give feedback about the contribution. It looks Facebook uses the union macro to query staging tables.

@portalhacker
Copy link
Contributor Author

@marcosmarxm correct!
Fivetran's staging/source package unions multiple sources into temporary models.
These temporary models are then used to create the staging models.

To minimize the conversion while keeping the most functionalities, I rebuilt the temporary models with Airbyte sources.
This way, I am still using Fivetran's staging and analytical models.

@portalhacker
Copy link
Contributor Author

@marcosmarxm just added the /vars file for the CI

@marcosmarxm marcosmarxm changed the title Source Facebook Marketing: added dbt converter feat(source-facebook-marketing): added fivetran dbt converter Aug 29, 2024
marcosmarxm
marcosmarxm approved these changes Aug 29, 2024
View reviewed changes
Copy link
Member

@marcosmarxm marcosmarxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are small details but the model looks good. Thanks @FloDevelops

@marcosmarxm marcosmarxm merged commit 6a97536 into airbytehq:main Aug 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marcosmarxm marcosmarxm marcosmarxm approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Create dbt converter for Airbyte Facebook Marketing source connector
2 participants
@portalhacker @marcosmarxm