Skip to content

This is End-To-End Data Engineering Project using Airflow and Python. In this project, we will extract data using Twitter API, use python to transform data, deploy the code on Airflow/EC2 and save the final result on Amazon S3

Notifications You must be signed in to change notification settings

alaahgag/twitter_pipeline

Repository files navigation

twitter_pipeline

Project Description

This is End-To-End Data Engineering Project using Airflow and Python. In this project, we will extract data using Twitter API, use python to transform data, deploy the code on Airflow/EC2 and save the final result on Amazon S3

Alt text

Running DAG on airflow

Alt text

Results on amazon S3

Alt text

About

This is End-To-End Data Engineering Project using Airflow and Python. In this project, we will extract data using Twitter API, use python to transform data, deploy the code on Airflow/EC2 and save the final result on Amazon S3

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages