pydf

Python package for Dataflow at GCP

User's Guide

Install from this repo

pip install -U git+https://github.com/liferoad/pydf

Developer's Guide

Requirements

• Python 3.9.x (3.10.x+ and 3.8.x- will not work)
  • Follow the instructions at https://github.com/pyenv/pyenv-installer, make sure that sqlite3 is installed on your local machine, then run the following
    • pyenv install 3.9
    • pyenv global 3.9

IDE

VSCode with Code Spell Checker + autoDocstring + Python + Pylance

Setup Environment

make init
source venv/bin/activate

If you want to clean up your environment,

make clean

Please always run these before you commit your code:

make format
make lint

Run tests:

make test