CONTRIBUTING.md

Contributing to the kfp SDK

For developing KFP v2 SDK, use the master branch.

For developing KFP v1 SDK, use the sdk/release-1.8 branch.

For general contribution guidelines including pull request conventions, see pipelines/CONTRIBUTING.md.

Pre-requisites

Clone the repo:

git clone https://github.com/kubeflow/pipelines.git && cd pipelines

We suggest using a tool like virtual env or something similar for isolating your environment and/or packages for you development environment. For this setup we'll stick with virtual env.

For supported python versions, see the sdk setup.py.

# optional, replace with your tool of choice
python -m venv env && source ./env/bin/activate

As with any Python package development, first ensure wheels and setuptools are installed:

python -m pip install -U pip wheel setuptools

All sdk development requirement versions are pinned in requirements-dev.txt.

pip install -r sdk/python/requirements-dev.txt

Install the SDK in development mode using the --editable or -e flag. This will allow you to implement and test changes iteratively. Read more about it here.

pip install -e sdk/python

The SDK also relies on a couple other python packages also found within KFP. These consists of the api proto package and the kfp kubernetes_platform package.

For the proto code, we need protobuf-compiler to generate the python code. Read here more about how to install this dependency.

You can install both packages either in development mode if you are planning to do active development on these, or simply do a regular install.

To install the proto package:

pushd api
make generate python-dev # omit -dev for regular install
popd

To install the kubernetes_platform package:

pushd kubernetes_platform
make python-dev # omit -dev for regular install
popd

Testing

We suggest running unit tests using pytest. From the project root, the following runs all KFP SDK unit tests:

pytest

To run tests in parallel for faster execution, you can run the tests using the pytest-xdist plugin:

pytest -n auto

Code Style

Dependencies for code style checks/changes can be found in the kfp SDK requirements-dev.txt.

Style Guide [Required]

The KFP SDK follows the Google Python Style Guide.

Formatting [Required]

Please format your code using yapf according to the .style.yapf file.

From the project root, run the following code to format your code:

yapf --in-place --recursive ./sdk/python

Docformatter [Required]

We encourage you to lint your docstrings using docformatter.

From the project root, run the following code to lint your docstrings:

docformatter --in-place --recursive ./sdk/python

Formatting Imports [Required]

Please organize your imports using isort according to the .isort.cfg file.

From the project root, run the following code to format your code:

isort sdk/python --sg sdk/python/kfp/deprecated

Pylint [Encouraged]

We encourage you to lint your code using pylint according to the project .pylintrc file.

From the project root, run the following code to lint your code:

pylint ./sdk/python/kfp

Note: kfp is not currently fully pylint-compliant. Consider substituting the path argument with the files touched by your development.

Static Type Checking [Encouraged]

Please use mypy to check your type annotations.

From the project root, run the following code to lint your docstrings:

mypy ./sdk/python/kfp/

Note: kfp is not currently fully mypy-compliant. Consider substituting the path argument with the files touched by your development.

Pre-commit [Recommended]

Consider using pre-commit with the provided .pre-commit-config.yaml to implement the above changes:

pre-commit install