Repository

The repository is organized as a monorepo meaning that it hosts all arcor2-related Python packages developed by the Robo@FIT research group. To make maintenance of the monorepo easier and to simplify development, testing, and releasing, we have adopted the Pants build system.

Content

The root of the repository contains:

• .github/workflows - definition of GitHub Actions, which we use for CI.
• 3rdparty - contains (mainly) global requirements.txt and constraints.txt.
• build-support - scripts useful for developers.
  • generate_constraints.sh - after updating requirements.txt, don't forget to run this, which updates constraints.txt.
  • install_kinect_prerequisites.sh - installs Kinect Azure SDK (system dependency, works on Ubuntu 18.04) which is needed by pyk4a.
  • nuke-cache.sh - Pants cache can get really huge and this will clear it.
  • setup-venv.sh - creates a virtualenv based on requirements.txt - a programmer might then use it in a terminal (run source build-support/.venv/bin/activate), or in e.g. PyCharm in order to get all imports available (set build-support/.venv/bin/python3.9 as a project interpreter).
• docker - Docker files and scripts used to build our images - mainly used by maintainers, but any developer might want to build their own local version of a service. See README or Docker-related part of the Releasing page.
• pants-plugins - our custom plugins for Pants, which simplify our BUILD files.
• src/python - contains individual Python packages.
• pants - Pants script, with minimal dependencies. When invoked, it initializes the environment on its own.
• pants.toml - Pants configuration, defines e.g. version of Pants, version of Python (we use the same version for all packages) - currently Python 3.9, etc.
• ...some other common stuff

Please note: we use LFS to store some big files (e.g. meshes), so you might need to run git lfs checkout.

Packages

Individual packages are directories in src/python. By convention, the package name always starts with arcor2_. The mandatory content of each package is:

• __init__.py
• CHANGELOG.md - with the format following Keep a Changelog.
• py.typed - because all the code has type annotations and therefore packages are marked as PEP 561 compatible.
• README.md - basic information about the package and its usage (e.g. description of environment variables), the format is not strictly given.
• VERSION - containing SemVer string defining package version.

The minimal content of the top-level __init__.py should be:

from arcor2 import package_version

def version() -> str:
    return package_version(__name__)

When using Pants, there is no manually created setup.py file. Each package contains a BUILD file, defining some basic metadata about the package, that will be used when generating distributions (either whl or tar.gz files). The main BUILD file of the package is highly simplified by arcor2_python_distribution macro and might look like this:

arcor2_python_distribution(
    name="arcor2_my_service",
    description="ARCOR2 Hyper Extra Service",
    dependencies=[
        "src/python/arcor2_my_service/whatever_module",
    ],
    binaries={
        "arcor2_my_service": "src/python/arcor2_my_service/scripts:my_service"
    }
)

...where any sub-package must also contain its own BUILD file, defining available targets. For instance, the directory scripts will contain __init__.py, my_service.py and the BUILD file with the following content:

python_sources()

arcor2_pex_binary(
    name = "my_service"
)

How to run my_service? There are two easy possibilities:

• Enter virtual environment and just run the script.
• Build a PEX file with ./pants package src/python/arcor2_my_service/scripts:my_service and run it - it is self-contained and executable.
  • There is also ./pants run ....