Skip to content

Commit

Permalink
python: Use uv everywhere, closes #112
Browse files Browse the repository at this point in the history
  • Loading branch information
jpmckinney committed Sep 12, 2024
1 parent a2a829b commit 50d555d
Show file tree
Hide file tree
Showing 8 changed files with 74 additions and 32 deletions.
12 changes: 5 additions & 7 deletions docs/github/maintainers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,15 @@ Onboard consultants or Start a project

If consultants need to make changes that require Admin privileges, instead, ask the consultants for instructions to make the changes yourself.

#. If using the ``stefanzweifel/git-auto-commit-action`` action in the ``lint.yml`` workflow, like in the :doc:`../python/django` Cookiecutter template:
#. If using the ``stefanzweifel/git-auto-commit-action`` action in the :ref:`lint.yml workflow<linting-ci>`, like in the :doc:`../python/django` Cookiecutter template to auto-fix requirements files:

#. Add the new repositories to the `Robots <https://github.com/orgs/open-contracting/teams/robots/repositories>`__ team
#. Set the *Permission level* to "Admin"

.. note::

The ``PAT`` environment variable is a personal access token of the ``ocp-deploy`` user with a ``repo:public_repo`` scope. This user is the only member of the Robots team, which needs permission to push fixes to protected branches.

#. Add the new repositories to `pre-commit ci <https://github.com/organizations/open-contracting/settings/installations/20658712>`__
#. Add the new repositories to `Coveralls <https://coveralls.io/repos/new>`__
#. Add any projects to :ref:`ReadTheDocs<readthedocs>` as appropriate
Expand All @@ -86,12 +90,6 @@ Per the `Software terms of reference (TOR) template <https://docs.google.com/doc

In order to protect the private deploy repositories, the `base permissions <https://github.com/organizations/open-contracting/settings/member_privileges>`__ for ``open-contracting`` members is *None*.

.. admonition:: pre-commit.ci workaround

pre-commit.ci `disallows network connections <https://github.com/pre-commit-ci/issues/issues/55>`__. As such, the ``pip-compile`` pre-commit hook is configured to be skipped in the ``ci`` section of the :ref:`.pre-commit-config.yaml file<linting-pre-commit>`, and is run by the :ref:`lint.yml workflow<linting-ci>`, instead.

This workflow auto fixes requirements files. To do so, it uses a personal access token of the ``ocp-deploy`` user with a ``repo:public_repo`` scope. The ``ocp-deploy`` user is the only member of the Robots team. As above, this team needs to be a repository administrator to push the auto fixes to protected branches.

Offboard consultants
--------------------

Expand Down
8 changes: 7 additions & 1 deletion docs/python/django.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,15 @@ Use the Django Cookiecutter template, and `add the repository to pre-commit.ci <

.. code-block:: bash
pip install cookiecutter
uv tool install cookiecutter
cookiecutter gh:open-contracting/software-development-handbook --directory cookiecutter-django
uv venv
uv pip sync requirements_dev.txt
uv tool install pre-commit
pre-commit install
For the ``stefanzweifel/git-auto-commit-action`` action, add the repository to the `Robots <https://github.com/orgs/open-contracting/teams/robots/repositories>`__ team, and set the *Permission level* to "Admin".

.. _django-layout:
Expand Down
6 changes: 6 additions & 0 deletions docs/python/linting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,12 @@ To avoid pushing commits that fail formatting or linting checks, new projects sh
.. literalinclude:: ../../cookiecutter-pypackage/{{cookiecutter.repository_name}}/.pre-commit-config.yaml
:language: yaml

.. note::

Applications set the correct Python version in the `default_language_version <https://pre-commit.com/#top_level-default_language_version>`__ section. Otherwise, pre-commit.ci (or the ``pre-commit`` command locally) can use the incorrect Python version for the ``pip-compile`` hook.

pre-commit.ci `disallows network connections <https://github.com/pre-commit-ci/issues/issues/55>`__. As such, the ``pip-compile`` hook is configured to be skipped in the ``ci`` section, and is run by the :ref:`lint.yml workflow<linting-ci>`, instead.

.. note::

`pre-commit/pre-commit-hooks <https://github.com/pre-commit/pre-commit-hooks>`__ is not used in the templates, as the errors it covers are rarely encountered.
Expand Down
8 changes: 7 additions & 1 deletion docs/python/packages.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,15 @@ Use the Pypackage Cookiecutter template, and `add the repository to pre-commit.c

.. code-block:: bash
pip install cookiecutter
uv tool install cookiecutter
cookiecutter gh:open-contracting/software-development-handbook --directory cookiecutter-pypackage
uv venv -p 3.9
uv pip install -e .[test]
uv tool install pre-commit
pre-commit install
.. seealso::

Package-rated content in :ref:`Directory layout<layout-packages>`, :ref:`Testing<automated-testing>` and :ref:`Linting<linting-ci>`
Expand Down
Empty file removed docs/python/pyproject-ruff.toml
Empty file.
62 changes: 39 additions & 23 deletions docs/python/requirements.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,18 @@ Requirements

- When you ``import`` a package for the first time in your application code, add its name to ``requirements.in`` in alphabetical order.
- When you ``import`` a package for the first time in your test code, that is never imported in your application code, add its name to ``requirements_dev.in`` in alphabetical order.
- After updating a ``.in`` file, update the ``.txt`` files with ``pip-compile; pip-compile requirements_dev.in``. Never edit the ``.txt`` files directly.
- To update your local environment, run ``pip-sync requirements_dev.txt``.
- After updating a ``.in`` file, update the ``.txt`` files with:

.. code-block:: bash
uv pip compile requirements.in -o requirements.txt
uv pip compile requirements_dev.in -o requirements_dev.txt
- To update your local environment, run:

.. code-block:: bash
uv pip sync requirements_dev.txt
Now that you have a :doc:`directory layout<layout>`, you can declare the project's requirements.

Expand Down Expand Up @@ -37,6 +47,18 @@ Get started

`Install uv <https://docs.astral.sh/uv/getting-started/installation/>`__.

Virtual environment
~~~~~~~~~~~~~~~~~~~

Create a ``.python-version`` file (for example, containing ``3.11``), then run:

.. code-block:: bash
uv venv
Requirements files
~~~~~~~~~~~~~~~~~~

A common starter ``requirements.in`` for :doc:`django` is:

.. literalinclude:: ../../cookiecutter-django/{{cookiecutter.project_slug}}/requirements.in
Expand All @@ -58,10 +80,8 @@ Add the requirement in alphabetical order to the appropriate ``.in`` file. Then,

.. code-block:: bash
pip-compile
pip-compile requirements_dev.in
If running ``pip-compile`` introduces unexpected differences, upgrade ``pip-tools`` to the latest version, and check that you are using the same version of Python as for other runs.
uv pip compile requirements.in -o requirements.txt
uv pip compile requirements_dev.in -o requirements_dev.txt
.. seealso::

Expand All @@ -85,36 +105,34 @@ psycopg2
Install requirements
--------------------

In development:

.. code-block:: bash
pip-sync requirements_dev.txt
In production:

.. code-block:: bash
pip-sync -q --pip-args "--exists-action w"
uv pip sync requirements_dev.txt
Upgrade requirements
--------------------

Requirements should be periodically updated, both for security updates and to better distribute the maintenance burden of upgrading versions over time.

Upgrade one requirement, for example:
Upgrade one requirement in ``requirements.in``, for example:

.. code-block:: bash
pip-compile -P requests
pip-compile -P requests requirements_dev.in
uv pip compile -P requests requirements.in -o requirements.txt
uv pip compile requirements_dev.in -o requirements_dev.txt
Upgrade all requirements:
Upgrade one requirement in ``requirements_dev.in``, for example:

.. code-block:: bash
pip-compile --upgrade
pip-compile --upgrade requirements_dev.in
uv pip compile -P pytest requirements_dev.in -o requirements_dev.txt
Upgrade all requirements (rare):

.. code-block:: bash
uv pip compile -U requirements.in -o requirements.txt
uv pip compile -U requirements_dev.in -o requirements_dev.txt
Linting
-------
Expand All @@ -132,5 +150,3 @@ If a requirement is reported as unused but is required:
#. If the package is optional – for example, it is imported in a ``try`` and ``except ImportError`` block – use the ``STANDARD_MAINTENANCE_SCRIPTS_IGNORE`` `environment variable <https://github.com/open-contracting/standard-maintenance-scripts#tests>`__.

#. If the package is used exclusively outside of application code – for example, as a command in the production environment or in a GitHub workflow, or in an `entry point <https://packaging.python.org/en/latest/specifications/entry-points/>`__ or as a Sphinx extension – use the ``STANDARD_MAINTENANCE_SCRIPTS_IGNORE`` environment variable.

If the repository uses :ref:`linting-pre-commit`, and you see errors originating from ``pip``, it is likely that you need to upgrade ``pip-tools`` in both ``.pre-commit-config.yaml`` and ``requirements_dev.txt``.
6 changes: 6 additions & 0 deletions docs/python/samples/pyproject-ruff.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[tool.ruff]
line-length = 119
target-version = "py311"

[tool.ruff.lint]
select = ["E", "C4", "F", "I", "W"]
4 changes: 4 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
[project]
name = "software-development-handbook"
version = "0.0.0"

[tool.ruff]
# https://github.com/astral-sh/ruff/issues/9381
extend-exclude = ["cookiecutter-django", "cookiecutter-pypackage"]

0 comments on commit 50d555d

Please sign in to comment.