Consider a Python manager for inline script metadata (PEP 723)

[blakeNaccarato]

PEP 723 specifies "inline script metadata", e.g. a way for Python scripts to specify their own requirements in a pyproject.toml-like field embedded in a specially-formatted comment in the script itself. This allows pipx run/pip-run/etc. to run dev tooling Python scripts that specify their own dependencies and avoids the chicken-and-egg problem of needing a Python environment to set up a Python environment.

It would be nice if dependencies in standalone scripts (e.g. in dev tooling) could be managed by Renovate! Some signals to pay attention to regarding the acceptance/adoption of PEP 723 are as follows. This PEP is currently "Accepted" status, but has not yet been marked "Final":

• PEP 723 officially marked "Final"

But there's a note just below the table of contents suggesting that it will reach "Final" status under the following conditions (I've linked to indications that these steps are complete):

• It has been specified at https://packaging.python.org/ (merged on 2024-01-21).

Implemented in "a couple of tools", and they mention pipx and pip-run by name:

• pipx 1.4.2 (2024-01-12)
• pip-run 12.5.0 (2024-01-20)
• uv via uv pipx run or similar: A requested feature (A uv run subcommand astral-sh/uv#1207, uv pipx {run,install} astral-sh/uv#1173)

Additionally, the January 2024 history entry suggests:

• The specification is "no longer provisional" (2024-01)

So this spec seems stable enough to start considering a manager implementation. Once PEP 723 is officially marked "Final," maybe a manager implementation can begin at Renovate? Thank you for your consideration!

[rarkins]

Is the format strictly defined (e.g. like toml/yaml), or flexible, like python code?

If it's not strictly defined, it won't be possible to support all possibilities, and potentially may not even be worth supporting a small subset. In such cases users would need to use Renovate's custom regex manager instead of a dedicated manager.

If it's strictly defined, or there are very common conventions, then potentially it's appropriate for its own manager.

[mkniewallner]

The schema is using a strictly defined TOML schema, but commented out in Python files: https://packaging.python.org/en/latest/specifications/inline-script-metadata/#inline-script-metadata. For instance, from the example:

# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "requests<3",
#   "rich",
# ]
# ///

Dependencies use PEP 508 format, that Renovate already parses for PEP 621 manager.

A regex could be used to detect such header, as recommended by the PEP. We could then try to parse the content as TOML. I made a dumb implementation in a fork just to get an idea of what this would look like if Renovate was to support it: mkniewallner@b6efe52.

What I'm afraid of is that technically, any Python file in a codebase could contain such header, so setting technical constraints aside, ideally we would need to pattern match against all Python files by default. I didn't benchmark that, but on huge codebases, I believe this could make Renovate really slow. Would a safer default be to explicitly require users to set the files that Renovate should handle for this specific manager?

[rarkins]

@secustor a candidate for your preflight checks?

[secustor]

Not a prime candidate, but indeed we could optimize with preflight.

Scan all .py files until we find the first ocurrence of # /// script and then enable this new manager.
Ideally though there would other references so we do not have to ever to read all python files

[mkniewallner]

Is preflight already something available and that we could use today? If not, what would would be the recommendation between:

• matching no file at all by default, and require users to explicitly specify the files where we have PEP 723-format scripts
• matching all Python files by default and search for occurrences

For option 1, I guess that this would be configured through fileMatch, so even if it's opt-in, people could still make a really broad regex to try to match any file, which could lead to a slowdown?

[secustor]

No, preflight is something we work on currently.

Yes, option 1 would be the way to go. This is the way we use for example in the Kubernetes manager.

[mkniewallner]

We should probably close this discussion, as this has been implemented in #31266.

[Mrjbudpayj]

Ty

…

On Sat, Nov 2, 2024, 12:42 PM Mathieu Kniewallner ***@***.***> wrote: We should probably close this discussion, as this has been implemented in #31266 <#31266>. — Reply to this email directly, view it on GitHub <#27889 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZOSUBNWN2YJUFRZRQDZ2B3Z6T6GVAVCNFSM6AAAAABESTIK6SVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMJTGAZTMNA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>