pants seems only support an old verision of ruff, without understanding recent ruff config syntax

[Sage0614]

Describe the bug

[tool.ruff]
extend-include =["*.ipynb"]

[tool.ruff.lint]
select = ["F"]
ignore = ["F841"]

pants doens't recogize either syntax above, it seems only support ruff to a certain verison, which I didn't find the specific version it support in documentation.

Pants version
2.19.1

OS
linux

Additional info
Add any other information about the problem here, such as attachments or links to gists, if relevant.

[sureshjoshi]

Does pants help tools tell you which version is installed? Or alternatively, any of the lockfiles?

[Sage0614]

Does pants help tools tell you which version is installed? Or alternatively, any of the lockfiles?

in the lockfile it is ruff 0.3.4, which is expected version as I configed with pyproject.toml; running ruff check or ruff format directly doesn't have problem

[benjyw]

Please post the error message you're seeing. And even more helpful would be a small repo that reproduces the problem. Thanks!

[Sage0614]

minimum reproducible example:
pants.toml

[GLOBAL]
pants_version = "2.19.1"
backend_packages = ["pants.backend.python","pants.backend.experimental.python.lint.ruff"]
[python]
interpreter_constraints = [ '>=3.11,<3.12' ]
[ruff]
config_discovery = true

pyproject.toml

[tool.poetry]
name = "bug-report"
version = "0.1.0"
description = ""
authors = ["Song Wang <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"
ibis-framework = {extras = ["duckdb"], version = "^7.0.0"}


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[tool.ruff]
target-version = "py311"
extend-include = ["*.ipynb"]
lint.select = ["F","E","W","I001"]
lint.ignore = ["F841"]

folder structure:

root
    |---project
         |--- __init__.py
pants.toml
pyproject.toml

run ruff check --fix shows:

All checks passed!

run pants lint:: shows:

10:47:19.82 [INFO] Completed: Building ruff.pex from resource://pants.backend.python.lint.ruff/ruff.lock
10:47:19.85 [ERROR] Completed: Lint with ruff - ruff failed (exit code 2).
error: TOML parse error at line 19, column 1
   |
19 | extend-include = ["*.ipynb"]
   | ^^^^^^^^^^^^^^
unknown field `extend-include`, expected one of `allowed-confusables`, `builtins`, `cache-dir`, `dummy-variable-rgx`, `exclude`, `extend`, `extend-exclude`, `extend-ignore`, `extend-select`, `external`, `fix`, `fix-only`, `fixable`, `format`, `force-exclude`, `ignore`, `ignore-init-module-imports`, `line-length`, `required-version`, `respect-gitignore`, `select`, `show-source`, `show-fixes`, `src`, `namespace-packages`, `target-version`, `task-tags`, `typing-modules`, `unfixable`, `update-check`, `flake8-annotations`, `flake8-bandit`, `flake8-bugbear`, `flake8-builtins`, `flake8-comprehensions`, `flake8-errmsg`, `flake8-quotes`, `flake8-self`, `flake8-tidy-imports`, `flake8-type-checking`, `flake8-gettext`, `flake8-implicit-str-concat`, `flake8-import-conventions`, `flake8-pytest-style`, `flake8-unused-arguments`, `isort`, `mccabe`, `pep8-naming`, `pycodestyle`, `pydocstyle`, `pylint`, `pyupgrade`, `per-file-ignores`




10:47:19.85 [INFO] Completed: Fix with ruff - ruff --fix made no changes.
✕ ruff failed.
✓ ruff --fix succeeded.

[huonw]

Hey @Sage0614, thanks for filing an issue and making a reproducer.

Pants has default versions of each tool hard-coded to make it easy to get started (and pinned to a specific version for reproducibility and software supply-chain assurance), but, as you observe, this default version becomes outdated as tools release new versions. In the 2.19 release series, the default version is 0.1, but this has been updated in more recent ones (in 2.20.0, releasing soon, it's 0.2, and 2.21 has #20647 which updates to 0.3.0).

However, no matter the Pants version, the default versions can be overridden, using a custom lockfile: https://www.pantsbuild.org/2.19/docs/python/overview/lockfiles#lockfiles-for-tools

So, something like:

[GLOBAL]
pants_version = "2.19.1"
backend_packages = ["pants.backend.python","pants.backend.experimental.python.lint.ruff"]
[python]
interpreter_constraints = [ '>=3.11,<3.12' ]
enable_resolves = true

[python.resolves]
ruff = "3rdparty/python/ruff.lock"

[ruff]
install_from_resolve = "ruff"
config_discovery = true

This would then be combined with a BUILD file like:

python_requirement(name="ruff", requirements=["ruff==0.3.4"], resolve="ruff")

Let us know if this helps.

[Sage0614]

the part I am missing is config [ruff] install_from_resolve option. add the configuration solve the problem. thanks!

[benjyw]

It's unfortunate that this wasn't more clear from the docs. Any thoughts on where specifically in the docs this information would have been helpful?

[Sage0614]

It's unfortunate that this wasn't more clear from the docs. Any thoughts on where specifically in the docs this information would have been helpful?

I look at the doc again and both the lockfiles part and ruff options part does mentioned that install_from_resolve is needed, so it is more of ignorance from my side, where I assume set the lockfile alone would specify the version of the tool, I would say a more helpful way maybe is add an example config of non default tool to https://github.com/pantsbuild/example-python/blob/main/pants.toml, and have some comment about what it does.

[huonw]

That sounds good.

I think it's also unhelpful that we don't indicate the version of these tools anywhere in the docs pages, so I've filed #20722.