Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add: option to select enabled (and required) extensions and code formatter languages #477

Merged
merged 1 commit into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 16 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ If a file is not properly formatted, the exit code will be non-zero.
foo@bar:~$ mdformat --help
usage: mdformat [-h] [--check] [--version] [--number] [--wrap {keep,no,INTEGER}]
[--end-of-line {lf,crlf,keep}] [--exclude PATTERN]
[--extensions EXTENSION] [--codeformatters LANGUAGE]
[paths ...]

CommonMark compliant Markdown formatter
Expand All @@ -112,6 +113,12 @@ options:
--end-of-line {lf,crlf,keep}
output file line ending mode (default: lf)
--exclude PATTERN exclude files that match the Unix-style glob pattern (multiple allowed)
--extensions EXTENSION
require and enable an extension plugin (multiple allowed) (use
`--no-extensions` to disable) (default: all enabled)
--codeformatters LANGUAGE
require and enable a code formatter plugin (multiple allowed)
(use `--no-codeformatters` to disable) (default: all enabled)
```

The `--exclude` option is only available on Python 3.13+.
Expand Down Expand Up @@ -142,18 +149,22 @@ Here's a few pointers to get you started:
Mdformat is a CommonMark formatter.
It doesn't have out-of-the-box support for syntax other than what is defined in [the CommonMark specification](https://spec.commonmark.org/current/).

The custom syntax that these Markdown engines introduce typically reinvents the meaning of
angle brackets, square brackets, parentheses, hash characters — characters that have a special meaning in CommonMark.
Mdformat often resorts to backslash escaping these characters to ensure the formatting changes it makes never alters a rendered document.
The custom syntax that these Markdown engines introduce typically redefines the meaning of
angle brackets, square brackets, parentheses, hash character — characters that are special in CommonMark.
Mdformat often resorts to backslash escaping these characters to ensure its formatting changes never alter a rendered document.

Additionally some engines, namely MkDocs, [do not support](https://github.com/mkdocs/mkdocs/issues/1835) CommonMark to begin, so incompatibilities are unavoidable.
Additionally some engines, namely MkDocs,
[do not support](https://github.com/mkdocs/mkdocs/issues/1835) CommonMark to begin with,
so incompatibilities are unavoidable.

Luckily mdformat is extensible by plugins.
For many Markdown engines you'll find support by searching
[the plugin docs](https://mdformat.readthedocs.io/en/stable/users/plugins.html)
or [mdformat GitHub topic](https://github.com/topics/mdformat).

You may also want to consider a documentation engine that adheres to CommonMark as its base syntax e.g. [mdBook](https://rust-lang.github.io/mdBook/) or [Sphinx with Markdown](https://www.sphinx-doc.org/en/master/usage/markdown.html).
You may also want to consider a documentation generator that adheres to CommonMark as its base syntax
e.g. [mdBook](https://rust-lang.github.io/mdBook/)
or [Sphinx with Markdown](https://www.sphinx-doc.org/en/master/usage/markdown.html).

### Why not use [Prettier](https://github.com/prettier/prettier) instead?

Expand Down
3 changes: 3 additions & 0 deletions docs/users/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ Note that there is currently no guarantee for a stable Markdown formatting style
- Added
- Plugin interface: `mdformat.plugins.ParserExtensionInterface.add_cli_argument_group`.
With this plugins can now read CLI arguments merged with values from `.mdformat.toml`.
- Option to select enabled (and required) extensions and code formatter languages
(`--extensions` and `--codeformatters` on the CLI,
and `extensions` and `codeformatters` keys in TOML).
- Improved plugin list at the end of `--help` output:
List languages supported by codeformatter plugin distributions,
and parser extensions added by parser extension distributions.
Expand Down
18 changes: 14 additions & 4 deletions docs/users/configuration_file.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,20 @@ Command line interface arguments take precedence over the configuration file.
# no configuration file at all. Change the values for non-default
# behavior.
#
wrap = "keep" # possible values: {"keep", "no", INTEGER}
number = false # possible values: {false, true}
end_of_line = "lf" # possible values: {"lf", "crlf", "keep"}
wrap = "keep" # options: {"keep", "no", INTEGER}
number = false # options: {false, true}
end_of_line = "lf" # options: {"lf", "crlf", "keep"}
# extensions = [ # options: a list of enabled extensions (default: all installed are enabled)
# "gfm",
# "toc",
# ]
# codeformatters = [ # options: a list of enabled code formatter languages (default: all installed are enabled)
# "python",
# "json",
# ]

# Python 3.13+ only:
exclude = [] # possible values: a list of file path pattern strings
exclude = [] # options: a list of file path pattern strings
```

## Exclude patterns
Expand All @@ -36,6 +44,8 @@ Glob patterns are matched against relative paths.
If `--exclude` is used on the command line, the paths are relative to current working directory.
Else the paths are relative to the parent directory of the file's `.mdformat.toml`.

Only files (recursively) contained by the base directory can be excluded.

Files that match an exclusion pattern are _always_ excluded,
even in the case that they are directly referenced in a command line invocation.

Expand Down
89 changes: 78 additions & 11 deletions src/mdformat/_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,20 +25,10 @@ def emit(self, record: logging.LogRecord) -> None:


def run(cli_args: Sequence[str]) -> int: # noqa: C901
# Enable all parser plugins
enabled_parserplugins = mdformat.plugins.PARSER_EXTENSIONS
# Enable code formatting for all languages that have a plugin installed
enabled_codeformatters = mdformat.plugins.CODEFORMATTERS

changes_ast = any(
getattr(plugin, "CHANGES_AST", False)
for plugin in enabled_parserplugins.values()
)

arg_parser = make_arg_parser(
mdformat.plugins._PARSER_EXTENSION_DISTS,
mdformat.plugins._CODEFORMATTER_DISTS,
enabled_parserplugins,
mdformat.plugins.PARSER_EXTENSIONS,
)
cli_opts = {
k: v for k, v in vars(arg_parser.parse_args(cli_args)).items() if v is not None
Expand Down Expand Up @@ -84,6 +74,45 @@ def run(cli_args: Sequence[str]) -> int: # noqa: C901
)
return 1

try:
enabled_parserplugins = (
mdformat.plugins.PARSER_EXTENSIONS
if opts["extensions"] is None
else {
k: mdformat.plugins.PARSER_EXTENSIONS[k] for k in opts["extensions"]
}
)
except KeyError as e:
print_error(
"Invalid extension required.",
paragraphs=[
f"The required {e.args[0]!r} extension is not available. "
"Please install a plugin that adds the extension, "
"or remove it from required extensions."
],
)
return 1
try:
enabled_codeformatters = (
mdformat.plugins.CODEFORMATTERS
if opts["codeformatters"] is None
else {
k: mdformat.plugins.CODEFORMATTERS[k]
for k in opts["codeformatters"]
}
)
except KeyError as e:
print_error(
"Invalid code formatter required.",
paragraphs=[
f"The required {e.args[0]!r} code formatter language "
"is not available. "
"Please install a plugin "
"that adds support for the language, "
"or remove it from required languages."
],
)
return 1
if path:
path_str = str(path)
# Unlike `path.read_text(encoding="utf-8")`, this preserves
Expand Down Expand Up @@ -111,6 +140,10 @@ def run(cli_args: Sequence[str]) -> int: # noqa: C901
format_errors_found = True
print_error(f'File "{path_str}" is not formatted.')
else:
changes_ast = any(
getattr(plugin, "CHANGES_AST", False)
for plugin in enabled_parserplugins.values()
)
if not changes_ast and not is_md_equal(
original_str,
formatted_str,
Expand Down Expand Up @@ -198,6 +231,40 @@ def make_arg_parser(
help="exclude files that match the Unix-style glob pattern "
"(multiple allowed)",
)
extensions_group = parser.add_mutually_exclusive_group()
extensions_group.add_argument(
"--extensions",
action="append",
metavar="EXTENSION",
help="require and enable an extension plugin "
"(multiple allowed) "
"(use `--no-extensions` to disable) "
"(default: all enabled)",
)
extensions_group.add_argument(
"--no-extensions",
action="store_const",
const=(),
dest="extensions",
help=argparse.SUPPRESS,
)
codeformatters_group = parser.add_mutually_exclusive_group()
codeformatters_group.add_argument(
"--codeformatters",
action="append",
metavar="LANGUAGE",
help="require and enable a code formatter plugin "
"(multiple allowed) "
"(use `--no-codeformatters` to disable) "
"(default: all enabled)",
)
codeformatters_group.add_argument(
"--no-codeformatters",
action="store_const",
const=(),
dest="codeformatters",
help=argparse.SUPPRESS,
)
for plugin in parser_extensions.values():
if hasattr(plugin, "add_cli_options"):
import warnings
Expand Down
14 changes: 14 additions & 0 deletions src/mdformat/_conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
"end_of_line": "lf",
"exclude": [],
"plugin": {},
"extensions": None,
"codeformatters": None,
}


Expand Down Expand Up @@ -72,6 +74,18 @@ def _validate_values(opts: Mapping, conf_path: Path) -> None: # noqa: C901
for plugin_conf in opts["plugin"].values():
if not isinstance(plugin_conf, dict):
raise InvalidConfError(f"Invalid 'plugin' value in {conf_path}")
if "extensions" in opts:
if not isinstance(opts["extensions"], list):
raise InvalidConfError(f"Invalid 'extensions' value in {conf_path}")
for extension in opts["extensions"]:
if not isinstance(extension, str):
raise InvalidConfError(f"Invalid 'extensions' value in {conf_path}")
if "codeformatters" in opts:
if not isinstance(opts["codeformatters"], list):
raise InvalidConfError(f"Invalid 'codeformatters' value in {conf_path}")
for lang in opts["codeformatters"]:
if not isinstance(lang, str):
raise InvalidConfError(f"Invalid 'codeformatters' value in {conf_path}")


def _validate_keys(opts: Mapping, conf_path: Path) -> None:
Expand Down
20 changes: 17 additions & 3 deletions tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,7 @@
import mdformat
from mdformat._cli import get_package_name, get_plugin_info_str, run, wrap_paragraphs
from mdformat.plugins import CODEFORMATTERS

UNFORMATTED_MARKDOWN = "\n\n# A header\n\n"
FORMATTED_MARKDOWN = "# A header\n"
from tests.utils import FORMATTED_MARKDOWN, UNFORMATTED_MARKDOWN


def test_no_files_passed():
Expand Down Expand Up @@ -412,3 +410,19 @@ def test_exclude(tmp_path):
file_path_1.write_text(UNFORMATTED_MARKDOWN)
assert run([str(file_path_1), "--exclude", bad_pattern]) == 0
assert file_path_1.read_text() == FORMATTED_MARKDOWN


def test_codeformatters__invalid(tmp_path, capsys):
file_path = tmp_path / "test.md"
file_path.write_text("")
assert run((str(file_path), "--codeformatters", "no-exists")) == 1
captured = capsys.readouterr()
assert "Error: Invalid code formatter required" in captured.err


def test_extensions__invalid(tmp_path, capsys):
file_path = tmp_path / "test.md"
file_path.write_text("")
assert run((str(file_path), "--extensions", "no-exists")) == 1
captured = capsys.readouterr()
assert "Error: Invalid extension required" in captured.err
11 changes: 6 additions & 5 deletions tests/test_config_file.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@
import pytest

from mdformat._cli import run
from mdformat._conf import read_toml_opts
from tests.test_cli import FORMATTED_MARKDOWN, UNFORMATTED_MARKDOWN
from tests.utils import FORMATTED_MARKDOWN, UNFORMATTED_MARKDOWN, run_with_clear_cache


def test_cli_override(tmp_path):
Expand Down Expand Up @@ -70,6 +69,10 @@ def test_invalid_toml(tmp_path, capsys):
("exclude", "exclude = ['1',3]"),
("plugin", "plugin = []"),
("plugin", "plugin.gfm = {}\nplugin.myst = 1"),
("codeformatters", "codeformatters = 'python'"),
("extensions", "extensions = 'gfm'"),
("codeformatters", "codeformatters = ['python', 1]"),
("extensions", "extensions = ['gfm', 1]"),
],
)
def test_invalid_conf_value(bad_conf, conf_key, tmp_path, capsys):
Expand All @@ -87,15 +90,13 @@ def test_invalid_conf_value(bad_conf, conf_key, tmp_path, capsys):


def test_conf_with_stdin(tmp_path, capfd, monkeypatch):
read_toml_opts.cache_clear()

config_path = tmp_path / ".mdformat.toml"
config_path.write_text("number = true")

monkeypatch.setattr(sys, "stdin", StringIO("1. one\n1. two\n1. three"))

with mock.patch("mdformat._cli.Path.cwd", return_value=tmp_path):
assert run(("-",)) == 0
assert run_with_clear_cache(("-",)) == 0
captured = capfd.readouterr()
assert captured.out == "1. one\n2. two\n3. three\n"

Expand Down
50 changes: 50 additions & 0 deletions tests/test_plugins.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
_load_entrypoints,
)
from mdformat.renderer import MDRenderer, RenderContext, RenderTreeNode
from tests.utils import run_with_clear_cache


def test_code_formatter(monkeypatch):
Expand Down Expand Up @@ -496,3 +497,52 @@ def test_load_entrypoints(tmp_path, monkeypatch):
loaded_eps, dist_infos = _load_entrypoints(entrypoints)
assert loaded_eps == {"ext1": mdformat.plugins, "ext2": mdformat.plugins}
assert dist_infos == {"mdformat-gfm": ("0.3.6", ["ext1", "ext2"])}


def test_no_codeformatters__toml(tmp_path, monkeypatch):
monkeypatch.setitem(CODEFORMATTERS, "json", JSONFormatterPlugin.format_json)
unformatted = """\
```json
{"a": "b"}
```
"""
formatted = """\
```json
{
"a": "b"
}
```
"""
file1_path = tmp_path / "file1.md"

# Without TOML
file1_path.write_text(unformatted)
assert run((str(tmp_path),)) == 0
assert file1_path.read_text() == formatted

# With TOML
file1_path.write_text(unformatted)
config_path = tmp_path / ".mdformat.toml"
config_path.write_text("codeformatters = []")
assert run_with_clear_cache((str(tmp_path),)) == 0
assert file1_path.read_text() == unformatted


def test_no_extensions__toml(tmp_path, monkeypatch):
plugin = ExampleASTChangingPlugin()
monkeypatch.setitem(PARSER_EXTENSIONS, "ast_changer", plugin)
unformatted = "text\n"
formatted = plugin.TEXT_REPLACEMENT + "\n"
file1_path = tmp_path / "file1.md"

# Without TOML
file1_path.write_text(unformatted)
assert run((str(tmp_path),)) == 0
assert file1_path.read_text() == formatted

# With TOML
file1_path.write_text(unformatted)
config_path = tmp_path / ".mdformat.toml"
config_path.write_text("extensions = []")
assert run_with_clear_cache((str(tmp_path),)) == 0
assert file1_path.read_text() == unformatted
10 changes: 10 additions & 0 deletions tests/utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
from mdformat._cli import run
from mdformat._conf import read_toml_opts

UNFORMATTED_MARKDOWN = "\n\n# A header\n\n"
FORMATTED_MARKDOWN = "# A header\n"


def run_with_clear_cache(*args, **kwargs):
read_toml_opts.cache_clear()
return run(*args, **kwargs)