Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a public API #472

Closed
wants to merge 65 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
d5969f0
Replaced old WheelFile with the proposed new one
agronholm Jun 9, 2020
a388212
Added support for using WheelFile with existing file handles
agronholm Jun 10, 2020
376cf99
Use posixpath.join() to put together archive names
agronholm Jun 18, 2020
b03dde9
Removed PathLike support from write_file and added the timestamp argu…
agronholm Jun 18, 2020
0fd872d
Don't test on Python 2.7
agronholm Jun 18, 2020
eb0e459
Removed the write_files_from_directory() method
agronholm Jun 18, 2020
fac7e16
Renamed write_metadata_file() to write_distinfo_file()
agronholm Jun 18, 2020
fe39329
Renamed read_metadata_file() to read_distinfo_file()
agronholm Jun 18, 2020
96b88b5
Reverted the use of posixpath.join()
agronholm Jun 18, 2020
3a3d731
Removed more Python 2 configuration
agronholm Jun 18, 2020
a975348
Added special support for the METADATA file
agronholm Jun 18, 2020
f79c25e
Removed the test workflow trigger for PRs
agronholm Jun 21, 2020
b2d7029
Refactored bdist_wheel to use the new WheelFile API
agronholm Jun 21, 2020
e7fe9bb
Refactored test suite and CLI tools to use tmp_path
agronholm Jun 21, 2020
d7b1f1d
Added the py.typed marker
agronholm Jun 21, 2020
4654492
Removed obsolete modules
agronholm Jun 21, 2020
6591155
Added type annotations to macosx_libfile
agronholm Jun 21, 2020
3667709
Fixed a few WheelFile tests
agronholm Jun 21, 2020
9e63bec
Restored Python 3.5 compatibility to WheelFile
agronholm Jun 21, 2020
cbca67e
Added Python 3.7 to the testing matrix
agronholm Jun 21, 2020
bf0afca
Updated the Github publish workflow
agronholm Jun 22, 2020
71474a5
Updated the Github codeqa/test workflow
agronholm Jun 24, 2020
da30173
Merge branch 'master' into publicapi
agronholm Dec 20, 2020
ec0aded
Removed Python 3.5 support
agronholm Feb 8, 2021
f1a5fd4
WIP refactored the Wheel API
agronholm Feb 8, 2021
fa01be4
Merge branch 'main' into publicapi
agronholm Oct 22, 2022
a2a53de
Improved the API and dropped bdist_wininst convert support
agronholm Oct 24, 2022
34f05ae
Removed install dependency on setuptools
agronholm Oct 24, 2022
85ac7c5
Updated version number
agronholm Oct 24, 2022
e3934a9
Switched to running coverage using "python -m"
agronholm Oct 24, 2022
946f7f9
Fixed no files beside .egg-info being added by bdist_wheel
agronholm Oct 24, 2022
b69c2bc
Fixed test_write_file
agronholm Oct 24, 2022
a2ca178
Added setuptools as test dependency
agronholm Oct 24, 2022
c7167fe
Fixed deprecated pytest imports
agronholm Oct 24, 2022
f4aff15
Made most wheel modules private
agronholm Oct 24, 2022
29a5dfc
Made all remaining modules private
agronholm Oct 24, 2022
9f268c3
Changed the default value for amount in WheelArchiveFile.read() to -1
agronholm Oct 24, 2022
c85a333
Fixed import in wheel.__main__
agronholm Oct 24, 2022
31d0dc9
Fixed the most critical type issues and added a MyPy check to CI (#473)
HexDecimal Oct 25, 2022
81471ec
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 25, 2022
ef1425d
Removed unused module
agronholm Oct 25, 2022
71efa9b
Updated the docs for "wheel convert"
agronholm Oct 25, 2022
c2171c2
Updated CLI help, type annotations and string interpolation
agronholm Oct 25, 2022
122f456
Made WheelContentElement into a named tuple
agronholm Oct 25, 2022
d14b5fc
Removed obsolete options
agronholm Oct 25, 2022
9f2bfc6
Fixed mypy error
agronholm Oct 25, 2022
edcb325
Added tests to type checking and finish annotations. (#477)
HexDecimal Oct 26, 2022
914cc81
Merged the license_paths property back to run()
agronholm Oct 25, 2022
cad0948
Merge branch 'main' into publicapi
agronholm Oct 26, 2022
ea9086a
Removed obsolete test
agronholm Oct 27, 2022
13ae2b6
Re-added test accidentally removed during merge
agronholm Oct 27, 2022
21256df
Merge branch 'main' into publicapi
agronholm Nov 6, 2022
b432977
Moved the bdist_wheel module back to its original location
agronholm Nov 7, 2022
14ca7de
Renamed test() to validate_record()
agronholm Nov 26, 2022
ee91735
Merge branch 'main' into publicapi
agronholm Nov 26, 2022
d5c7dc3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 26, 2022
eb12145
Added a fix for #489
agronholm Nov 26, 2022
91d0e2e
Fixed cache not being used on Windows and macOS in the test workflow
agronholm Nov 26, 2022
9a18f66
Merge branch 'main' into publicapi
agronholm Dec 10, 2022
d783f6b
Added compatibility shim for wheel.wheelfile
agronholm Dec 10, 2022
ff85f76
Added automatic detection of .dist-info directory
agronholm Dec 11, 2022
6892de4
Refactored code to make mypy happy
agronholm Dec 11, 2022
513f1f7
Refactored pkg_resources imports
agronholm Dec 11, 2022
09c5f64
Fixed the rest of mypy errors
agronholm Dec 11, 2022
59fd006
Added a standalone function for writing a WHEEL file
agronholm Dec 11, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@ name: Publish packages to PyPI

on:
create:
tags: "*"
tags:
- "[0-9]+.[0-9]+.[0-9]+"
- "[0-9]+.[0-9]+.[0-9]+[a-b][0-9]+"
- "[0-9]+.[0-9]+.[0-9]+rc[0-9]+"

jobs:
publish:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')
steps:
- uses: actions/checkout@v3
- name: Set up Python
Expand Down
25 changes: 15 additions & 10 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,19 +35,24 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- uses: actions/cache@v3
with:
path: ~/.cache/pip
key: pip-test-${{ matrix.python-version }}-${{ matrix.os }}
- name: Install the project
run: "pip install --no-binary=:all: ."
- name: Install test dependencies
run: pip install .[test] coverage[toml]
cache: pip
cache-dependency-path: setup.cfg
- name: Install the project and its test dependencies
run: pip install --no-binary=setuptools,wheel .[test] coverage[toml]
- name: Test with pytest
run: |
coverage run -m pytest -W always
coverage xml
python -m coverage run -m pytest -W always
python -m coverage xml
- name: Send coverage data to Codecov
uses: codecov/codecov-action@v3
with:
file: coverage.xml

mypy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install the project and its Mypy dependencies
run: pip install --no-binary=setuptools,wheel .[types]
- name: Test with Mypy
run: mypy
2 changes: 2 additions & 0 deletions docs/news.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ Release Notes

**UNRELEASED**

- Added a public API
- Dropped support for converting ``bdist_wininst`` based installers into wheels
- Updated vendored ``packaging`` to 22.0

**0.38.4 (2022-11-09)**
Expand Down
5 changes: 3 additions & 2 deletions docs/reference/wheel_convert.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,15 @@ Usage
Description
-----------

Convert one or more eggs (``.egg``; made with ``bdist_egg``) or Windows
installers (``.exe``; made with ``bdist_wininst``) into wheels.
Convert one or more eggs (``.egg``; made with ``bdist_egg``) into wheels.

Egg names must match the standard format:

* ``<project>-<version>-pyX.Y`` for pure Python wheels
* ``<project>-<version>-pyX.Y-<arch>`` for binary wheels

Each argument can be either an ``.egg`` file, an unpacked egg directory or a directory
containing eggs (packed or unpacked).

Options
-------
Expand Down
20 changes: 18 additions & 2 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,16 @@ where = src

[options.extras_require]
test =
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logically it makes sense to separate these out but is is necessarily worth it? The additional install time is so trivial for the small amount of dependencies so I'm just not sure its really worth the split. It also makes the CI caching harder.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are you talking about?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, you mean the separation of test and typing dependencies? That wasn't actually done by me, and I'm still not 100% convinced that we shouldn't be using pre-commit for that. It's not critical either way, and I don't see any particular issues that it would cause for caching. I could of course be convinced otherwise.

pytest >= 3.0.0
pytest >= 6.2.0
setuptools >= 57
types =
mypy >= 0.982
pytest >= 6.2
types-setuptools >= 65.5.0.2

[options.entry_points]
console_scripts =
wheel = wheel.cli:main
wheel = wheel._cli:main
distutils.commands =
bdist_wheel = wheel.bdist_wheel:bdist_wheel

Expand All @@ -57,10 +62,21 @@ max-line-length = 88

[tool:pytest]
testpaths = tests
addopts = --tb=short
filterwarnings =
ignore::DeprecationWarning:wheel.wheelfile

[coverage:run]
source = wheel
omit = */vendored/*

[coverage:report]
show_missing = true

[mypy]
files = src,tests
python_version = 3.8
exclude = testdata/

[mypy-wheel.vendored.*]
ignore_errors = True
17 changes: 16 additions & 1 deletion src/wheel/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@
from __future__ import annotations

__version__ = "0.38.4"
__all__ = [
"WheelError",
"WheelReader",
"WheelWriter",
"make_filename",
"write_wheelfile",
]
__version__ = "1.0.0a1"

from ._wheelfile import (
WheelError,
WheelReader,
WheelWriter,
make_filename,
write_wheelfile,
)
9 changes: 5 additions & 4 deletions src/wheel/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,20 @@
from __future__ import annotations

import sys
from typing import NoReturn


def main(): # needed for console script
def main() -> NoReturn: # needed for console script
if __package__ == "":
# To be able to run 'python wheel-0.9.whl/wheel':
import os.path

path = os.path.dirname(os.path.dirname(__file__))
sys.path[0:0] = [path]
import wheel.cli
import wheel._cli

sys.exit(wheel.cli.main())
sys.exit(wheel._cli.main())


if __name__ == "__main__":
sys.exit(main())
main()
24 changes: 13 additions & 11 deletions src/wheel/cli/__init__.py → src/wheel/_cli/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,41 +4,41 @@

from __future__ import annotations

import argparse
import os
import sys
from argparse import ArgumentParser, Namespace


class WheelError(Exception):
pass


def unpack_f(args):
def unpack_f(args: Namespace) -> None:
from .unpack import unpack

unpack(args.wheelfile, args.dest)


def pack_f(args):
def pack_f(args: Namespace) -> None:
from .pack import pack

pack(args.directory, args.dest_dir, args.build_number)


def convert_f(args):
def convert_f(args: Namespace) -> None:
from .convert import convert

convert(args.files, args.dest_dir, args.verbose)


def version_f(args):
def version_f(args: Namespace) -> None:
from .. import __version__

print("wheel %s" % __version__)
print(f"wheel {__version__}")


def parser():
p = argparse.ArgumentParser()
def parser() -> ArgumentParser:
p = ArgumentParser()
s = p.add_subparsers(help="commands")

unpack_parser = s.add_parser("unpack", help="Unpack wheel")
Expand All @@ -61,8 +61,10 @@ def parser():
)
repack_parser.set_defaults(func=pack_f)

convert_parser = s.add_parser("convert", help="Convert egg or wininst to wheel")
convert_parser.add_argument("files", nargs="*", help="Files to convert")
convert_parser = s.add_parser("convert", help="Convert eggs to wheels")
convert_parser.add_argument(
"files", nargs="*", help=".egg files or directories to convert"
)
convert_parser.add_argument(
"--dest-dir",
"-d",
Expand All @@ -81,7 +83,7 @@ def parser():
return p


def main():
def main() -> int:
p = parser()
args = p.parse_args()
if not hasattr(args, "func"):
Expand Down
126 changes: 126 additions & 0 deletions src/wheel/_cli/convert.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
from __future__ import annotations

import os
import re
import zipfile
from collections.abc import Generator, Iterable
from email.message import Message
from email.parser import HeaderParser
from os import PathLike
from pathlib import Path, PurePath
from typing import IO, Any

from .. import WheelWriter, make_filename
from . import WheelError

egg_info_re = re.compile(
r"""
(?P<name>.+?)-(?P<ver>.+?)
(-(?P<pyver>py\d\.\d+)
(-(?P<arch>.+?))?
)?.egg$""",
re.VERBOSE,
)


def egg2wheel(egg_path: Path, dest_dir: Path) -> None:
def egg_file_source() -> Generator[tuple[PurePath, IO[bytes]], Any, None]:
with zipfile.ZipFile(egg_path) as zf:
for zinfo in zf.infolist():
with zf.open(zinfo) as fp:
yield PurePath(zinfo.filename), fp

def egg_dir_source() -> Generator[tuple[PurePath, IO[bytes]], Any, None]:
for root, _dirs, files in os.walk(egg_path, followlinks=False):
root_path = Path(root)
for fname in files:
file_path = root_path / fname
with file_path.open("rb") as fp:
yield file_path, fp

match = egg_info_re.match(egg_path.name)
if not match:
raise WheelError(f"Invalid egg file name: {egg_path.name}")

# Assume pure Python if there is no specified architecture
# Assume all binary eggs are for CPython
egg_info = match.groupdict()
pyver = egg_info["pyver"].replace(".", "")
arch = (egg_info["arch"] or "any").replace(".", "_").replace("-", "_")
abi = "cp" + pyver[2:] if arch != "any" else "none"
root_is_purelib = arch is None

if egg_path.is_dir():
# buildout-style installed eggs directory
source = egg_dir_source()
else:
source = egg_file_source()

wheel_name = make_filename(
egg_info["name"], egg_info["ver"], impl_tag=pyver, abi_tag=abi, plat_tag=arch
)
metadata = Message()
with WheelWriter(
dest_dir / wheel_name, generator="egg2wheel", root_is_purelib=root_is_purelib
) as wf:
for path, fp in source:
if path.parts[0] == "EGG-INFO":
if path.parts[1] == "requires.txt":
requires = fp.read().decode("utf-8")
extra = specifier = ""
for line in requires.splitlines():
line = line.strip()
if line.startswith("[") and line.endswith("]"):
extra, _, specifier = line[1:-1].strip().partition(":")
metadata["Provides-Extra"] = extra
elif line:
specifiers: list[str] = []
if extra:
specifiers += f"extra == {extra!r}"

if specifier:
specifiers += specifier

if specifiers:
line = line + " ; " + " and ".join(specifiers)

metadata["Requires-Dist"] = line
elif path.parts[1] in ("entry_points.txt", "top_level.txt"):
wf.write_distinfo_file(path.parts[1], fp)
elif path.parts[1] == "PKG-INFO":
pkg_info = HeaderParser().parsestr(fp.read().decode("utf-8"))
pkg_info.replace_header("Metadata-Version", "2.1")
del pkg_info["Provides-Extra"]
del pkg_info["Requires-Dist"]
for header, value in pkg_info.items():
metadata[header] = value
else:
wf.write_file(path, fp)

if metadata:
wf.write_metadata(metadata.items())


def convert(
files: Iterable[str | PathLike[str]], dest_dir: str | PathLike[str], verbose: bool
) -> None:
dest_path = Path(dest_dir)
paths: list[Path] = []
for fname in files:
path = Path(fname)
if path.is_file():
paths.append(path)
elif path.is_dir():
paths.extend(path.iterdir())

for path in paths:
if path.suffix != ".egg":
continue

if verbose:
print(f"{path}... ", flush=True)

egg2wheel(path, dest_path)

if verbose:
print("OK")
Loading