Skip to content

Commit

Permalink
Merge branch 'main' into imaginary-class-109218
Browse files Browse the repository at this point in the history
  • Loading branch information
skirpichev authored Jul 31, 2024
2 parents 7e68e91 + a9344cd commit 4664e4e
Show file tree
Hide file tree
Showing 173 changed files with 5,694 additions and 3,264 deletions.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@ Doc/c-api/stable.rst @encukou
**/*idlelib* @terryjreedy
/Doc/library/idle.rst @terryjreedy

**/*annotationlib* @JelleZijlstra
**/*typing* @JelleZijlstra @AlexWaygood

**/*ftplib @giampaolo
Expand Down
47 changes: 28 additions & 19 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -249,27 +249,38 @@ jobs:
arch: ${{ matrix.arch }}

build_macos:
name: 'macOS'
needs: check_source
if: needs.check_source.outputs.run_tests == 'true'
uses: ./.github/workflows/reusable-macos.yml
with:
config_hash: ${{ needs.check_source.outputs.config_hash }}
# Cirrus and macos-14 are M1, macos-13 is default GHA Intel.
# Cirrus used for upstream, macos-14 for forks.
os-matrix: '["ghcr.io/cirruslabs/macos-runner:sonoma", "macos-14", "macos-13"]'

build_macos_free_threading:
name: 'macOS (free-threading)'
name: >-
macOS
${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }}
needs: check_source
if: needs.check_source.outputs.run_tests == 'true'
strategy:
fail-fast: false
matrix:
# Cirrus and macos-14 are M1, macos-13 is default GHA Intel.
# macOS 13 only runs tests against the GIL-enabled CPython.
# Cirrus used for upstream, macos-14 for forks.
os:
- ghcr.io/cirruslabs/macos-runner:sonoma
- macos-14
- macos-13
is-fork: # only used for the exclusion trick
- ${{ github.repository_owner != 'python' }}
free-threading:
- false
- true
exclude:
- os: ghcr.io/cirruslabs/macos-runner:sonoma
is-fork: true
- os: macos-14
is-fork: false
- os: macos-13
free-threading: true
uses: ./.github/workflows/reusable-macos.yml
with:
config_hash: ${{ needs.check_source.outputs.config_hash }}
free-threading: true
# Cirrus and macos-14 are M1.
# Cirrus used for upstream, macos-14 for forks.
os-matrix: '["ghcr.io/cirruslabs/macos-runner:sonoma", "macos-14"]'
free-threading: ${{ matrix.free-threading }}
os: ${{ matrix.os }}

build_ubuntu:
name: >-
Expand Down Expand Up @@ -337,7 +348,7 @@ jobs:
with:
save: false
- name: Configure CPython
run: ./configure --config-cache --enable-slower-safety --with-pydebug --with-openssl=$OPENSSL_DIR
run: ./configure CFLAGS="-fdiagnostics-format=json" --config-cache --enable-slower-safety --with-pydebug --with-openssl=$OPENSSL_DIR
- name: Build CPython
run: make -j4
- name: Display build info
Expand Down Expand Up @@ -596,7 +607,6 @@ jobs:
- check-docs
- check_generated_files
- build_macos
- build_macos_free_threading
- build_ubuntu
- build_ubuntu_ssltests
- build_wasi
Expand Down Expand Up @@ -632,7 +642,6 @@ jobs:
&& '
check_generated_files,
build_macos,
build_macos_free_threading,
build_ubuntu,
build_ubuntu_ssltests,
build_wasi,
Expand Down
22 changes: 6 additions & 16 deletions .github/workflows/reusable-macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,14 @@ on:
required: false
type: boolean
default: false
os-matrix:
required: false
os:
description: OS to run the job
required: true
type: string

jobs:
build_macos:
name: build and test (${{ matrix.os }})
name: build and test (${{ inputs.os }})
timeout-minutes: 60
env:
HOMEBREW_NO_ANALYTICS: 1
Expand All @@ -23,18 +24,7 @@ jobs:
HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK: 1
PYTHONSTRICTEXTENSIONBUILD: 1
TERM: linux
strategy:
fail-fast: false
matrix:
os: ${{fromJson(inputs.os-matrix)}}
is-fork:
- ${{ github.repository_owner != 'python' }}
exclude:
- os: "ghcr.io/cirruslabs/macos-runner:sonoma"
is-fork: true
- os: "macos-14"
is-fork: false
runs-on: ${{ matrix.os }}
runs-on: ${{ inputs.os }}
steps:
- uses: actions/checkout@v4
- name: Runner image version
Expand All @@ -43,7 +33,7 @@ jobs:
uses: actions/cache@v4
with:
path: config.cache
key: ${{ github.job }}-${{ matrix.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}
key: ${{ github.job }}-${{ inputs.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}
- name: Install Homebrew dependencies
run: brew install pkg-config [email protected] xz gdbm tcl-tk
- name: Configure CPython
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/reusable-tsan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@ jobs:
# Install clang-18
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 18
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-18 100
sudo update-alternatives --set clang /usr/bin/clang-18
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-18 100
sudo update-alternatives --set clang++ /usr/bin/clang++-18
sudo ./llvm.sh 17 # gh-121946: llvm-18 package is temporarily broken
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-17 100
sudo update-alternatives --set clang /usr/bin/clang-17
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-17 100
sudo update-alternatives --set clang++ /usr/bin/clang++-17
# Reduce ASLR to avoid TSAN crashing
sudo sysctl -w vm.mmap_rnd_bits=28
- name: TSAN Option Setup
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/reusable-ubuntu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,17 +67,20 @@ jobs:
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: >-
../cpython-ro-srcdir/configure
CFLAGS="-fdiagnostics-format=json"
--config-cache
--with-pydebug
--enable-slower-safety
--with-openssl=$OPENSSL_DIR
${{ fromJSON(inputs.free-threading) && '--disable-gil' || '' }}
- name: Build CPython out-of-tree
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: make -j4
run: set -o pipefail; make -j4 2>&1 | tee compiler_output.txt
- name: Display build info
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: make pythoninfo
- name: Check compiler warnings
run: python Tools/build/check_warnings.py --compiler-output-file-path=${{ env.CPYTHON_BUILDDIR }}/compiler_output.txt --warning-ignore-file-path ${GITHUB_WORKSPACE}/Tools/build/.warningignore_ubuntu
- name: Remount sources writable for tests
# some tests write to srcdir, lack of pyc files slows down testing
run: sudo mount $CPYTHON_RO_SRCDIR -oremount,rw
Expand Down
25 changes: 19 additions & 6 deletions Android/android.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env python3

import argparse
from glob import glob
import os
import re
import shutil
Expand All @@ -16,16 +17,21 @@
CROSS_BUILD_DIR = CHECKOUT / "cross-build"


def delete_if_exists(path):
if path.exists():
def delete_glob(pattern):
# Path.glob doesn't accept non-relative patterns.
for path in glob(str(pattern)):
path = Path(path)
print(f"Deleting {path} ...")
shutil.rmtree(path)
if path.is_dir() and not path.is_symlink():
shutil.rmtree(path)
else:
path.unlink()


def subdir(name, *, clean=None):
path = CROSS_BUILD_DIR / name
if clean:
delete_if_exists(path)
delete_glob(path)
if not path.exists():
if clean is None:
sys.exit(
Expand Down Expand Up @@ -150,10 +156,17 @@ def configure_host_python(context):


def make_host_python(context):
# The CFLAGS and LDFLAGS set in android-env include the prefix dir, so
# delete any previously-installed Python libs and include files to prevent
# them being used during the build.
host_dir = subdir(context.host)
prefix_dir = host_dir / "prefix"
delete_glob(f"{prefix_dir}/include/python*")
delete_glob(f"{prefix_dir}/lib/libpython*")

os.chdir(host_dir / "build")
run(["make", "-j", str(os.cpu_count())], host=context.host)
run(["make", "install", f"prefix={host_dir}/prefix"], host=context.host)
run(["make", "install", f"prefix={prefix_dir}"], host=context.host)


def build_all(context):
Expand All @@ -164,7 +177,7 @@ def build_all(context):


def clean_all(context):
delete_if_exists(CROSS_BUILD_DIR)
delete_glob(CROSS_BUILD_DIR)


# To avoid distributing compiled artifacts without corresponding source code,
Expand Down
9 changes: 8 additions & 1 deletion Android/testbed/app/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,17 @@ plugins {

val PYTHON_DIR = File(projectDir, "../../..").canonicalPath
val PYTHON_CROSS_DIR = "$PYTHON_DIR/cross-build"

val ABIS = mapOf(
"arm64-v8a" to "aarch64-linux-android",
"x86_64" to "x86_64-linux-android",
)
).filter { File("$PYTHON_CROSS_DIR/${it.value}").exists() }
if (ABIS.isEmpty()) {
throw GradleException(
"No Android ABIs found in $PYTHON_CROSS_DIR: see Android/README.md " +
"for building instructions."
)
}

val PYTHON_VERSION = File("$PYTHON_DIR/Include/patchlevel.h").useLines {
for (line in it) {
Expand Down
9 changes: 2 additions & 7 deletions Doc/bugs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,16 @@ Documentation bugs
==================

If you find a bug in this documentation or would like to propose an improvement,
please submit a bug report on the :ref:`tracker <using-the-tracker>`. If you
please submit a bug report on the :ref:`issue tracker <using-the-tracker>`. If you
have a suggestion on how to fix it, include that as well.

You can also open a discussion item on our
`Documentation Discourse forum <https://discuss.python.org/c/documentation/26>`_.

If you find a bug in the theme (HTML / CSS / JavaScript) of the
documentation, please submit a bug report on the `python-doc-theme bug
documentation, please submit a bug report on the `python-doc-theme issue
tracker <https://github.com/python/python-docs-theme>`_.

If you're short on time, you can also email documentation bug reports to
[email protected] (behavioral bugs can be sent to [email protected]).
'docs@' is a mailing list run by volunteers; your request will be noticed,
though it may take a while to be processed.

.. seealso::

`Documentation bugs`_
Expand Down
1 change: 1 addition & 0 deletions Doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
# ---------------------

extensions = [
'audit_events',
'c_annotations',
'glossary_search',
'lexers',
Expand Down
1 change: 1 addition & 0 deletions Doc/contents.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
installing/index.rst
howto/index.rst
faq/index.rst
deprecations/index.rst
glossary.rst

about.rst
Expand Down
46 changes: 46 additions & 0 deletions Doc/deprecations/c-api-pending-removal-in-3.14.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
Pending Removal in Python 3.14
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* The ``ma_version_tag`` field in :c:type:`PyDictObject` for extension modules
(:pep:`699`; :gh:`101193`).

* Creating :c:data:`immutable types <Py_TPFLAGS_IMMUTABLETYPE>` with mutable
bases (:gh:`95388`).

* Functions to configure Python's initialization, deprecated in Python 3.11:

* ``PySys_SetArgvEx()``: set :c:member:`PyConfig.argv` instead.
* ``PySys_SetArgv()``: set :c:member:`PyConfig.argv` instead.
* ``Py_SetProgramName()``: set :c:member:`PyConfig.program_name` instead.
* ``Py_SetPythonHome()``: set :c:member:`PyConfig.home` instead.

The :c:func:`Py_InitializeFromConfig` API should be used with
:c:type:`PyConfig` instead.

* Global configuration variables:

* :c:var:`Py_DebugFlag`: use :c:member:`PyConfig.parser_debug` instead.
* :c:var:`Py_VerboseFlag`: use :c:member:`PyConfig.verbose` instead.
* :c:var:`Py_QuietFlag`: use :c:member:`PyConfig.quiet` instead.
* :c:var:`Py_InteractiveFlag`: use :c:member:`PyConfig.interactive` instead.
* :c:var:`Py_InspectFlag`: use :c:member:`PyConfig.inspect` instead.
* :c:var:`Py_OptimizeFlag`: use :c:member:`PyConfig.optimization_level` instead.
* :c:var:`Py_NoSiteFlag`: use :c:member:`PyConfig.site_import` instead.
* :c:var:`Py_BytesWarningFlag`: use :c:member:`PyConfig.bytes_warning` instead.
* :c:var:`Py_FrozenFlag`: use :c:member:`PyConfig.pathconfig_warnings` instead.
* :c:var:`Py_IgnoreEnvironmentFlag`: use :c:member:`PyConfig.use_environment` instead.
* :c:var:`Py_DontWriteBytecodeFlag`: use :c:member:`PyConfig.write_bytecode` instead.
* :c:var:`Py_NoUserSiteDirectory`: use :c:member:`PyConfig.user_site_directory` instead.
* :c:var:`Py_UnbufferedStdioFlag`: use :c:member:`PyConfig.buffered_stdio` instead.
* :c:var:`Py_HashRandomizationFlag`: use :c:member:`PyConfig.use_hash_seed`
and :c:member:`PyConfig.hash_seed` instead.
* :c:var:`Py_IsolatedFlag`: use :c:member:`PyConfig.isolated` instead.
* :c:var:`Py_LegacyWindowsFSEncodingFlag`: use :c:member:`PyPreConfig.legacy_windows_fs_encoding` instead.
* :c:var:`Py_LegacyWindowsStdioFlag`: use :c:member:`PyConfig.legacy_windows_stdio` instead.
* :c:var:`!Py_FileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding` instead.
* :c:var:`!Py_HasFileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding` instead.
* :c:var:`!Py_FileSystemDefaultEncodeErrors`: use :c:member:`PyConfig.filesystem_errors` instead.
* :c:var:`!Py_UTF8Mode`: use :c:member:`PyPreConfig.utf8_mode` instead. (see :c:func:`Py_PreInitialize`)

The :c:func:`Py_InitializeFromConfig` API should be used with
:c:type:`PyConfig` instead.
20 changes: 20 additions & 0 deletions Doc/deprecations/c-api-pending-removal-in-3.15.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
Pending Removal in Python 3.15
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* The bundled copy of ``libmpdecimal``.
* :c:func:`PyImport_ImportModuleNoBlock`: use :c:func:`PyImport_ImportModule` instead.
* :c:func:`PyWeakref_GET_OBJECT`: use :c:func:`PyWeakref_GetRef` instead.
* :c:func:`PyWeakref_GetObject`: use :c:func:`PyWeakref_GetRef` instead.
* :c:type:`!Py_UNICODE_WIDE` type: use :c:type:`wchar_t` instead.
* :c:type:`Py_UNICODE` type: use :c:type:`wchar_t` instead.
* Python initialization functions:

* :c:func:`PySys_ResetWarnOptions`: clear :data:`sys.warnoptions` and
:data:`!warnings.filters` instead.
* :c:func:`Py_GetExecPrefix`: get :data:`sys.exec_prefix` instead.
* :c:func:`Py_GetPath`: get :data:`sys.path` instead.
* :c:func:`Py_GetPrefix`: get :data:`sys.prefix` instead.
* :c:func:`Py_GetProgramFullPath`: get :data:`sys.executable` instead.
* :c:func:`Py_GetProgramName`: get :data:`sys.executable` instead.
* :c:func:`Py_GetPythonHome`: get :c:member:`PyConfig.home` or
the :envvar:`PYTHONHOME` environment variable instead.
31 changes: 31 additions & 0 deletions Doc/deprecations/c-api-pending-removal-in-future.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
Pending Removal in Future Versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The following APIs are deprecated and will be removed,
although there is currently no date scheduled for their removal.

* :c:macro:`Py_TPFLAGS_HAVE_FINALIZE`: unneeded since Python 3.8.
* :c:func:`PyErr_Fetch`: use :c:func:`PyErr_GetRaisedException` instead.
* :c:func:`PyErr_NormalizeException`: use :c:func:`PyErr_GetRaisedException` instead.
* :c:func:`PyErr_Restore`: use :c:func:`PyErr_SetRaisedException` instead.
* :c:func:`PyModule_GetFilename`: use :c:func:`PyModule_GetFilenameObject` instead.
* :c:func:`PyOS_AfterFork`: use :c:func:`PyOS_AfterFork_Child` instead.
* :c:func:`PySlice_GetIndicesEx`: use :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices` instead.
* :c:func:`!PyUnicode_AsDecodedObject`: use :c:func:`PyCodec_Decode` instead.
* :c:func:`!PyUnicode_AsDecodedUnicode`: use :c:func:`PyCodec_Decode` instead.
* :c:func:`!PyUnicode_AsEncodedObject`: use :c:func:`PyCodec_Encode` instead.
* :c:func:`!PyUnicode_AsEncodedUnicode`: use :c:func:`PyCodec_Encode` instead.
* :c:func:`PyUnicode_READY`: unneeded since Python 3.12
* :c:func:`!PyErr_Display`: use :c:func:`PyErr_DisplayException` instead.
* :c:func:`!_PyErr_ChainExceptions`: use ``_PyErr_ChainExceptions1`` instead.
* :c:member:`!PyBytesObject.ob_shash` member:
call :c:func:`PyObject_Hash` instead.
* :c:member:`!PyDictObject.ma_version_tag` member.
* Thread Local Storage (TLS) API:

* :c:func:`PyThread_create_key`: use :c:func:`PyThread_tss_alloc` instead.
* :c:func:`PyThread_delete_key`: use :c:func:`PyThread_tss_free` instead.
* :c:func:`PyThread_set_key_value`: use :c:func:`PyThread_tss_set` instead.
* :c:func:`PyThread_get_key_value`: use :c:func:`PyThread_tss_get` instead.
* :c:func:`PyThread_delete_key_value`: use :c:func:`PyThread_tss_delete` instead.
* :c:func:`PyThread_ReInitTLS`: unneeded since Python 3.7.
Loading

0 comments on commit 4664e4e

Please sign in to comment.