Merge bitcoin#24263: doc: Fix gen-manpages, rewrite in Python

87f5406 doc: Swap gen-manpages and update RC steps in release process (laanwj)
42c2028 doc: Fix gen-manpages, rewrite in Python (laanwj)

Pull request description:

  Rewrite the manual page generation script in Python.

  This:
  - solves '-' stripping issue (fixes bitcoin#22681)
  - makes that a copyright footer is generated correctly again

  Also change the release process to swap gen-manpages and update RC steps, so that the pages will have the correct rc and/or final version.

ACKs for top commit:
  dongcarl:
    Code Review ACK 87f5406
  fanquake:
    ACK 87f5406 - tested generating and opening the man pages locally, but didn't run through the release process. Will propose some changes to address consolidating the help / version output.

Tree-SHA512: 39254721ca84e4f223a321c554f2e08c36428b15019a0f9fa3eff408b4c6f1e1d74941143f4d2927427afa3ad7a7e6f999d6ec660132d817809b640a87ae9f7d

contrib/devtools/README.md

@@ -76,7 +76,7 @@ year rather than two hyphenated years.
 If the file already has a copyright for `The Bitcoin Core developers`, the
 script will exit.
 
-gen-manpages.sh
+gen-manpages.py
 ===============
 
 A small script to automatically create manpages in ../../doc/man by running the release binaries with the -help option.
@@ -87,7 +87,7 @@ repostitory. To use this tool with out-of-tree builds set `BUILDDIR`. For
 example:
 
 ```bash
-BUILDDIR=$PWD/build contrib/devtools/gen-manpages.sh
+BUILDDIR=$PWD/build contrib/devtools/gen-manpages.py
 ```
 
 security-check.py and test-security-check.py

contrib/devtools/gen-manpages.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+# Copyright (c) 2022 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+import os
+import subprocess
+import sys
+import tempfile
+
+BINARIES = [
+'src/bitcoind',
+'src/bitcoin-cli',
+'src/bitcoin-tx',
+'src/bitcoin-wallet',
+'src/bitcoin-util',
+'src/qt/bitcoin-qt',
+]
+
+# Paths to external utilities.
+git = os.getenv('GIT', 'git')
+help2man = os.getenv('HELP2MAN', 'help2man')
+
+# If not otherwise specified, get top directory from git.
+topdir = os.getenv('TOPDIR')
+if not topdir:
+    r = subprocess.run([git, 'rev-parse', '--show-toplevel'], stdout=subprocess.PIPE, check=True, universal_newlines=True)
+    topdir = r.stdout.rstrip()
+
+# Get input and output directories.
+builddir = os.getenv('BUILDDIR', topdir)
+mandir = os.getenv('MANDIR', os.path.join(topdir, 'doc/man'))
+
+# Verify that all the required binaries are usable, and extract copyright
+# message in a first pass.
+copyright = None
+versions = []
+for relpath in BINARIES:
+    abspath = os.path.join(builddir, relpath)
+    try:
+        r = subprocess.run([abspath, '--version'], stdout=subprocess.PIPE, universal_newlines=True)
+    except IOError:
+        print(f'{abspath} not found or not an executable', file=sys.stderr)
+        sys.exit(1)
+    # take first line (which must contain version)
+    verstr = r.stdout.split('\n')[0]
+    # last word of line is the actual version e.g. v22.99.0-5c6b3d5b3508
+    verstr = verstr.split()[-1]
+    assert verstr.startswith('v')
+
+    # Only bitcoin-qt prints the copyright message on --version, so store it specifically.
+    if relpath == 'src/qt/bitcoin-qt':
+        copyright = r.stdout.split('\n')[1:]
+
+    versions.append((abspath, verstr))
+
+if any(verstr.endswith('-dirty') for (_, verstr) in versions):
+    print("WARNING: Binaries were built from a dirty tree.")
+    print('man pages generated from dirty binaries should NOT be committed.')
+    print('To properly generate man pages, please commit your changes (or discard them), rebuild, then run this script again.')
+    print()
+
+with tempfile.NamedTemporaryFile('w', suffix='.h2m') as footer:
+    # Create copyright footer, and write it to a temporary include file.
+    assert copyright
+    footer.write('[COPYRIGHT]\n')
+    footer.write('\n'.join(copyright).strip())
+    footer.flush()
+
+    # Call the binaries through help2man to produce a manual page for each of them.
+    for (abspath, verstr) in versions:
+        outname = os.path.join(mandir, os.path.basename(abspath) + '.1')
+        print(f'Generating {outname}…')
+        subprocess.run([help2man, '-N', '--version-string=' + verstr, '--include=' + footer.name, '-o', outname, abspath], check=True)

doc/release-process.md

@@ -6,13 +6,14 @@ Release Process
 ### Before every release candidate
 
 * Update translations see [translation_process.md](https://github.com/bitcoin/bitcoin/blob/master/doc/translation_process.md#synchronising-translations).
-* Update manpages, see [gen-manpages.sh](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-manpagessh).
 * Update release candidate version in `configure.ac` (`CLIENT_VERSION_RC`).
+* Update manpages (after rebuilding the binaries), see [gen-manpages.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-manpagespy).
 
 ### Before every major and minor release
 
 * Update [bips.md](bips.md) to account for changes since the last release (don't forget to bump the version number on the first line).
 * Update version in `configure.ac` (don't forget to set `CLIENT_VERSION_RC` to `0`).
+* Update manpages (see previous section)
 * Write release notes (see "Write the release notes" below).
 
 ### Before every major release