Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TOTHINK: In git snapshot rebuilds, suffix NUT version with number of commits since release #1949

Closed
jimklimov opened this issue May 26, 2023 · 7 comments
Labels
CI Entries related to continuous integration infrastructure (historically also recipes like Makefiles) enhancement packaging
Milestone

Comments

@jimklimov
Copy link
Member

jimklimov commented May 26, 2023

Autoconf tricks to allow embedding the git version are explored in autoconf's own build scripts:

A practical implementation example (not with git version, but still - with a command) can be seen at https://github.com/exfatprogs/exfatprogs/blob/master/configure.ac#L3-L11 or MidnightCommander/mc@eab8439 with a fully-fledged script.

The idea here might be to determine a release tag (annotated? by pattern?) as opposed to major milestones, release candidates etc. tagged during development), and count the number of commits since then.

A release might be tagged v2.8.0 (disregarding items like v2.8.0-Windows, Windows-v2.8.0-alpha4 or v2.8.1-rc1 which might have happened later, but maybe including patterns like v2.8.0-signed (which should refer to same commit as original v2.8.0 but with a signed tag object) and then the custom build would be configured as AC_INIT([nut], [2.8.0.123]...) for 123 commits after that release.

If this tweak to metadata works, it would be more idiomatic (and welcoming to rolling-release packaging form master branch by CI, for example) than calling all such custom builds formally 2.8.0.1 and detailing the git maturity level in plain comments or help messages which are not easily machine-parsable.

For bonus points, we may want to track the codebase maturity compared to both master and last release, perhaps 2.8.0.123.45 meaning that "master" branch had 123 commits since last release (2.8.0) and then the currently branched-off effort for a PR added 45 commits on top of that newest commit in the PR branch that is also in known master branch history (the branching or last-resync point), or alternately - just say 345 commits tracked in the PR branch overall when also counting since that release, which may be easier to tackle.

Then a build from newer master branch with 130 commits (or of a PR with foundations on that baseline) would win in terms of automated packaging version resolution, etc. despite the longer (and variable) amount of commits in development branches.

By either count, a release (tag, tarball) would then be pedantically (equivalent to) 2.8.1.0 or 2.8.1.0.0; for most packaging systems trailing zeroes may be ignored so 2.8.1 as usual.

UPDATE from Jul 2024: In current configure.ac we track AC_INIT with a fixed version string (e.g. 2.8.2 committed manually when making a release, and 2.8.2.1 persisting for development versions), and separately define NUT_SOURCE_GITREV via shell script parsing of tailored git describe output. This is further stripped into NUT_SOURCE_GITREV_NUMERIC substituted into e.g. PyPI packaging manifest:

:; grep NUT_SOU config.log
NUT_SOURCE_GITREV='2.8.2-3880-g118c427c7'
NUT_SOURCE_GITREV_NUMERIC='2.8.2.3880'

Elaborating on the idea of tracking two version increments on top of a formal release - of trunk (master) since last release and of current code since trunk, so that snapshot package versions are incremental and easy to upgrade:

  • As a trick seen from mc recipe linked above, --abbrev=0 can be used to strip the commit count and current hash, leaving just the nearest git tag name.
  • Note in configure.ac we add a lot of arguments to strip non-release tags from consideration.
  • Code snippet to experiment with:
getver() (TAG="`git describe --match 'v[0-9]*.[0-9]*.[0-9]' --exclude '*-signed' --exclude '*rc*' --exclude '*alpha*' --exclude '*beta*' --exclude '*Windows*' --exclude '*IPM*' --always --abbrev=0`";
echo "${TAG/v}.`git log --oneline "$TAG"..master | wc -l`.`git log --oneline master..HEAD | wc -l`" | sed -e 's/\.0$//' -e 's/\.0$//'; )
  • As an example, the FTY branch has so far thousands of DMF and other commits that are not part of master-branch history. Even though it is lately regularly merged-into from master, there are many commits seen as unique to it (thanks to backports per Upstreaming improvements from 42ity/nut fork #1316 many of these bring no diff's in content, just metadata). So the current state in one PR branch is NUT release v2.8.2 as baseline (the TAG), with current master adding 695 commits on top of that, and the branch having 3200 commits different from master:
nut-DMF$ getver
2.8.2.695.3200
  • Same expression for a release, with intentionally stripped one or two dot-zeroes in the end of line:
:; git checkout master
:; getver
2.8.2.695

:; git tag -a v2.8.3 -m test
:; getver
2.8.3

:; git tag -d v2.8.3

UPDATE2: Looking at "git merge-base" of current HEAD and known master trunk history, as the branch-off point (or just an older tag or other commit present in trunk) yields reasonable results:

getver() (TRUNK=master;
TAG="`git describe --match 'v[0-9]*.[0-9]*.[0-9]' --exclude '*-signed' --exclude '*rc*' --exclude '*alpha*' --exclude '*beta*' --exclude '*Windows*' --exclude '*IPM*' --always --abbrev=0`";
BASE="`git merge-base HEAD $TRUNK`" ;
echo "${TAG/v}.`git log --oneline $TAG..$BASE | wc -l`.`git log --oneline $TRUNK..HEAD | wc -l`" | sed -e 's/\.0$//' -e 's/\.0$//'; )

# PR branch recently synced from trunk
:; git checkout issue-2450 && getver
2.8.2.695.60

# Recent abandoned PR (current NUT release tag as predecessor, fewer trunk commits in between)
:; git checkout issue-1316-str_concat && getver
Switched to a new branch 'issue-1316-str_concat'
2.8.2.338.1

# Older abandoned PR, with one commit (and older NUT release tag as predecessor)
:; git checkout issue-1754-bcmxcp && getver
Switched to a new branch 'issue-1754-bcmxcp'
2.8.1.328.1

# Older commit ON trunk
git checkout master~30 && getver
HEAD is now at c91684f4b Merge pull request #2526 from jimklimov/issue-2523-cleanup
2.8.2.580

# Older tag
:; git checkout v2.8.1 && getver
HEAD is now at 4ba352d8f configure.ac: update AC_INIT for NUT v2.8.1 release
2.8.1

Primary question would rather be how to reconcile any such value derived from git with static offline-only files available in tarballs (return to some version.txt?); perhaps have a default-version file with e.g. 2.8.2.1 as used in AC_INIT now, tracked in SCM (and tarballs by extension) and updated from autogen.sh or configure.ac only if git is usable. Note that building or developing based on code from tarballs without git, and re-generating with autogen.sh along the way, is a valid scenario.

CC @clepple @aquette : WDYT?

@jimklimov jimklimov added enhancement CI Entries related to continuous integration infrastructure (historically also recipes like Makefiles) labels May 26, 2023
@jimklimov jimklimov added this to the 2.8.2 milestone May 26, 2023
@dak180
Copy link

dak180 commented May 29, 2023

Autorevision is made for doing exactly this and there are even examples using make.

@jimklimov
Copy link
Member Author

Looks neat, thanks!

@jimklimov
Copy link
Member Author

Looking around, I saw we have precedent of a similar approach in configuration of nut-website at https://github.com/networkupstools/nut-website/blob/64361632b90ed427b5bc2e364e37ccdc8118ffe6/configure.ac#L5

Granted, changes or mishaps there are less intrusive (no end-users or tarballs to get wrong) in case of mistakes.

@jimklimov

This comment was marked as duplicate.

@jimklimov jimklimov modified the milestones: 2.8.2, 2.8.3 Apr 5, 2024
@jimklimov
Copy link
Member Author

jimklimov commented Jul 24, 2024

Variant of the above updates, with debugging and restored tail of the git describe (commit count since tag and current hash):

getver() (
    [ x"${TRUNK-}" != x ] || TRUNK=master;
    DESC="`git describe --match 'v[0-9]*.[0-9]*.[0-9]' --exclude '*-signed' --exclude '*rc*' --exclude '*alpha*' --exclude '*beta*' --exclude '*Windows*' --exclude '*IPM*' --always`";
    TAG="`echo "${DESC}" | sed 's/-[0-9][0-9]*-g[0-9a-fA-F][0-9a-fA-F]*$//'`";
    SUFFIX="`echo "${DESC}" | sed 's/^.*\(-[0-9][0-9]*-g[0-9a-fA-F][0-9a-fA-F]*\)$/\1/'`" && [ x"${SUFFIX}" != x"${TAG}" ] || SUFFIX="";
    BASE="`git merge-base HEAD "${TRUNK}"`" ;
    VER5="${TAG#v}.`git log --oneline "${TAG}..${BASE}" | wc -l | tr -d ' '`.`git log --oneline "${TRUNK}..HEAD" | wc -l | tr -d ' '`";
    DESC5="${VER5}${SUFFIX}";
    VER50="`echo "${VER5}" | sed -e 's/\.0$//' -e 's/\.0$//'`";
    DESC50="${VER50}${SUFFIX}";
    echo "DESC='${DESC}' => TAG='${TAG}' + SUFFIX='${SUFFIX}'; BASE='${BASE}' => VER5='${VER5}' => VER50='${VER50}' => DESC50='${DESC50}'" >&2
    # echo "${DESC5}"
    echo "${DESC50}"
)

Checked with bash 3 and 5, dash, ksh, zsh and even busybox sh; OpenIndiana, FreeBSD and OpenBSD sh. Not compatible with csh and tcsh.

Some experiments (output line-wrapped):

# RELEASE
:; git checkout v2.8.2 && getver >/dev/null
DESC='v2.8.2' => TAG='v2.8.2' + SUFFIX=''; BASE='440ca2348e665abf3787c30bdcd9373b479f4efc' 
=> VER5='2.8.2.0.0' => VER50='2.8.2' 
=> DESC50='2.8.2'

### Development snapshots
:; git checkout FTY && getver >/dev/null
DESC='v2.8.2-3854-g8fa89a270' => TAG='v2.8.2' + SUFFIX='-3854-g8fa89a270'; BASE='7dcc938bdc9b26a2e9b861940f70d0e17d3c9921' 
=> VER5='2.8.2.680.3174' => VER50='2.8.2.680.3174' 
=> DESC50='2.8.2.680.3174-3854-g8fa89a270'

:; git checkout master~20 && getver >/dev/null
DESC='v2.8.2-655-g91ea6270b' => TAG='v2.8.2' + SUFFIX='-655-g91ea6270b'; BASE='91ea6270b287220a22cc1fd1a6de420b026d0c6c' 
=> VER5='2.8.2.655.0' => VER50='2.8.2.655' 
=> DESC50='2.8.2.655-655-g91ea6270b'

:; git checkout issue-1316-str_concat && getver >/dev/null
DESC='v2.8.2-339-ga1d3b8fe9' => TAG='v2.8.2' + SUFFIX='-339-ga1d3b8fe9'; BASE='dc099e8e57729a221e8343fdf1449ee0466e06a3' 
=> VER5='2.8.2.338.1' => VER50='2.8.2.338.1' 
=> DESC50='2.8.2.338.1-339-ga1d3b8fe9'

:; git checkout issue-1754-bcmxcp && getver >/dev/null
DESC='v2.8.1-329-g410da30fb' => TAG='v2.8.1' + SUFFIX='-329-g410da30fb'; BASE='b583c96a0fcf5f68adc3463ad768f20258eada27' 
=> VER5='2.8.1.328.1' => VER50='2.8.1.328.1' 
=> DESC50='2.8.1.328.1-329-g410da30fb'

So this hack seems to best marry the two worlds for NUT purposes. Note that the local git repo index must know of the fresh trunk history which is not always a given (see #2551) to properly generate such information.

Also note that use of annotated git tags in maintenance is crucial for this feature by default, as they are the ones intended for releases (or extra git describe --tags arg is needed). We had an oopsie moment with 2.8.0:

:; git checkout v2.8.0-signed && getver >/dev/null
HEAD is now at ff16dabca configure.ac: cut the release of NUT v2.8.0

DESC='v2.7.4-5426-gff16dabca' => TAG='v2.7.4' + SUFFIX='-5426-gff16dabca'; BASE='ff16dabca191e5fd8ddc20137317bdebee554d8d' 
=> VER5='2.7.4.5426.0' => VER50='2.7.4.5426' 
=> DESC50='2.7.4.5426-5426-gff16dabca'

In an extreme case of a different project's repo with no annotated tags, just the hash is returned:

DESC='87f4cf7' => TAG='87f4cf7' + SUFFIX=''; BASE='87f4cf7fbb1b7762d9db6e6fe86100e09c82285a' 
=> VER5='87f4cf7.0.0' => VER50='87f4cf7' => DESC50='87f4cf7'

# ...and with `git describe --tags` for ordinary ones to be seen it is a bit better:
DESC='v0.22.0-42-g87f4cf7' => TAG='v0.22.0' + SUFFIX='-42-g87f4cf7'; BASE='87f4cf7fbb1b7762d9db6e6fe86100e09c82285a' 
=> VER5='0.22.0.42.0' => VER50='0.22.0.42' 
=> DESC50='0.22.0.42-42-g87f4cf7'

Interesting report from a system whose knowledge of master is rather outdated in its local index (before recent series of resync of FTY branch from the trunk). A FTY-based PR is identified thusly, as an horde of commits straight on top of last tag:

### trunk history in the index:
# commit 462aaf242ab69f2d8ca3463bb76c9bc641533b7c (master)
# Date:   Sat Feb 17 14:14:48 2024 +0100

### Current HEAD:
# commit 4a5d61e5b43944853d30a25509666aaf0f44c4de (HEAD -> issue-2344, origin/issue-2344)
# Date:   Tue Jul 23 11:59:32 2024 +0200

# Note the ".0" in the middle
:; getver
DESC='v2.8.2-3866-g4a5d61e5b' => TAG='v2.8.2' + SUFFIX='-3866-g4a5d61e5b'; BASE='462aaf242ab69f2d8ca3463bb76c9bc641533b7c' 
=> VER5='2.8.2.0.4154' => VER50='2.8.2.0.4154' 
=> DESC50='2.8.2.0.4154-3866-g4a5d61e5b'

# With a more recent pointer it is rectified:
:; TRUNK=origin/master getver
DESC='v2.8.2-3866-g4a5d61e5b' => TAG='v2.8.2' + SUFFIX='-3866-g4a5d61e5b'; BASE='7dcc938bdc9b26a2e9b861940f70d0e17d3c9921' 
=> VER5='2.8.2.680.3186' => VER50='2.8.2.680.3186' 
=> DESC50='2.8.2.680.3186-3866-g4a5d61e5b'

jimklimov added a commit to jimklimov/nut that referenced this issue Jul 24, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 24, 2024
…uctured version from Git (if available) [networkupstools#1949]

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 24, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 24, 2024
@jimklimov
Copy link
Member Author

jimklimov commented Jul 25, 2024

For autotools integration, got interesting inspiration from discussion at https://lists.gnu.org/archive/html/bug-autoconf/2024-01/msg00022.html to have the information generated into an m4 file (whether by autogen.sh during build from git, or passed via tarball), and include that into the configure.ac as m4 code:

jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 25, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 28, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 28, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 28, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 29, 2024
…VERSION_FORCED_SEMVER` files in our release rituals, we have support anyway and want to test it in-vivo [networkupstools#1949]

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 29, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
…tps://www.networkupstools.org/" if PACKAGE_URL points completely elsewhere) [networkupstools#1949]

Tested with "VER" command

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
…ed "https://www.networkupstools.org/" if PACKAGE_URL points completely elsewhere) [networkupstools#1949]

Tested with "GET VAR dummy server.info" command

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
…fully (and abort if there is no common BASE) [networkupstools#1949]

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
…tps://www.networkupstools.org/" if PACKAGE_URL points completely elsewhere) [networkupstools#1949]

Tested with "VER" command

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
…ed "https://www.networkupstools.org/" if PACKAGE_URL points completely elsewhere) [networkupstools#1949]

Tested with "GET VAR dummy server.info" command

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
…fully (and abort if there is no common BASE) [networkupstools#1949]

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
…tps://www.networkupstools.org/" if PACKAGE_URL points completely elsewhere) [networkupstools#1949]

Tested with "VER" command

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
…ed "https://www.networkupstools.org/" if PACKAGE_URL points completely elsewhere) [networkupstools#1949]

Tested with "GET VAR dummy server.info" command

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
…fully (and abort if there is no common BASE) [networkupstools#1949]

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 30, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 31, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Jul 31, 2024
…_GIT_TRUNK if not provided explicitly [networkupstools#1949]

Signed-off-by: Jim Klimov <[email protected]>
jimklimov added a commit to jimklimov/nut that referenced this issue Aug 1, 2024
jimklimov added a commit to jimklimov/nut that referenced this issue Aug 1, 2024
jimklimov added a commit that referenced this issue Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Entries related to continuous integration infrastructure (historically also recipes like Makefiles) enhancement packaging
Projects
Status: In Progress
Development

No branches or pull requests

2 participants