Policy on updating master with existing fixes from develop

[striezel]

Is your feature request related to a problem? Please describe.

Basically, there should be a merge of develop into master to get the newer fixes onto master and possibly into one of the next Boost releases.

For example, as of now (2022-05-05) those fixes are currently on develop but not on master yet, just to name some:

• for_each_pixel not work for any_image #579
• Fix C++20 incompatibilities in IO extensions #617
• Fix for_each_pixel for non 1d iterable views #621
• fix: is_equal_to_sixteen in PNG I/O was less-than test #650

Currently, the master branch is more than 100 commits behind develop, so there is a lot more to merge than the mentioned issue fixes.

Describe the solution you'd like

• Simply put: git checkout master && git merge develop should do it, but there are merge conflicts. So it needs the expertise of somebody like @mloskot who is very familiar with the code base to resolve these conflicts properly.
• Alternatively: Cherry pick fixes from develop onto master.

Additional context

The apparent need for an update has already been mentioned in #658 (comment):

@mloskot It looks like master needs an update...

and

@sdebionne Yes, I've neglected the master and not updated it from the develop for a while. I'll do it tonight.

Since there seems to be no further discussion about it (yet;)), I've created this issue as a place for discussion and to get things rolling.

I am not sure what the merge policy / release policy for Boost.GIL is, but I guess there still needs to be some kind of documentation updates (for example: changelog) before the merge can happen.

[sdebionne]

Thanks for opening this issue and summing up the situation. I guess we could start with merging MRs that fixe issues tagged with

• cat/annoyance
• cat/bug

IMHO, new features that lack documentations, such as the image processing GSOCs / Numerical Toolbox, should stay in develop.

[mloskot]

IMHO, new features that lack documentations, such as the image processing GSOCs / Numerical Toolbox, should stay in develop.

I assume we have eventually agreed on compromise here, as per your later response #663 (reply in thread)

[striezel]

But even for those annoyance fixes / bug fixes there should probably be some kind of documentation. So what is the usual procedure for that? I am asking so that maybe someone can already prepare a bit of that as a PR.

[mloskot]

was under the impression that a grand true merge is not desired at the moment

Well, it may not be desired in terms of achieving optimal quality result, but something is better than nothing. If we can't do what we want, we do what we can.

creating the release notes / changelog ...

We aim to create curated user-friendly release notes - that is why I keep marking PRs w/ milestones and labels. The changelog is available via git log.

Did you ever wonder why there was only one commit (f6a3553) in PR 665 apart from the release notes update and the merge conflict resolution? And why that commit still showed as verified?

I'm sorry, but I hadn't paid enough attention apparently - I making the merge while on the go using my mobile. My fault.
Thank you for pointing that out.

I aimed for baby steps, which in that case meant several tiny true merges (...) this approach of several tiny true merges (...) has at least two advantages

I agree, that would be ideal procedure, but, as I've mentioned, we will need to meet the quality vs feasibility somewhere half-way due to time and man-power constraints.
The points 2. and 3. mean that we will re-review quite a lot of content and we will be picky again, postponing changes here and there hoping that we will be able to get the docs done.

I do realise I may sound sloppy, but I think we all have to soften our stance and agree that "better may become an enemy of good" here.

Assuming that GIL in develop

• features that exist in master have not become broken
• new features in develop are usable
• develop is 'just' lacking content and test cases

then we can justify releasing the stuff in develop as soon as possible following easiest method available.

I've just tried the grand merge and it was not bad:

/d/boost/gil (master|u= origin/master) $ git merge --no-ff develop
Auto-merging .github/workflows/ci.yml
CONFLICT (add/add): Merge conflict in .github/workflows/ci.yml
Auto-merging CMakeLists.txt
Auto-merging CONTRIBUTING.md
Auto-merging README.md
CONFLICT (content): Merge conflict in README.md
Auto-merging RELEASES.md
CONFLICT (content): Merge conflict in RELEASES.md
Auto-merging test/Jamfile
CONFLICT (content): Merge conflict in test/Jamfile
Auto-merging test/core/channel/algorithm_channel_relation.cpp
CONFLICT (content): Merge conflict in test/core/channel/algorithm_channel_relation.cpp
Automatic merge failed; fix conflicts and then commit the result.
...
Unmerged paths:
  (use "git add <file>..." to mark resolution)
        both added:      .github/workflows/ci.yml
        both modified:   README.md
        both modified:   RELEASES.md
        both modified:   test/Jamfile
        both modified:   test/core/channel/algorithm_channel_relation.cpp

/d/boost/gil (master|MERGING|u= origin/master) $ git add -A
/d/boost/gil (master|MERGING|u= origin/master) $ git merge --continue
[master 6ef11459a] Merge branch 'develop'
/d/boost/gil (master|u+107 origin/master) $

Next, I will collect notes to RELEASES.md:

1. Run through the PRs of the current milestone
2. Skim through the git log to see if any important changes have been omitted (e.g. not PR'ed or not labelled)

Voila! :-)

Any objections to this approach?

[striezel]

I agree, that would be ideal procedure, but, as I've mentioned, we will need to meet the quality vs feasibility somewhere half-way due to time and man-power constraints. The points 2. and 3. mean that we will re-review quite a lot of content and we will be picky again, postponing changes here and there hoping that we will be able to get the docs done.

Well, I was hoping on reviews not being too picky there. After all, that stuff has already been reviewed when it was merged to develop. But yes, one single big merge is certainly less time-consuming and I can understand why you are taking that approach.

Any objections to this approach?

No, not from me. Go on.

Documentation can still be updated after the merge.

[striezel]

I've just tried the grand merge and it was not bad

What is the progress on this one? Any estimate on when the grand true merge will happen?
If you need help, e. g. with the preparation of the release notes, give a shout. Strictly speaking, release notes are "just" another type of documentation, so they could also be updated separately, after the merge.

[mloskot]

I'm going to complete it today, at least the steps 1, 2, and 3 from #667

Strictly speaking, release notes are "just" another type of documentation, so they could also be updated separately, after the merge.

Yes, I agree and that is the plan.

Thanks for your help @striezel

[mloskot]

UPDATE: Merged!

[sdebionne]

I could not find guidelines for libraries, but there is some documentation available at the superproject scope.

We need to provide release notes to the release manager, I am not sure if it's automatically consolidated in the documentation or if we need to open a PR on on the website...

[mloskot]

Yes, as I explained in my #663 (comment), we manually collect our notes, then we submit PR against the website with our notes (in QuickBook format). Not ideal, but that's what Boost does for now.
There has been lots of discussions about better format and automation of notes collection from Boost libraries, see the mailing list archives, but nothing has been approved yet.

[mloskot]

(I have taken the liberty to turn this into the Discussion. I hope it's OK.)

First, thank you @striezel for raising this issue.

The lack of merging of develop into master follows the GSoC work and is due to the problems with CI (removal of Travis CI) and lack of time over the last year on my end. Simply, the amount of GSoC work merged into develop was substantial and it required (and still does) some significant amount of work (i.e. documenting and testing) in order to make it release-ready. And, given the lack of time, especially close to the recent Boost releases, I decided to postpone the merging.

I am not sure what the merge policy / release policy for Boost.GIL is, but I guess there still needs to be some kind of documentation updates (for example: changelog) before the merge can happen.

There is no policy. We simply try to follow common sense and merge develop into master whenever the former is in decent shape.

The procedure is not documented (it should!), but it's simple

1. Ensure current CI-s of both, develop and master, are green
2. Update the RELEASES.md (introduced it in Add release notes in Markdown format #404 as internal maintainer-friendly way of maintaining the release notes)
3. Run git switch master && git merge develop
4. Copy the notes from RELEASES.md to QuickBook-formatted notes on the Boost website (e.g. Add GIL release notes for Boost 1.76.0 website#605)

The documentation is published automatically via GHA (since #574).

Collection of the release notes may become PITA and even labelling of Issues/PR does not clear the chore, so I some time ago started using https://www.conventionalcommits.org for my own commits and PR merge commits, and it helps to grep-out the relevant stuff.

As soon as there is consensus among us that develop is ready to merge into master, I can merge it and collect the release notes.
Alternatively, we can cherry-pick individual fixes and I do it after someone

• comments on issue/PR requesting it to be picked into master
• opens issue nagging me a particular PR/commit should be picked into master

I usually prefer the true merges as it helps keep the master as close to develop as possible, but I don't die-hard for the merges.
I hope it explains the merging status quo.

[mloskot]

One more thing, in our CONTRIBUTING.md on Pull Requests, I intentionally did not mention anything about documentation and the only hard requirement is that a feature or a fix must be covered with a test. Although it's not ideal or its even wrong, the documentation received secondary priority.
That is to not to kill good contributions prematurely due to lack of documentation.
We lack of contributors, man power, so we need to compromise - something is better than nothing; better is enemy of good :-)

[striezel]

(I have taken the liberty to turn this into the Discussion. I hope it's OK.)

Yes, that is OK. When I created the issue I just thought that there may be a rather straightforward way to get things merged. But seeing now that there still are some concerns regarding some features on develop, I believe that a discussion is acutally a better format than a simple issue.

Merge policy

As soon as there is consensus among us that develop is ready to merge into master, I can merge it and collect the release notes. Alternatively, we can cherry-pick individual fixes and I do it after someone

• comments on issue/PR requesting it to be picked into master
• opens issue nagging me a particular PR/commit should be picked into master

I usually prefer the true merges as it helps keep the master as close to develop as possible, but I don't die-hard for the merges. I hope it explains the merging status quo.

I prefer true merges, too, because cherry-picking means that one has to keep track of what has been cherry-picked and what not. While cherry-picking certainly helps to get stuff "merged" in the short term, I feel that it makes things more complicated in the long run, because it takes a while to figure out which commits have been cherry-picked onto another branch. With true merges this becomes rather trivial.

Observations + suggestions for branch model

The current branch model with master and develop works well when everything that gets merged into develop is in the desired shape and potentially ready to be included in the next release. However, as soon as this mixes with stuff that is accepted but does not (yet) meet the requirements for a future release, this model falls apart. In that case there should be at least three branches: master, develop and something like acceptable-for-develop-but-not-for-the-next-release (clunky name but you get the idea). With that model master branch would stay what it is, develop only gets the stuff that is ready (including tests + documentation), and the third branch gets the stuff that is OK but still needs work. That way one could always merge develop into master in time for the next release.

Or maybe there should be dedicated feature branches where work is done. This would be best for handling features / fixes individually, but I am not sure that the current lack of time allows for such a detailed approach. And it may possibly "kill" some contributions, because they won't get merged until they are completely done, including documentation.

GSoC

The lack of merging of develop into master follows the GSoC work and is due to the problems with CI (removal of Travis CI) and lack of time over the last year on my end. Simply, the amount of GSoC work merged into develop was substantial and it required (and still does) some significant amount of work (i.e. documenting and testing) in order to make it release-ready. And, given the lack of time, especially close to the recent Boost releases, I decided to postpone the merging.

There is quite a lot stuff with the GSoC label that got merged: https://github.com/boostorg/gil/pulls?q=is%3Apr+label%3Agoogle-summer-of-code+is%3Aclosed Any hints where to start? And is there a realistic chance that the contributors of GSoC may return to add tests and documentation for their contributions? Otherwise I do not see how the GSoC contributions can be put into a shape where they are fit for merge into master in the near future (meaning: months, not years), because other people either do not seem to have the time or the knowledge required to do this. So this will probably be the tricky part.

Procedure / policy for releases

There is no policy. We simply try to follow common sense and merge develop into master whenever the former is in decent shape.

The procedure is not documented (it should!), but it's simple

1. Ensure current CI-s of both, develop and master, are green
2. Update the RELEASES.md (introduced it in Add release notes in Markdown format #404 as internal maintainer-friendly way of maintaining the release notes)
3. Run git switch master && git merge develop
4. Copy the notes from RELEASES.md to QuickBook-formatted notes on the Boost website (e.g. Add GIL release notes for Boost 1.76.0 website#605)

The documentation is published automatically via GHA (since #574).

Collection of the release notes may become PITA and even labelling of Issues/PR does not clear the chore, so I some time ago started using https://www.conventionalcommits.org for my own commits and PR merge commits, and it helps to grep-out the relevant stuff.

OK, thanks for providing the overview on how to handle things. This should be enough as a general idea and to get things moving. 👍

Seeing that the CI is currently not green - neither on develop nor master - I will probably send a few more pull request which try to attack this problem, but I cannot promise anything here.

[striezel]

(That is more of a reminder to / information for myself.)
Here's the timeline for Boost 1.80.0: https://www.boost.org/community/index.html

2022-06-22: Boost 1.80.0 closed for new libraries and breaking changes

Release branch is closed for new libraries and breaking changes to existing libraries. Still open for bug fixes and other routine changes to all libraries without release manager review.

2022-06-29: Boost 1.80.0 closed for major changes

Release closed for major code changes. Still open for serious problem fixes and docs changes without release manager review.

2022-07-06: Boost 1.80.0 closed for beta

Release closed for all changes

2022-07-13: Boost 1.80.0 beta

Beta posted for download.

2022-07-14: Boost 1.80.0 open for bug fixes

Release open for bug fixes and documentation updates. Other changes by permission of a release manager.

2022-08-03: Boost 1.80.0 closed

Release closed for all changes

2022-08-10: Boost 1.80.0 release

Release posted for download.

---

TL;DR: Despite the fact that the next Boost release still is ca. three months away (10th August), we basically only have a one and a half month (until 22nd June / 29th June) for the bigger changes.