Add a reusable workflow to update the sub-project to the next release

[laeubi]

Currently updating to the next release is a manual step, this adds a new reusable workflow that can be used to automate the required steps that include:

• update the project version and child modules
• update the parent version
• run a build and look for required version bumps

FYI @akurtakov @SDawley @sravanlakkimsetti this is the reusable part for the subrepos, if a repository use the workflow and a new milestone is created it looks for example like this:

• Update for release 4.28 laeubi/eclipse.platform#4

If there are no concerns, I'll merge this and enable this for e.g. platform, platform.ui, p2 and equinox so we have some prototypes, if that works out well one might want to roll-out this to even more repositories. Or if we want we can even roll this out right now to all repositories, but then I probably would need some help with apply / testing as the Milestones are about to be created not later than March 2.

[iloveeclipse]

@laeubi : in case you need to change/merge something in JDT, just ping me.

[akurtakov]

Go for it . Next week we start using it and do changes if further needed.

[laeubi]

@laeubi : in case you need to change/merge something in JDT, just ping me.

If you like we can use it for jdt as well, I'll prepare a PR for platform with instructions how to apply / test, its quite straight forward, just a bit of clumsy to test for a single person...

[iloveeclipse]

If you like we can use it for jdt as well

It is not me, it is SDK build that would like to see all bits updated, including JDT.

[mickaelistria]

update the project version and child modules

Does it apply blindly on all modules? Bumping all versions is typically undesired; we only want to bump version for modules that have payload change.

[iloveeclipse]

I'm not sure I understand what this PR is supposed to do, but if that unconditionally bumps service segment for all bundles at release start, why shouldn't we do that? Sure, if bundle has no changes its version will increase every release for no reason. But what is real drawback of that?

But on the other side, all this constant trouble with first PR's merge where the checks fail because the bundle version was not changed - is it really needed? Only PRs with API changes would need a manual update, and most "usual" contributions would require less work.

[mickaelistria]

But what is real drawback of that?

The drawbacks is that we produce more artifacts, so:

• More artifacts to download/install to users (more resources used on both sides)
• More artifacts to validate: imagine a jar produced by the project is "certified" by some consumers of good enough quality; as long as we keep the version -and the jar- the certification remains; if we stop it then the certification is lost even without any change

all this constant trouble with first PR's merge where the checks fail because the bundle version was not changed - is it really needed?

This trouble should be investigated. Bumping the version of the parent should usually not require a version change. Only a few bundles for which a parent version bump affects the payload should be bumped and it's even sure they would actually need that bump.

So IMO, this is attacking the problem from the wrong angle. Instead of fixing "why do we have to bump versions", it does "bump all versions". This is IMO reducing the quality of the project releng.

[akurtakov]

FWIW, from releng POV I have been bumping versions without investigating why the comparator complains already as I never had the time to look into why so it remains to be seen whether there will be more bumps than usual. IMHO I would sacrifice a number of downloads for the sake of less manual work.

[mickaelistria]

MHO I would sacrifice a number of downloads for the sake of less manual work.

The proposal here is basically to get rid of reproducible versions qualifiers and reproducible builds; and thus to produce more and more artifacts. In the light of upcoming EU Cybersecurity Resilience Act, I believe it becomes important to produce less and less artifacts to save numerous audits.

[iloveeclipse]

to save numerous audits.

Out of curiosity : who is supposed to do the audits?
I assume they will be automated anyway, so simple comparison of jar file contents should show no or minor differences.

[mickaelistria]

Out of curiosity : who is supposed to do the audits?

that I believe is far from being settled, as the CRA itself is far from being settled.
I suspect that the one who makes money with X is expected to pay for certification of X. In the case of a shared project, that may become a service centralized by the foundation (members fund the foundation, that pays the desired certification).

I assume they will be automated anyway, so simple comparison of jar file contents should show no or minor differences.

If version is changed and we stop caring about reproducible artifacts, then some content of the jars are different (at least the MANIFEST), so one would need to check those contents. If jar is the same as the current process allows, there is no need to drill down.
We currently also have the guarantee that 1 version of the jar <=> 1 version of the source code. Getting rid of reproducible versions breaks it.

[akurtakov]

From @laeubi 's example laeubi/eclipse.platform#4 this is not what happens - only two bundles and exactly these that we have been bumping every time as comparator complained about them have been prepared so my understanding is that it's not blindly bumping every version.

[mickaelistria]

so my understanding is that it's not blindly bumping every version.

OK, if it is so, then it's the best of both world. I just got a bit worried as the script seems to use the "good old" version bump mojos, but maybe I just failed at noticing an important detail. @laeubi do you confirm only the bundles for which there is a comparator error are changed?

[laeubi]

• First the workflow updates the project version and parent version to the next release version, this is a simple update with no checks but at least a semantic one in contrast to the current search+replace script.
• Then the workflow performs a build and if any version checks fail, bump the version for this projects and starts again (up to 10 times).

The second part is mostly done because @akurtakov mentioned that in the past sometimes there where some versionbumps required, I have not yet found such a case, maybe because now Tycho has enhanced comparator support or the new baseline is not available, I think that's something we will only see when the new stream starts and then do some analysis and probably improve the workflow or the tools or the process.

In the end the main idea is, that opening a new stream should actually involve much less manual work and its more "dependabot-style", that scripts prepare all the boring parts and we hopefully press just the merge button most of the time and can then concentrate on the remaining few percent where an actual user intervention is required.

[mickaelistria]

updates the project version

This means project version for the root pom; and not recursively applied to all children (at this moment)?
If so, then it seems perfect.

[laeubi]

updates the project version

This means project version for the root pom; and not recursively applied to all children (at this moment)? If so, then it seems perfect.

It is working according to these rules:
https://tycho.eclipseprojects.io/doc/latest/tycho-release/tycho-versions-plugin/set-version-mojo.html

because of this, the example run here:
https://github.com/laeubi/eclipse.platform/pull/4/files

also updates two bundles that have the same version as the parent (what seems desired here)

[mickaelistria]

OK, great. Thanks!

[laeubi]

@laeubi : in case you need to change/merge something in JDT, just ping me.

I just tried this with JDT and found a small bug in the version updater (see eclipse-tycho/tycho#2155 ) once this is fixed, I'll create a PR for JDT-CORE.

[HannesWell]

Wouldn't it be more precise to trigger the Workflow when a Release-tag ist created instead of when a milestone ist created?
Im general milestones could be created at any time, but the release tag ist only created once the release is done.
In regards of implementation this should not be a problem since creating/pushing tags creates an Event too.

[laeubi]

Tags can be created at any time also ... but the do not carry any semantics that are required here , also we need to open the stream for the NEXT release if we perform this AFTER the release it is a bit late ;-)

[HannesWell]

Tags can be created at any time also ... but the do not carry any semantics that are required here , also we need to open the stream for the NEXT release if we perform this AFTER the release it is a bit late ;-)

OK it might be a bit late yes. But what kind of semantic are you referring to exactly? Isn't it all what's need the version of the current release and the information that it is a release?
But anyway waiting for the tag is probably too late, as you said.

So what about doing this when the release/maintenance branch is created? I think that is exactly the point in time when we want to open the master again. From the branch name the release-number can be derived (just like from the milestone).
Maybe a sanity check would be reasonable if the just created release/maintenance branch points to the current master.

On the other hand, how are the maintenance branches created?
Maybe it would make sense to create them together in the same job like the 'open-next-cycle' changes?

[laeubi]

isn't it all what's need the version of the current release and the information that it is a release?

First we need the NEXT release not the current one, then we need to know that it is a release (and not an ordinary tag) and I'm also using the release date to derive the release name (e.g. 2023-06).

So Milestones have different advantages:

1. The naming is not limited and there is a clear separation of the fields (Name, Description Date)
2. They can have different states and triggers (created, modified, closed)
3. Issues / PR can be assigned to them (for example the PR that is created is automatically assigned to the Milestone) and even we currently do not use that much we probably should do so
4. And actually the main goal is to have once overlapping releases cycles

On the other hand, how are the maintenance branches created?
Maybe it would make sense to create them together in the same job like the 'open-next-cycle' changes?

This would better be done on Milestone CLOSE I think, but actually it would be better to get rid of "maintaince" at all and always use simple branches, for the overlapping release case it should be more like this:

• At RC1 time a new branch is created R_4.27, what is picked up automatically by a multi-branch pipeline
• When this build is done and the update repo for the 4.28 is filled (see Feed the next streams update site repository with the contents of M1 / M2 / RC1 / RC2 prior to release #884) the M3 Milestone is closed
• Closing the M3 triggers creation of the next Milestones created
• Creating of the next release Milestone triggers the opening of the master branch for next release

That way the master is only "closed" for a few hours, not much manual intervention required and if there is any bugfix it can simply be backported / cherry picked to the R_4.27 branch with usual PRs and validation checks.