Decouple version module number timing from project versioning

[apiuser]

Feature request from fooman, posted on GitHub Apr 19, 2016

Expected result

1. When a new Magento 2.x version is released a merchant can upgrade immediately

Actual result

1. When a new Magento 2.x version is released a merchant needs to wait for extension providers/ SIs to assess what changes have made it into this release

Currently it seems the only time a module's version number can change is when the Magento 2 project releases. From an external developer's stand point a better approach would be for the modules to change their version number the moment a version number affecting change is merged into develop. This allows an external developer to follow development between releases without having to wait until the final avalanche of version number increases on release day and then having to play catch up with all changes in a short period of time.

As a further suggestion the following policy would help extension developers prepare for an upcoming release:

At the end of the penultimate sprint before release a Release Candidate is tagged (or earlier). From there until final release only patch level fixes are allowed on the Magento module level (if a fix is required that is breaking a new Release Candidate would be added).

[apiuser]

Comment from alankent, posted on GitHub May 16, 2016

Hi Fooman, sorry for delay. We were trying to sort out relative priorities here internally. Having independent version numbers for modules at releases ties in with the Git repo structure. You want the source code branching scheme (git) aligned with the release version numbers (composer packages). One solution for example is to split the repo to have separate repos per module. There are other strategies that involve less work, but more discipline around release strategies.

For example, we create branches at release time to create patches from them. We just create one release branch per CE/EE release, not per module. If the version number is unchanged (because the module is unchanged), then we end up with the same code with same version number on two branches. How then to patch it on the release branch?

It is a solvable problem - it is more around reaching agreement on the strategy that is the delay. For example, we could always do minor number increments for every module per CE/EE release, changing it to a major number increment if we do a backwards incompatible change. Then as soon as we cut the release branch, we do a minor number increment on every module.

This is not strictly SemVer of course. We would be incrementing version number without actual change. That has been the hesitancy. An alternative strategy (trying to summarize correctly here!) is to not increment the version number until first change, then increment it (risk: developers have to "get it right" or have automated tools to help). If no changes, then we need to be careful about patches - the same code might be in the 2.0 and 2.1 and mainline release branches. We need to apply identical changes in all branches (everywhere the code is with the same version number), again which has risk (easy to make a mistake).

There are other variations, but hopefully the challenge comes across. A part of the internal argument has been around strict SemVer versus robustness. The more tools we need to get it right, the more fearful I am of mistakes (and the more effort to get it done).

I suspect the short term solution maybe to do minor number increments of all modules (which we could do immediately after a release branch is cut), changing to major number increment for modules if the API is a backwards incompatible change. I think that would address your request to see API changes sooner, but it would increment numbers faster than SemVer dictates.

Alternatively we need to commit to break up the Git repo into lots of little repos, which can make internal development more painful (and hence potentially slower). It is also potentially a lot of work as there are various test scripts that assume the current directory structure - lots of things may break. So it is not something we would want to rush.

[apiuser]

Comment from fooman, posted on GitHub May 18, 2016

Thanks @alankent for getting back to me. This issue is closely related to #4228.

In reply to some of your questions

If the version number is unchanged (because the module is unchanged), then we end up with the same code with same version number on two branches. How then to patch it on the release branch?

I am not sure that I follow - if it is the same version number it should be the same code - hence no need to patch or the patch applies the same way.

One solution for example is to split the repo to have separate repos per module.

Alternatively we need to commit to break up the Git repo into lots of little repos, which can make internal development more painful (and hence potentially slower).

I am still not sure that I understand the internal complications fully but breaking up the repos seems logical. My naive view would have been that if each module owner/team has their own repo they can work faster since independent. In terms of bringing it all back together this is the job of the Magento2 composer metapackage. It includes the version constraints like magento-sales: 100.0.* or magento-sales: ^100.0.2 so the moment the Sales module owner applies a fix to their module and releases new versions 104.2.n+1, 100.0.n+1 it would get updated in the Magento2 project the moment composer update is run for each branch (2.1->104, 2.0->100).

I am guessing that the module owner needs to maintain branches and backport any potential fixes is what you might have referred to earlier. But I am currently not sure how that issue is different to what you have today - the branches are just confined to the module now instead of on the project level.

In preparation for a release / QA it makes sense to me to lock down the constraint so when a release is pushed the exact mix of module version numbers that will get released is known. Once the release is done the constraints are loosened so that development moves on.

All this hinges on the definition of #4228 of course, since it would allow module owners at Magento to trust their dependencies. So for example Magento Sales would have magento-framework: ^100.0.3 on their 100 branch and magento-framework: ^103.0.0 on their 104 branch (branches 101, 102 and 103 would not be maintained since they did not make it into a Magento2 release and were created mid cycle).

[apiuser]

Comment from piotrekkaminski, posted on GitHub May 24, 2016

@alankent do we have any solution/response to this (cc @antonkril )