RFC: Release process changes

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

RFC: Release process changes

Neil Nelson via cfe-dev
Hi,

I would like to propose a few changes to the LLVM release process.  The
current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html

There are two parts to this proposal.  The first is a list of clarifications,
which are things we are currently doing that aren't documented. The second
is a list of changes which would actually modify how releases are currently
managed.



*** Proposed Clarifications ***



**  Release manager is allowed to commit changes to the release branch without
    code owner approval.  However, the release manager is encouraged to consult
    with code owners or patch reviewers for non-trivial changes.

It's not practical to get code owner approval every time.  Either because there
is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
This proposed clarification matches how releases are currently managed.


** There is no official release criteria.

We have time-based releases and when the release is 'ready' has been
up to the discretion of the release manager.  Changing the release
criteria is out of the scope of this proposal, but I do think it would
be good to have a discussion about this as a community, so I'm going to
start a separate thread to discuss this.



*** Proposed Changes ***



** Create a time-based bug-fix release schedule.  After each major release, make
   a new bug-fix release every 2 weeks for 12 weeks (6 releases total).

** Eliminate release candidates for bug-fix releases.

The current unofficial bug-fix release schedule is:

X.Y.1-rc1 (6 weeks after major release)
X.Y.1-rc2 (10 weeks after major release)
X.Y.1-final (12 weeks after major release)

I think this change will improve the overall test coverage of the release branch.
I don't think the branch itself or even the release candidates get the same
level of testing as the final releases.  If we are consistently snapshotting
the release branch and putting out releases, I think this will make it easier
and thus more likely that users will test out the release branch code.

Additionally, with more frequent bug-fix release it removes the need to have
release candidate releases. Every bug-fix release (up until the last one)
would serve the same purpose as our current release candidates in that they
are intended to give users an easier way to test the code before the final
release.


** Create clear rules for what kind of backports are accepted during each
   release phase.

* Before RC1:Patches should be limited to bug fixes, important optimization
  improvements, or completion of features that were started before the branch
  was created.  As with all phases, release managers and code owners can reject
  patches that are deemed too invasive.

* Before RC2: Patches should be limited to bug fixes or backend specific
  improvements that are determined to be very safe.

* Before RC3/Final: Major Release* Patches should be limited to critical
  bugs or regressions.
 
* Bug fix releases: Patches should be limited to bug fixes or very safe
  and critical performance improvements.  Patches must maintain both API and
  ABI compatibility with the previous major release.
 
* Final bug fix release: Patches should be limited to critical bug fixes only.



What does everyone thing about these changes?


-Tom

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Release-testers] RFC: Release process changes

Neil Nelson via cfe-dev
On Thu, 2020-05-21 at 11:59 -0700, Tom Stellard via Release-testers
wrote:

> Hi,
>
> I would like to propose a few changes to the LLVM release process.  The
> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
>
> There are two parts to this proposal.  The first is a list of clarifications,
> which are things we are currently doing that aren't documented. The second
> is a list of changes which would actually modify how releases are currently
> managed.
>
>
>
> *** Proposed Clarifications ***
>
>
>
> **  Release manager is allowed to commit changes to the release branch without
>     code owner approval.  However, the release manager is encouraged to consult
>     with code owners or patch reviewers for non-trivial changes.
>
> It's not practical to get code owner approval every time.  Either because there
> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
> This proposed clarification matches how releases are currently managed.
>
>
> ** There is no official release criteria.
>
> We have time-based releases and when the release is 'ready' has been
> up to the discretion of the release manager.  Changing the release
> criteria is out of the scope of this proposal, but I do think it would
> be good to have a discussion about this as a community, so I'm going to
> start a separate thread to discuss this.
>
>
>
> *** Proposed Changes ***
>
>
>
> ** Create a time-based bug-fix release schedule.  After each major release, make
>    a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
>
> ** Eliminate release candidates for bug-fix releases.
>
> The current unofficial bug-fix release schedule is:
>
> X.Y.1-rc1 (6 weeks after major release)
> X.Y.1-rc2 (10 weeks after major release)
> X.Y.1-final (12 weeks after major release)
>
> I think this change will improve the overall test coverage of the release branch.
> I don't think the branch itself or even the release candidates get the same
> level of testing as the final releases.  If we are consistently snapshotting
> the release branch and putting out releases, I think this will make it easier
> and thus more likely that users will test out the release branch code.
>
> Additionally, with more frequent bug-fix release it removes the need to have
> release candidate releases. Every bug-fix release (up until the last one)
> would serve the same purpose as our current release candidates in that they
> are intended to give users an easier way to test the code before the final
> release.
>
>
> ** Create clear rules for what kind of backports are accepted during each
>    release phase.
>
> * Before RC1:Patches should be limited to bug fixes, important optimization
>   improvements, or completion of features that were started before the branch
>   was created.  As with all phases, release managers and code owners can reject
>   patches that are deemed too invasive.
>
> * Before RC2: Patches should be limited to bug fixes or backend specific
>   improvements that are determined to be very safe.
>
> * Before RC3/Final: Major Release* Patches should be limited to critical
>   bugs or regressions.
>  
> * Bug fix releases: Patches should be limited to bug fixes or very safe
>   and critical performance improvements.  Patches must maintain both API and
>   ABI compatibility with the previous major release.
>  
> * Final bug fix release: Patches should be limited to critical bug fixes only.
>
>
>
> What does everyone thing about these changes?
>
Sounds reasonable to me.  I think it would certainly benefit users not
to have wait so long for x.1 fixes, and it would mean downstreams have
to backport less.


--
Best regards,
Michał Górny


_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
All of this sounds reasonable to me, but we don't directly follow the
upstream release cadence so I'm an interested observer at most.

Philip

On 5/21/20 11:59 AM, Tom Stellard via llvm-dev wrote:

> Hi,
>
> I would like to propose a few changes to the LLVM release process.  The
> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
>
> There are two parts to this proposal.  The first is a list of clarifications,
> which are things we are currently doing that aren't documented. The second
> is a list of changes which would actually modify how releases are currently
> managed.
>
>
>
> *** Proposed Clarifications ***
>
>
>
> **  Release manager is allowed to commit changes to the release branch without
>      code owner approval.  However, the release manager is encouraged to consult
>      with code owners or patch reviewers for non-trivial changes.
>
> It's not practical to get code owner approval every time.  Either because there
> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
> This proposed clarification matches how releases are currently managed.
>
>
> ** There is no official release criteria.
>
> We have time-based releases and when the release is 'ready' has been
> up to the discretion of the release manager.  Changing the release
> criteria is out of the scope of this proposal, but I do think it would
> be good to have a discussion about this as a community, so I'm going to
> start a separate thread to discuss this.
>
>
>
> *** Proposed Changes ***
>
>
>
> ** Create a time-based bug-fix release schedule.  After each major release, make
>     a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
>
> ** Eliminate release candidates for bug-fix releases.
>
> The current unofficial bug-fix release schedule is:
>
> X.Y.1-rc1 (6 weeks after major release)
> X.Y.1-rc2 (10 weeks after major release)
> X.Y.1-final (12 weeks after major release)
>
> I think this change will improve the overall test coverage of the release branch.
> I don't think the branch itself or even the release candidates get the same
> level of testing as the final releases.  If we are consistently snapshotting
> the release branch and putting out releases, I think this will make it easier
> and thus more likely that users will test out the release branch code.
>
> Additionally, with more frequent bug-fix release it removes the need to have
> release candidate releases. Every bug-fix release (up until the last one)
> would serve the same purpose as our current release candidates in that they
> are intended to give users an easier way to test the code before the final
> release.
>
>
> ** Create clear rules for what kind of backports are accepted during each
>     release phase.
>
> * Before RC1:Patches should be limited to bug fixes, important optimization
>    improvements, or completion of features that were started before the branch
>    was created.  As with all phases, release managers and code owners can reject
>    patches that are deemed too invasive.
>
> * Before RC2: Patches should be limited to bug fixes or backend specific
>    improvements that are determined to be very safe.
>
> * Before RC3/Final: Major Release* Patches should be limited to critical
>    bugs or regressions.
>  
> * Bug fix releases: Patches should be limited to bug fixes or very safe
>    and critical performance improvements.  Patches must maintain both API and
>    ABI compatibility with the previous major release.
>  
> * Final bug fix release: Patches should be limited to critical bug fixes only.
>
>
>
> What does everyone thing about these changes?
>
>
> -Tom
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Release-testers] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On 2020-05-21, Michał Górny via cfe-dev wrote:

>On Thu, 2020-05-21 at 11:59 -0700, Tom Stellard via Release-testers
>wrote:
>> Hi,
>>
>> I would like to propose a few changes to the LLVM release process.  The
>> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
>>
>> There are two parts to this proposal.  The first is a list of clarifications,
>> which are things we are currently doing that aren't documented. The second
>> is a list of changes which would actually modify how releases are currently
>> managed.
>>
>>
>>
>> *** Proposed Clarifications ***
>>
>>
>>
>> **  Release manager is allowed to commit changes to the release branch without
>>     code owner approval.  However, the release manager is encouraged to consult
>>     with code owners or patch reviewers for non-trivial changes.
>>
>> It's not practical to get code owner approval every time.  Either because there
>> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
>> This proposed clarification matches how releases are currently managed.
>>
>>
>> ** There is no official release criteria.
>>
>> We have time-based releases and when the release is 'ready' has been
>> up to the discretion of the release manager.  Changing the release
>> criteria is out of the scope of this proposal, but I do think it would
>> be good to have a discussion about this as a community, so I'm going to
>> start a separate thread to discuss this.
>>
>>
>>
>> *** Proposed Changes ***
>>
>>
>>
>> ** Create a time-based bug-fix release schedule.  After each major release, make
>>    a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
>>
>> ** Eliminate release candidates for bug-fix releases.
>>
>> The current unofficial bug-fix release schedule is:
>>
>> X.Y.1-rc1 (6 weeks after major release)
>> X.Y.1-rc2 (10 weeks after major release)
>> X.Y.1-final (12 weeks after major release)
>>
>> I think this change will improve the overall test coverage of the release branch.
>> I don't think the branch itself or even the release candidates get the same
>> level of testing as the final releases.  If we are consistently snapshotting
>> the release branch and putting out releases, I think this will make it easier
>> and thus more likely that users will test out the release branch code.
>>
>> Additionally, with more frequent bug-fix release it removes the need to have
>> release candidate releases. Every bug-fix release (up until the last one)
>> would serve the same purpose as our current release candidates in that they
>> are intended to give users an easier way to test the code before the final
>> release.

Just to confirm: are bug-fix releases X.Y.1 X.Y.2 X.Y.3 ...

Seems reasonable. Package maintainers on various distributions may
have more words here.
It seems that we have a +1 from Gentoo now.


>>
>> ** Create clear rules for what kind of backports are accepted during each
>>    release phase.
>>
>> * Before RC1:Patches should be limited to bug fixes, important optimization
>>   improvements, or completion of features that were started before the branch
>>   was created.  As with all phases, release managers and code owners can reject
>>   patches that are deemed too invasive.
>>
>> * Before RC2: Patches should be limited to bug fixes or backend specific
>>   improvements that are determined to be very safe.
>>
>> * Before RC3/Final: Major Release* Patches should be limited to critical
>>   bugs or regressions.
>>
>> * Bug fix releases: Patches should be limited to bug fixes or very safe
>>   and critical performance improvements.  Patches must maintain both API and
>>   ABI compatibility with the previous major release.
>>
>> * Final bug fix release: Patches should be limited to critical bug fixes only.
>>
>>
>>
>> What does everyone thing about these changes?
>>
>
>Sounds reasonable to me.  I think it would certainly benefit users not
>to have wait so long for x.1 fixes, and it would mean downstreams have
>to backport less.
>
>
>--
>Best regards,
>Michał Górny
>



>_______________________________________________
>cfe-dev mailing list
>[hidden email]
>https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Release-testers] RFC: Release process changes

Neil Nelson via cfe-dev
On 05/21/2020 05:38 PM, Fāng-ruì Sòng wrote:

> On 2020-05-21, Michał Górny via cfe-dev wrote:
>> On Thu, 2020-05-21 at 11:59 -0700, Tom Stellard via Release-testers
>> wrote:
>>> Hi,
>>>
>>> I would like to propose a few changes to the LLVM release process.  The
>>> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
>>>
>>> There are two parts to this proposal.  The first is a list of clarifications,
>>> which are things we are currently doing that aren't documented. The second
>>> is a list of changes which would actually modify how releases are currently
>>> managed.
>>>
>>>
>>>
>>> *** Proposed Clarifications ***
>>>
>>>
>>>
>>> **  Release manager is allowed to commit changes to the release branch without
>>>     code owner approval.  However, the release manager is encouraged to consult
>>>     with code owners or patch reviewers for non-trivial changes.
>>>
>>> It's not practical to get code owner approval every time.  Either because there
>>> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
>>> This proposed clarification matches how releases are currently managed.
>>>
>>>
>>> ** There is no official release criteria.
>>>
>>> We have time-based releases and when the release is 'ready' has been
>>> up to the discretion of the release manager.  Changing the release
>>> criteria is out of the scope of this proposal, but I do think it would
>>> be good to have a discussion about this as a community, so I'm going to
>>> start a separate thread to discuss this.
>>>
>>>
>>>
>>> *** Proposed Changes ***
>>>
>>>
>>>
>>> ** Create a time-based bug-fix release schedule.  After each major release, make
>>>    a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
>>>
>>> ** Eliminate release candidates for bug-fix releases.
>>>
>>> The current unofficial bug-fix release schedule is:
>>>
>>> X.Y.1-rc1 (6 weeks after major release)
>>> X.Y.1-rc2 (10 weeks after major release)
>>> X.Y.1-final (12 weeks after major release)
>>>
>>> I think this change will improve the overall test coverage of the release branch.
>>> I don't think the branch itself or even the release candidates get the same
>>> level of testing as the final releases.  If we are consistently snapshotting
>>> the release branch and putting out releases, I think this will make it easier
>>> and thus more likely that users will test out the release branch code.
>>>
>>> Additionally, with more frequent bug-fix release it removes the need to have
>>> release candidate releases. Every bug-fix release (up until the last one)
>>> would serve the same purpose as our current release candidates in that they
>>> are intended to give users an easier way to test the code before the final
>>> release.
>
> Just to confirm: are bug-fix releases X.Y.1 X.Y.2 X.Y.3 ...
>

Yes, this is correct.

-Tom

> Seems reasonable. Package maintainers on various distributions may
> have more words here.
> It seems that we have a +1 from Gentoo now.
>
>
>>>
>>> ** Create clear rules for what kind of backports are accepted during each
>>>    release phase.
>>>
>>> * Before RC1:Patches should be limited to bug fixes, important optimization
>>>   improvements, or completion of features that were started before the branch
>>>   was created.  As with all phases, release managers and code owners can reject
>>>   patches that are deemed too invasive.
>>>
>>> * Before RC2: Patches should be limited to bug fixes or backend specific
>>>   improvements that are determined to be very safe.
>>>
>>> * Before RC3/Final: Major Release* Patches should be limited to critical
>>>   bugs or regressions.
>>>
>>> * Bug fix releases: Patches should be limited to bug fixes or very safe
>>>   and critical performance improvements.  Patches must maintain both API and
>>>   ABI compatibility with the previous major release.
>>>
>>> * Final bug fix release: Patches should be limited to critical bug fixes only.
>>>
>>>
>>>
>>> What does everyone thing about these changes?
>>>
>>
>> Sounds reasonable to me.  I think it would certainly benefit users not
>> to have wait so long for x.1 fixes, and it would mean downstreams have
>> to backport less.
>>
>>
>> --
>> Best regards,
>> Michał Górny
>>
>
>
>
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On 21 May 2020, at 14:59, Tom Stellard via llvm-dev wrote:

> Hi,
>
> I would like to propose a few changes to the LLVM release process.  
> The
> current process is documented here:  
> https://llvm.org/docs/HowToReleaseLLVM.html
>
> There are two parts to this proposal.  The first is a list of
> clarifications,
> which are things we are currently doing that aren't documented. The
> second
> is a list of changes which would actually modify how releases are
> currently
> managed.
>
>
>
> *** Proposed Clarifications ***
>
>
>
> **  Release manager is allowed to commit changes to the release branch
> without
>     code owner approval.  However, the release manager is encouraged
> to consult
>     with code owners or patch reviewers for non-trivial changes.
>
> It's not practical to get code owner approval every time.  Either
> because there
> is no code owner or because the number of backports is too high (e.g.
> pre-rc1 / pre-rc2).
> This proposed clarification matches how releases are currently
> managed.

If this is how things are currently managed, it’s hard to argue
against it,
but I do think that — independently — we should make a stronger
effort to
ensure that we have active code owners covering the entire codebase.

My sense is that the ownership problem is deepest in two specific parts
of the project: compiler-rt and LLVM itself.  Do you agree?

John.

>
>
> ** There is no official release criteria.
>
> We have time-based releases and when the release is 'ready' has been
> up to the discretion of the release manager.  Changing the release
> criteria is out of the scope of this proposal, but I do think it would
> be good to have a discussion about this as a community, so I'm going
> to
> start a separate thread to discuss this.
>
>
>
> *** Proposed Changes ***
>
>
>
> ** Create a time-based bug-fix release schedule.  After each major
> release, make
>    a new bug-fix release every 2 weeks for 12 weeks (6 releases
> total).
>
> ** Eliminate release candidates for bug-fix releases.
>
> The current unofficial bug-fix release schedule is:
>
> X.Y.1-rc1 (6 weeks after major release)
> X.Y.1-rc2 (10 weeks after major release)
> X.Y.1-final (12 weeks after major release)
>
> I think this change will improve the overall test coverage of the
> release branch.
> I don't think the branch itself or even the release candidates get the
> same
> level of testing as the final releases.  If we are consistently
> snapshotting
> the release branch and putting out releases, I think this will make it
> easier
> and thus more likely that users will test out the release branch code.
>
> Additionally, with more frequent bug-fix release it removes the need
> to have
> release candidate releases. Every bug-fix release (up until the last
> one)
> would serve the same purpose as our current release candidates in that
> they
> are intended to give users an easier way to test the code before the
> final
> release.
>
>
> ** Create clear rules for what kind of backports are accepted during
> each
>    release phase.
>
> * Before RC1:Patches should be limited to bug fixes, important
> optimization
>   improvements, or completion of features that were started before the
> branch
>   was created.  As with all phases, release managers and code owners
> can reject
>   patches that are deemed too invasive.
>
> * Before RC2: Patches should be limited to bug fixes or backend
> specific
>   improvements that are determined to be very safe.
>
> * Before RC3/Final: Major Release* Patches should be limited to
> critical
>   bugs or regressions.
>
> * Bug fix releases: Patches should be limited to bug fixes or very
> safe
>   and critical performance improvements.  Patches must maintain both
> API and
>   ABI compatibility with the previous major release.
>
> * Final bug fix release: Patches should be limited to critical bug
> fixes only.
>
>
>
> What does everyone thing about these changes?
>
>
> -Tom
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
Thanks Tom,

This sounds great.  Two minor comments:

On 21/05/2020 19:59, Tom Stellard via cfe-dev wrote:

> ** Create a time-based bug-fix release schedule.  After each major release, make
>     a new bug-fix release every 2 weeks for 12 weeks (6 releases total).

Presumably that's a maximum time delay?  If there are no bugs with
simple back-portable fixes found in a two-week window, I hope the
release manager isn't required to do an N.M+1 that is identical to N.M
except for the version number?


> * Before RC2: Patches should be limited to bug fixes or backend specific
>    improvements that are determined to be very safe.

It would be good to make this 'back-end or platform' rather than just
'back-end'.  If there is a small tweak that is going to be added as a
patch by the downstream package maintainer for a given platform, it's
probably a good idea to have it in the upstream repository.

David

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Openmp-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On Thu, May 21, 2020 at 8:59 PM Tom Stellard via Openmp-dev
<[hidden email]> wrote:

>
> Hi,
>
> I would like to propose a few changes to the LLVM release process.  The
> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
>
> There are two parts to this proposal.  The first is a list of clarifications,
> which are things we are currently doing that aren't documented. The second
> is a list of changes which would actually modify how releases are currently
> managed.
>
>
>
> *** Proposed Clarifications ***
>
>
>
> **  Release manager is allowed to commit changes to the release branch without
>     code owner approval.  However, the release manager is encouraged to consult
>     with code owners or patch reviewers for non-trivial changes.
>
> It's not practical to get code owner approval every time.  Either because there
> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
> This proposed clarification matches how releases are currently managed.

+1

Maybe even stronger than "is allowed to commit", I think we should
really think about it as the release manager owning the branch, and
has full authority over what goes into it or not. Consulting code
owners often makes sense of course, but for many patches, consulting
the code owner (when there is one) is an unnecessary slowdown.

> ** There is no official release criteria.
>
> We have time-based releases and when the release is 'ready' has been
> up to the discretion of the release manager.  Changing the release
> criteria is out of the scope of this proposal, but I do think it would
> be good to have a discussion about this as a community, so I'm going to
> start a separate thread to discuss this.
>
>
>
> *** Proposed Changes ***
>
>
>
> ** Create a time-based bug-fix release schedule.  After each major release, make
>    a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
>
> ** Eliminate release candidates for bug-fix releases.
>
> The current unofficial bug-fix release schedule is:
>
> X.Y.1-rc1 (6 weeks after major release)
> X.Y.1-rc2 (10 weeks after major release)
> X.Y.1-final (12 weeks after major release)
>
> I think this change will improve the overall test coverage of the release branch.
> I don't think the branch itself or even the release candidates get the same
> level of testing as the final releases.  If we are consistently snapshotting
> the release branch and putting out releases, I think this will make it easier
> and thus more likely that users will test out the release branch code.
>
> Additionally, with more frequent bug-fix release it removes the need to have
> release candidate releases. Every bug-fix release (up until the last one)
> would serve the same purpose as our current release candidates in that they
> are intended to give users an easier way to test the code before the final
> release.

My first thought is that doing all these releases sounds like a lot of
work. Would you be doing all of them, or would there be some other
arrangement? I suppose if we release this often, and also skip the
RCs, we might become more efficient at it :-)

Secondly, is having this many releases useful for downstream? One
concern might be that downstream consumers just wait for the .6 one,
and then there's no benefit and also no extra testing of the branch.
Is it mainly increasing test coverage of the branch that's the
motivation, or is it the demand for more bug-fix releases?

Not having at least one release candidate sounds a bit scary to be.
Without them we could get into a situation where everything works fine
on the release manager's machines, but is completely broken on other
platforms, and no way to fix until the next dot release. Maybe that
kind of breakage is less likely after the major release, but it still
seems it could make these dot releases less stable?

> ** Create clear rules for what kind of backports are accepted during each
>    release phase.
>
> * Before RC1:Patches should be limited to bug fixes, important optimization
>   improvements, or completion of features that were started before the branch
>   was created.  As with all phases, release managers and code owners can reject
>   patches that are deemed too invasive.
>
> * Before RC2: Patches should be limited to bug fixes or backend specific
>   improvements that are determined to be very safe.
>
> * Before RC3/Final: Major Release* Patches should be limited to critical
>   bugs or regressions.
>
> * Bug fix releases: Patches should be limited to bug fixes or very safe
>   and critical performance improvements.  Patches must maintain both API and
>   ABI compatibility with the previous major release.
>
> * Final bug fix release: Patches should be limited to critical bug fixes only.

These sound good to me.


Thanks,
Hans
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [Openmp-dev] RFC: Release process changes

Neil Nelson via cfe-dev
On Mon, 25 May 2020 at 23:10, Hans Wennborg via llvm-dev
<[hidden email]> wrote:
> +1
>
> Maybe even stronger than "is allowed to commit", I think we should
> really think about it as the release manager owning the branch, and
> has full authority over what goes into it or not. Consulting code
> owners often makes sense of course, but for many patches, consulting
> the code owner (when there is one) is an unnecessary slowdown.

Agree, with one condition: this is a "best effort" to speed up the
process, not to create a tug-of-war between release managers and code
owners.

All rules still apply: developers can ask for post-commit reversal if
a problem is found, which can delay the release further and create
merge problems if it flip-flops for too long.


> My first thought is that doing all these releases sounds like a lot of
> work.

Indeed. And AFAIK we don't have that many users for them. Most users
use some form of "internally thoroughly tested stable master".

Back then, we agreed to intermediate releases somewhat reluctantly,
but the overall value of having a "hind-sight more stable final
release" is appealing. Multiple of them? Not sure.

To me it sounds like we're replacing one process with three steps,
where the final step's quality is "guaranteed to our best ability" for
6 one-step processes, but where the quality of any intermediate step
is bound to the same testing rules.

If we test them like "final", now we have *a lot* more work in
between, and if we test them like "RC1", then we can't guarantee
quality.

There is a lot more work than just running a three-stage check +
test-suite. Distros test on their packages, users test on their
projects, etc. I don't think we can scale that process every two
weeks.

I'm happy to be wrong, and if distros, languages and downstream
projects are happy with it, so am I. But I'd ask them first (ie not in
llvm-dev), before taking a decision...

cheers,
--renato
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
These changes and clarifications make sense to me.

On Thu, 21 May 2020, 12:00 Tom Stellard via llvm-dev, <[hidden email]> wrote:
Hi,

I would like to propose a few changes to the LLVM release process.  The
current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html

There are two parts to this proposal.  The first is a list of clarifications,
which are things we are currently doing that aren't documented. The second
is a list of changes which would actually modify how releases are currently
managed.



*** Proposed Clarifications ***



**  Release manager is allowed to commit changes to the release branch without
    code owner approval.  However, the release manager is encouraged to consult
    with code owners or patch reviewers for non-trivial changes.

It's not practical to get code owner approval every time.  Either because there
is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
This proposed clarification matches how releases are currently managed.


** There is no official release criteria.

We have time-based releases and when the release is 'ready' has been
up to the discretion of the release manager.  Changing the release
criteria is out of the scope of this proposal, but I do think it would
be good to have a discussion about this as a community, so I'm going to
start a separate thread to discuss this.



*** Proposed Changes ***



** Create a time-based bug-fix release schedule.  After each major release, make
   a new bug-fix release every 2 weeks for 12 weeks (6 releases total).

** Eliminate release candidates for bug-fix releases.

The current unofficial bug-fix release schedule is:

X.Y.1-rc1 (6 weeks after major release)
X.Y.1-rc2 (10 weeks after major release)
X.Y.1-final (12 weeks after major release)

I think this change will improve the overall test coverage of the release branch.
I don't think the branch itself or even the release candidates get the same
level of testing as the final releases.  If we are consistently snapshotting
the release branch and putting out releases, I think this will make it easier
and thus more likely that users will test out the release branch code.

Additionally, with more frequent bug-fix release it removes the need to have
release candidate releases. Every bug-fix release (up until the last one)
would serve the same purpose as our current release candidates in that they
are intended to give users an easier way to test the code before the final
release.


** Create clear rules for what kind of backports are accepted during each
   release phase.

* Before RC1:Patches should be limited to bug fixes, important optimization
  improvements, or completion of features that were started before the branch
  was created.  As with all phases, release managers and code owners can reject
  patches that are deemed too invasive.

* Before RC2: Patches should be limited to bug fixes or backend specific
  improvements that are determined to be very safe.

* Before RC3/Final: Major Release* Patches should be limited to critical
  bugs or regressions.

* Bug fix releases: Patches should be limited to bug fixes or very safe
  and critical performance improvements.  Patches must maintain both API and
  ABI compatibility with the previous major release.

* Final bug fix release: Patches should be limited to critical bug fixes only.



What does everyone thing about these changes?


-Tom

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On 05/21/2020 05:54 PM, John McCall wrote:

> On 21 May 2020, at 14:59, Tom Stellard via llvm-dev wrote:
>> Hi,
>>
>> I would like to propose a few changes to the LLVM release process.  The
>> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
>>
>> There are two parts to this proposal.  The first is a list of clarifications,
>> which are things we are currently doing that aren't documented. The second
>> is a list of changes which would actually modify how releases are currently
>> managed.
>>
>>
>>
>> *** Proposed Clarifications ***
>>
>>
>>
>> **  Release manager is allowed to commit changes to the release branch without
>>     code owner approval.  However, the release manager is encouraged to consult
>>     with code owners or patch reviewers for non-trivial changes.
>>
>> It's not practical to get code owner approval every time.  Either because there
>> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
>> This proposed clarification matches how releases are currently managed.
>
> If this is how things are currently managed, it’s hard to argue against it,
> but I do think that — independently — we should make a stronger effort to
> ensure that we have active code owners covering the entire codebase.
>
> My sense is that the ownership problem is deepest in two specific parts
> of the project: compiler-rt and LLVM itself.  Do you agree?
>

There are usually less backports for compiler-rt, so that hasn't been
an issue for me, but I do agree that LLVM itself could use more code owners.

-Tom

> John.
>
>>
>>
>> ** There is no official release criteria.
>>
>> We have time-based releases and when the release is 'ready' has been
>> up to the discretion of the release manager.  Changing the release
>> criteria is out of the scope of this proposal, but I do think it would
>> be good to have a discussion about this as a community, so I'm going to
>> start a separate thread to discuss this.
>>
>>
>>
>> *** Proposed Changes ***
>>
>>
>>
>> ** Create a time-based bug-fix release schedule.  After each major release, make
>>    a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
>>
>> ** Eliminate release candidates for bug-fix releases.
>>
>> The current unofficial bug-fix release schedule is:
>>
>> X.Y.1-rc1 (6 weeks after major release)
>> X.Y.1-rc2 (10 weeks after major release)
>> X.Y.1-final (12 weeks after major release)
>>
>> I think this change will improve the overall test coverage of the release branch.
>> I don't think the branch itself or even the release candidates get the same
>> level of testing as the final releases.  If we are consistently snapshotting
>> the release branch and putting out releases, I think this will make it easier
>> and thus more likely that users will test out the release branch code.
>>
>> Additionally, with more frequent bug-fix release it removes the need to have
>> release candidate releases. Every bug-fix release (up until the last one)
>> would serve the same purpose as our current release candidates in that they
>> are intended to give users an easier way to test the code before the final
>> release.
>>
>>
>> ** Create clear rules for what kind of backports are accepted during each
>>    release phase.
>>
>> * Before RC1:Patches should be limited to bug fixes, important optimization
>>   improvements, or completion of features that were started before the branch
>>   was created.  As with all phases, release managers and code owners can reject
>>   patches that are deemed too invasive.
>>
>> * Before RC2: Patches should be limited to bug fixes or backend specific
>>   improvements that are determined to be very safe.
>>
>> * Before RC3/Final: Major Release* Patches should be limited to critical
>>   bugs or regressions.
>>
>> * Bug fix releases: Patches should be limited to bug fixes or very safe
>>   and critical performance improvements.  Patches must maintain both API and
>>   ABI compatibility with the previous major release.
>>
>> * Final bug fix release: Patches should be limited to critical bug fixes only.
>>
>>
>>
>> What does everyone thing about these changes?
>>
>>
>> -Tom
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> [hidden email]
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On 05/22/2020 04:57 AM, David Chisnall via cfe-dev wrote:

> Thanks Tom,
>
> This sounds great.  Two minor comments:
>
> On 21/05/2020 19:59, Tom Stellard via cfe-dev wrote:
>
>> ** Create a time-based bug-fix release schedule.  After each major release, make
>>     a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
>
> Presumably that's a maximum time delay?  If there are no bugs with simple back-portable fixes found in a two-week window, I hope the release manager isn't required to do an N.M+1 that is identical to N.M except for the version number?
>

That's right.  In this case, the release manager would just postpone the release another
2 weeks.

>
>> * Before RC2: Patches should be limited to bug fixes or backend specific
>>    improvements that are determined to be very safe.
>
> It would be good to make this 'back-end or platform' rather than just 'back-end'.  If there is a small tweak that is going to be added as a patch by the downstream package maintainer for a given platform, it's probably a good idea to have it in the upstream repository.
>

Does platform in this case mean operating system?

-Tom

> David
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [Openmp-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On Tue, May 26, 2020 at 4:22 AM Renato Golin via cfe-dev
<[hidden email]> wrote:

>
> On Mon, 25 May 2020 at 23:10, Hans Wennborg via llvm-dev
> <[hidden email]> wrote:
> > +1
> >
> > Maybe even stronger than "is allowed to commit", I think we should
> > really think about it as the release manager owning the branch, and
> > has full authority over what goes into it or not. Consulting code
> > owners often makes sense of course, but for many patches, consulting
> > the code owner (when there is one) is an unnecessary slowdown.
>
> Agree, with one condition: this is a "best effort" to speed up the
> process, not to create a tug-of-war between release managers and code
> owners.
>
> All rules still apply: developers can ask for post-commit reversal if
> a problem is found, which can delay the release further and create
> merge problems if it flip-flops for too long.

I think the proposed release processes make sense to me, and agree
with Renato's points.

~Aaron
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On 26 May 2020, at 18:24, Tom Stellard wrote:

> On 05/21/2020 05:54 PM, John McCall wrote:
>> On 21 May 2020, at 14:59, Tom Stellard via llvm-dev wrote:
>>> Hi,
>>>
>>> I would like to propose a few changes to the LLVM release process.  
>>> The
>>> current process is documented here:  
>>> https://llvm.org/docs/HowToReleaseLLVM.html
>>>
>>> There are two parts to this proposal.  The first is a list of
>>> clarifications,
>>> which are things we are currently doing that aren't documented. The
>>> second
>>> is a list of changes which would actually modify how releases are
>>> currently
>>> managed.
>>>
>>>
>>>
>>> *** Proposed Clarifications ***
>>>
>>>
>>>
>>> **  Release manager is allowed to commit changes to the release
>>> branch without
>>>     code owner approval.  However, the release manager is encouraged
>>> to consult
>>>     with code owners or patch reviewers for non-trivial changes.
>>>
>>> It's not practical to get code owner approval every time.  Either
>>> because there
>>> is no code owner or because the number of backports is too high
>>> (e.g. pre-rc1 / pre-rc2).
>>> This proposed clarification matches how releases are currently
>>> managed.
>>
>> If this is how things are currently managed, it’s hard to argue
>> against it,
>> but I do think that — independently — we should make a stronger
>> effort to
>> ensure that we have active code owners covering the entire codebase.
>>
>> My sense is that the ownership problem is deepest in two specific
>> parts
>> of the project: compiler-rt and LLVM itself.  Do you agree?
>>
>
> There are usually less backports for compiler-rt, so that hasn't been
> an issue for me, but I do agree that LLVM itself could use more code
> owners.

Okay, thanks.

John.
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
This set of changes sounds good to me. Thanks!

-eric

On Thu, May 21, 2020 at 12:00 PM Tom Stellard via llvm-dev <[hidden email]> wrote:
Hi,

I would like to propose a few changes to the LLVM release process.  The
current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html

There are two parts to this proposal.  The first is a list of clarifications,
which are things we are currently doing that aren't documented. The second
is a list of changes which would actually modify how releases are currently
managed.



*** Proposed Clarifications ***



**  Release manager is allowed to commit changes to the release branch without
    code owner approval.  However, the release manager is encouraged to consult
    with code owners or patch reviewers for non-trivial changes.

It's not practical to get code owner approval every time.  Either because there
is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
This proposed clarification matches how releases are currently managed.


** There is no official release criteria.

We have time-based releases and when the release is 'ready' has been
up to the discretion of the release manager.  Changing the release
criteria is out of the scope of this proposal, but I do think it would
be good to have a discussion about this as a community, so I'm going to
start a separate thread to discuss this.



*** Proposed Changes ***



** Create a time-based bug-fix release schedule.  After each major release, make
   a new bug-fix release every 2 weeks for 12 weeks (6 releases total).

** Eliminate release candidates for bug-fix releases.

The current unofficial bug-fix release schedule is:

X.Y.1-rc1 (6 weeks after major release)
X.Y.1-rc2 (10 weeks after major release)
X.Y.1-final (12 weeks after major release)

I think this change will improve the overall test coverage of the release branch.
I don't think the branch itself or even the release candidates get the same
level of testing as the final releases.  If we are consistently snapshotting
the release branch and putting out releases, I think this will make it easier
and thus more likely that users will test out the release branch code.

Additionally, with more frequent bug-fix release it removes the need to have
release candidate releases. Every bug-fix release (up until the last one)
would serve the same purpose as our current release candidates in that they
are intended to give users an easier way to test the code before the final
release.


** Create clear rules for what kind of backports are accepted during each
   release phase.

* Before RC1:Patches should be limited to bug fixes, important optimization
  improvements, or completion of features that were started before the branch
  was created.  As with all phases, release managers and code owners can reject
  patches that are deemed too invasive.

* Before RC2: Patches should be limited to bug fixes or backend specific
  improvements that are determined to be very safe.

* Before RC3/Final: Major Release* Patches should be limited to critical
  bugs or regressions.

* Bug fix releases: Patches should be limited to bug fixes or very safe
  and critical performance improvements.  Patches must maintain both API and
  ABI compatibility with the previous major release.

* Final bug fix release: Patches should be limited to critical bug fixes only.



What does everyone thing about these changes?


-Tom

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Openmp-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On 05/25/2020 05:48 AM, Hans Wennborg wrote:

> On Thu, May 21, 2020 at 8:59 PM Tom Stellard via Openmp-dev
> <[hidden email]> wrote:
>>
>> Hi,
>>
>> I would like to propose a few changes to the LLVM release process.  The
>> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
>>
>> There are two parts to this proposal.  The first is a list of clarifications,
>> which are things we are currently doing that aren't documented. The second
>> is a list of changes which would actually modify how releases are currently
>> managed.
>>
>>
>>
>> *** Proposed Clarifications ***
>>
>>
>>
>> **  Release manager is allowed to commit changes to the release branch without
>>     code owner approval.  However, the release manager is encouraged to consult
>>     with code owners or patch reviewers for non-trivial changes.
>>
>> It's not practical to get code owner approval every time.  Either because there
>> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
>> This proposed clarification matches how releases are currently managed.
>
> +1
>
> Maybe even stronger than "is allowed to commit", I think we should
> really think about it as the release manager owning the branch, and
> has full authority over what goes into it or not. Consulting code
> owners often makes sense of course, but for many patches, consulting
> the code owner (when there is one) is an unnecessary slowdown.
>
>> ** There is no official release criteria.
>>
>> We have time-based releases and when the release is 'ready' has been
>> up to the discretion of the release manager.  Changing the release
>> criteria is out of the scope of this proposal, but I do think it would
>> be good to have a discussion about this as a community, so I'm going to
>> start a separate thread to discuss this.
>>
>>
>>
>> *** Proposed Changes ***
>>
>>
>>
>> ** Create a time-based bug-fix release schedule.  After each major release, make
>>    a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
>>
>> ** Eliminate release candidates for bug-fix releases.
>>
>> The current unofficial bug-fix release schedule is:
>>
>> X.Y.1-rc1 (6 weeks after major release)
>> X.Y.1-rc2 (10 weeks after major release)
>> X.Y.1-final (12 weeks after major release)
>>
>> I think this change will improve the overall test coverage of the release branch.
>> I don't think the branch itself or even the release candidates get the same
>> level of testing as the final releases.  If we are consistently snapshotting
>> the release branch and putting out releases, I think this will make it easier
>> and thus more likely that users will test out the release branch code.
>>
>> Additionally, with more frequent bug-fix release it removes the need to have
>> release candidate releases. Every bug-fix release (up until the last one)
>> would serve the same purpose as our current release candidates in that they
>> are intended to give users an easier way to test the code before the final
>> release.
>
> My first thought is that doing all these releases sounds like a lot of
> work. Would you be doing all of them, or would there be some other
> arrangement? I suppose if we release this often, and also skip the
> RCs, we might become more efficient at it :-)
>

Yes, I would plan to do all the releases.  For 9.0.1, there were
3 RCs, so 4 releases in total.  Doing 6 instead of 4 is not that much
more work in my opinion.  Also, we may end up skipping releases if
there aren't any new changes in the branch.  But doing extra
releases would be good motivation to try to automates more parts of the
release process.

If we do feel like 6 is too many we could lengthen the interval to 3 weeks,
which would give us just 4 releases.

> Secondly, is having this many releases useful for downstream? One
> concern might be that downstream consumers just wait for the .6 one,
> and then there's no benefit and also no extra testing of the branch.
> Is it mainly increasing test coverage of the branch that's the
> motivation, or is it the demand for more bug-fix releases?
>

From me as a distro package maintainer, I'm more likely to package
a final release than a bug-fix release.  Especially if I know there won't
be another release candidate or final release coming very soon after.

Besides increasing testing coverage, I think it helps other projects
avoid having to do things like this: https://lkml.org/lkml/2020/5/5/1446
In this case, the kernel release cycle is about 2 months, so they can't
wait 3 months for a fix.


> Not having at least one release candidate sounds a bit scary to be.
> Without them we could get into a situation where everything works fine
> on the release manager's machines, but is completely broken on other
> platforms, and no way to fix until the next dot release. Maybe that
> kind of breakage is less likely after the major release, but it still
> seems it could make these dot releases less stable?
>

I think in order to do this we need to have a certain level of CI testing.
I've been experimenting with this in the last few release cycles.  Right now,
in the release branch we have tests running make-check for clang, llvm, lld,
libclc, and running make for lldb.  Each test runs on Linux, Mac, and Windows.
I feel comfortable enough with this that it would catch most obvious issues
with new bug-fix releases.  Especially since the number of changes is low
as compared to major releases.

I think there would need to be some flexibility to do another bug-fix release
ASAP if there is a major problem.  

-Tom

>> ** Create clear rules for what kind of backports are accepted during each
>>    release phase.
>>
>> * Before RC1:Patches should be limited to bug fixes, important optimization
>>   improvements, or completion of features that were started before the branch
>>   was created.  As with all phases, release managers and code owners can reject
>>   patches that are deemed too invasive.
>>
>> * Before RC2: Patches should be limited to bug fixes or backend specific
>>   improvements that are determined to be very safe.
>>
>> * Before RC3/Final: Major Release* Patches should be limited to critical
>>   bugs or regressions.
>>
>> * Bug fix releases: Patches should be limited to bug fixes or very safe
>>   and critical performance improvements.  Patches must maintain both API and
>>   ABI compatibility with the previous major release.
>>
>> * Final bug fix release: Patches should be limited to critical bug fixes only.
>
> These sound good to me.
>
>
> Thanks,
> Hans
>

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Openmp-dev] RFC: Release process changes

Neil Nelson via cfe-dev
That all makes sense to me. The new process sgtm.

On Wed, May 27, 2020 at 5:20 AM Tom Stellard <[hidden email]> wrote:

>
> On 05/25/2020 05:48 AM, Hans Wennborg wrote:
> > On Thu, May 21, 2020 at 8:59 PM Tom Stellard via Openmp-dev
> > <[hidden email]> wrote:
> >>
> >> Hi,
> >>
> >> I would like to propose a few changes to the LLVM release process.  The
> >> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
> >>
> >> There are two parts to this proposal.  The first is a list of clarifications,
> >> which are things we are currently doing that aren't documented. The second
> >> is a list of changes which would actually modify how releases are currently
> >> managed.
> >>
> >>
> >>
> >> *** Proposed Clarifications ***
> >>
> >>
> >>
> >> **  Release manager is allowed to commit changes to the release branch without
> >>     code owner approval.  However, the release manager is encouraged to consult
> >>     with code owners or patch reviewers for non-trivial changes.
> >>
> >> It's not practical to get code owner approval every time.  Either because there
> >> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
> >> This proposed clarification matches how releases are currently managed.
> >
> > +1
> >
> > Maybe even stronger than "is allowed to commit", I think we should
> > really think about it as the release manager owning the branch, and
> > has full authority over what goes into it or not. Consulting code
> > owners often makes sense of course, but for many patches, consulting
> > the code owner (when there is one) is an unnecessary slowdown.
> >
> >> ** There is no official release criteria.
> >>
> >> We have time-based releases and when the release is 'ready' has been
> >> up to the discretion of the release manager.  Changing the release
> >> criteria is out of the scope of this proposal, but I do think it would
> >> be good to have a discussion about this as a community, so I'm going to
> >> start a separate thread to discuss this.
> >>
> >>
> >>
> >> *** Proposed Changes ***
> >>
> >>
> >>
> >> ** Create a time-based bug-fix release schedule.  After each major release, make
> >>    a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
> >>
> >> ** Eliminate release candidates for bug-fix releases.
> >>
> >> The current unofficial bug-fix release schedule is:
> >>
> >> X.Y.1-rc1 (6 weeks after major release)
> >> X.Y.1-rc2 (10 weeks after major release)
> >> X.Y.1-final (12 weeks after major release)
> >>
> >> I think this change will improve the overall test coverage of the release branch.
> >> I don't think the branch itself or even the release candidates get the same
> >> level of testing as the final releases.  If we are consistently snapshotting
> >> the release branch and putting out releases, I think this will make it easier
> >> and thus more likely that users will test out the release branch code.
> >>
> >> Additionally, with more frequent bug-fix release it removes the need to have
> >> release candidate releases. Every bug-fix release (up until the last one)
> >> would serve the same purpose as our current release candidates in that they
> >> are intended to give users an easier way to test the code before the final
> >> release.
> >
> > My first thought is that doing all these releases sounds like a lot of
> > work. Would you be doing all of them, or would there be some other
> > arrangement? I suppose if we release this often, and also skip the
> > RCs, we might become more efficient at it :-)
> >
>
> Yes, I would plan to do all the releases.  For 9.0.1, there were
> 3 RCs, so 4 releases in total.  Doing 6 instead of 4 is not that much
> more work in my opinion.  Also, we may end up skipping releases if
> there aren't any new changes in the branch.  But doing extra
> releases would be good motivation to try to automates more parts of the
> release process.
>
> If we do feel like 6 is too many we could lengthen the interval to 3 weeks,
> which would give us just 4 releases.
>
> > Secondly, is having this many releases useful for downstream? One
> > concern might be that downstream consumers just wait for the .6 one,
> > and then there's no benefit and also no extra testing of the branch.
> > Is it mainly increasing test coverage of the branch that's the
> > motivation, or is it the demand for more bug-fix releases?
> >
>
> From me as a distro package maintainer, I'm more likely to package
> a final release than a bug-fix release.  Especially if I know there won't
> be another release candidate or final release coming very soon after.
>
> Besides increasing testing coverage, I think it helps other projects
> avoid having to do things like this: https://lkml.org/lkml/2020/5/5/1446
> In this case, the kernel release cycle is about 2 months, so they can't
> wait 3 months for a fix.
>
>
> > Not having at least one release candidate sounds a bit scary to be.
> > Without them we could get into a situation where everything works fine
> > on the release manager's machines, but is completely broken on other
> > platforms, and no way to fix until the next dot release. Maybe that
> > kind of breakage is less likely after the major release, but it still
> > seems it could make these dot releases less stable?
> >
>
> I think in order to do this we need to have a certain level of CI testing.
> I've been experimenting with this in the last few release cycles.  Right now,
> in the release branch we have tests running make-check for clang, llvm, lld,
> libclc, and running make for lldb.  Each test runs on Linux, Mac, and Windows.
> I feel comfortable enough with this that it would catch most obvious issues
> with new bug-fix releases.  Especially since the number of changes is low
> as compared to major releases.
>
> I think there would need to be some flexibility to do another bug-fix release
> ASAP if there is a major problem.
>
> -Tom
>
> >> ** Create clear rules for what kind of backports are accepted during each
> >>    release phase.
> >>
> >> * Before RC1:Patches should be limited to bug fixes, important optimization
> >>   improvements, or completion of features that were started before the branch
> >>   was created.  As with all phases, release managers and code owners can reject
> >>   patches that are deemed too invasive.
> >>
> >> * Before RC2: Patches should be limited to bug fixes or backend specific
> >>   improvements that are determined to be very safe.
> >>
> >> * Before RC3/Final: Major Release* Patches should be limited to critical
> >>   bugs or regressions.
> >>
> >> * Bug fix releases: Patches should be limited to bug fixes or very safe
> >>   and critical performance improvements.  Patches must maintain both API and
> >>   ABI compatibility with the previous major release.
> >>
> >> * Final bug fix release: Patches should be limited to critical bug fixes only.
> >
> > These sound good to me.
> >
> >
> > Thanks,
> > Hans
> >
>
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] [Openmp-dev] RFC: Release process changes

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On Tue, May 26, 2020 at 08:19:57PM -0700, Tom Stellard via lldb-dev wrote:

> On 05/25/2020 05:48 AM, Hans Wennborg wrote:
> > On Thu, May 21, 2020 at 8:59 PM Tom Stellard via Openmp-dev
> > <[hidden email]> wrote:
> >>
> >> Hi,
> >>
> >> I would like to propose a few changes to the LLVM release process.  The
> >> current process is documented here:  https://llvm.org/docs/HowToReleaseLLVM.html
> >>
> >> There are two parts to this proposal.  The first is a list of clarifications,
> >> which are things we are currently doing that aren't documented. The second
> >> is a list of changes which would actually modify how releases are currently
> >> managed.
> >>
> >>
> >>
> >> *** Proposed Clarifications ***
> >>
> >>
> >>
> >> **  Release manager is allowed to commit changes to the release branch without
> >>     code owner approval.  However, the release manager is encouraged to consult
> >>     with code owners or patch reviewers for non-trivial changes.
> >>
> >> It's not practical to get code owner approval every time.  Either because there
> >> is no code owner or because the number of backports is too high (e.g. pre-rc1 / pre-rc2).
> >> This proposed clarification matches how releases are currently managed.
> >
> > +1
> >
> > Maybe even stronger than "is allowed to commit", I think we should
> > really think about it as the release manager owning the branch, and
> > has full authority over what goes into it or not. Consulting code
> > owners often makes sense of course, but for many patches, consulting
> > the code owner (when there is one) is an unnecessary slowdown.
> >
> >> ** There is no official release criteria.
> >>
> >> We have time-based releases and when the release is 'ready' has been
> >> up to the discretion of the release manager.  Changing the release
> >> criteria is out of the scope of this proposal, but I do think it would
> >> be good to have a discussion about this as a community, so I'm going to
> >> start a separate thread to discuss this.
> >>
> >>
> >>
> >> *** Proposed Changes ***
> >>
> >>
> >>
> >> ** Create a time-based bug-fix release schedule.  After each major release, make
> >>    a new bug-fix release every 2 weeks for 12 weeks (6 releases total).
> >>
> >> ** Eliminate release candidates for bug-fix releases.
> >>
> >> The current unofficial bug-fix release schedule is:
> >>
> >> X.Y.1-rc1 (6 weeks after major release)
> >> X.Y.1-rc2 (10 weeks after major release)
> >> X.Y.1-final (12 weeks after major release)
> >>
> >> I think this change will improve the overall test coverage of the release branch.
> >> I don't think the branch itself or even the release candidates get the same
> >> level of testing as the final releases.  If we are consistently snapshotting
> >> the release branch and putting out releases, I think this will make it easier
> >> and thus more likely that users will test out the release branch code.
> >>
> >> Additionally, with more frequent bug-fix release it removes the need to have
> >> release candidate releases. Every bug-fix release (up until the last one)
> >> would serve the same purpose as our current release candidates in that they
> >> are intended to give users an easier way to test the code before the final
> >> release.
> >
> > My first thought is that doing all these releases sounds like a lot of
> > work. Would you be doing all of them, or would there be some other
> > arrangement? I suppose if we release this often, and also skip the
> > RCs, we might become more efficient at it :-)
> >
>
> Yes, I would plan to do all the releases.  For 9.0.1, there were
> 3 RCs, so 4 releases in total.  Doing 6 instead of 4 is not that much
> more work in my opinion.  Also, we may end up skipping releases if
> there aren't any new changes in the branch.  But doing extra
> releases would be good motivation to try to automates more parts of the
> release process.
>
> If we do feel like 6 is too many we could lengthen the interval to 3 weeks,
> which would give us just 4 releases.
>
> > Secondly, is having this many releases useful for downstream? One
> > concern might be that downstream consumers just wait for the .6 one,
> > and then there's no benefit and also no extra testing of the branch.
> > Is it mainly increasing test coverage of the branch that's the
> > motivation, or is it the demand for more bug-fix releases?
> >
>
> From me as a distro package maintainer, I'm more likely to package
> a final release than a bug-fix release.  Especially if I know there won't
> be another release candidate or final release coming very soon after.
As the FreeBSD package maintainer, a regular cadence of releases without
RCs seems fine with sufficient CI.  Personally, every 3 weeks makes
me happier than every 2.  It's not a lot of work to do updates (I've
automated most of the bits that aren't dealing with removing patches
that have sense been merged), but users get grumpy if I update the
package too often (lots of FreeBSD users build their own packages on
surprisingly under-powered systems.)

-- Brooks

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

signature.asc (465 bytes) Download Attachment