RFC: Using GitHub Actions for CI testing on the release/* branches

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
Hi,

I would like to start using GitHub Actions[1] for CI testing on the release/*
branches.  As far as I know we don't have any buildbots listening to the
release branches, and I think GitHub Actions are a good way for us to quickly
bring-up some CI jobs there.

My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.

I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
but in order to get these running in the main repository, I would need to:

1. Create a new repository in the LLVM organization called 'actions' for storing some custom
builds steps for our CI jobs (see [4]).
2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
branch.

In the future, I would also like to add buil and tests jobs for other sub-projects
once I am able to get those working.

In addition to being used for post-commit testing, having these CI definitions in the
main tree will make it easier for me (or anyone) to do pre-commit testing for the
release branch in a personal fork.  It will also allow me to experiment with some new
workflows to help make managing the releases much easier.

I think this will be a good way to test Actions in a low traffic environment to
see if they are something we would want to use for CI on the master branch.

Given that we are close to the end of the 9.0.1 cycle, unless there are any
strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
usefulness.  Let me know what you think.

Thanks,
Tom

[1] https://github.com/features/actions
[2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
[3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
[4] https://github.com/tstellar/actions

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
Not having given it deep thought/analysis, nor understanding much of the GIT infrastructure here, but: Sounds good to me, for whatever that's worth :)

On Mon, Nov 11, 2019 at 4:32 PM Tom Stellard via llvm-dev <[hidden email]> wrote:
Hi,

I would like to start using GitHub Actions[1] for CI testing on the release/*
branches.  As far as I know we don't have any buildbots listening to the
release branches, and I think GitHub Actions are a good way for us to quickly
bring-up some CI jobs there.

My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.

I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
but in order to get these running in the main repository, I would need to:

1. Create a new repository in the LLVM organization called 'actions' for storing some custom
builds steps for our CI jobs (see [4]).
2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
branch.

In the future, I would also like to add buil and tests jobs for other sub-projects
once I am able to get those working.

In addition to being used for post-commit testing, having these CI definitions in the
main tree will make it easier for me (or anyone) to do pre-commit testing for the
release branch in a personal fork.  It will also allow me to experiment with some new
workflows to help make managing the releases much easier.

I think this will be a good way to test Actions in a low traffic environment to
see if they are something we would want to use for CI on the master branch.

Given that we are close to the end of the 9.0.1 cycle, unless there are any
strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
usefulness.  Let me know what you think.

Thanks,
Tom

[1] https://github.com/features/actions
[2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
[3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
[4] https://github.com/tstellar/actions

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] [llvm-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
Although I've not had any Github Action experience, and despite being one who has raised objections to other parts of adopting github features, this sounds like a sensible idea. It seems odd to me that we don't already have some form of CI for our releases, so anything that improves on that is great.

James

On Tue, 12 Nov 2019 at 00:34, David Blaikie via lldb-dev <[hidden email]> wrote:
Not having given it deep thought/analysis, nor understanding much of the GIT infrastructure here, but: Sounds good to me, for whatever that's worth :)

On Mon, Nov 11, 2019 at 4:32 PM Tom Stellard via llvm-dev <[hidden email]> wrote:
Hi,

I would like to start using GitHub Actions[1] for CI testing on the release/*
branches.  As far as I know we don't have any buildbots listening to the
release branches, and I think GitHub Actions are a good way for us to quickly
bring-up some CI jobs there.

My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.

I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
but in order to get these running in the main repository, I would need to:

1. Create a new repository in the LLVM organization called 'actions' for storing some custom
builds steps for our CI jobs (see [4]).
2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
branch.

In the future, I would also like to add buil and tests jobs for other sub-projects
once I am able to get those working.

In addition to being used for post-commit testing, having these CI definitions in the
main tree will make it easier for me (or anyone) to do pre-commit testing for the
release branch in a personal fork.  It will also allow me to experiment with some new
workflows to help make managing the releases much easier.

I think this will be a good way to test Actions in a low traffic environment to
see if they are something we would want to use for CI on the master branch.

Given that we are close to the end of the 9.0.1 cycle, unless there are any
strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
usefulness.  Let me know what you think.

Thanks,
Tom

[1] https://github.com/features/actions
[2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
[3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
[4] https://github.com/tstellar/actions

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
lldb-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] [llvm-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
Hi Tom,

This sounds very interesting! +1 from me.
What does this imply for the release testers?
Would it be possible / desirable for us to add self-hosted runners for
other architectures? I'm assuming GitHub only provides x86, right? I
totally missed any details about that in their docs, but it seems to
be implied here [1]. I'm not saying we should go all-possible-arches
from the start, we can definitely try it out only for x86 this time
around if you think that would be the best approach.

Thanks,
Diana

[1] https://github.blog/2019-11-05-self-hosted-runners-for-github-actions-is-now-in-beta/

On Tue, 12 Nov 2019 at 10:33, James Henderson via lldb-dev
<[hidden email]> wrote:

>
> Although I've not had any Github Action experience, and despite being one who has raised objections to other parts of adopting github features, this sounds like a sensible idea. It seems odd to me that we don't already have some form of CI for our releases, so anything that improves on that is great.
>
> James
>
> On Tue, 12 Nov 2019 at 00:34, David Blaikie via lldb-dev <[hidden email]> wrote:
>>
>> Not having given it deep thought/analysis, nor understanding much of the GIT infrastructure here, but: Sounds good to me, for whatever that's worth :)
>>
>> On Mon, Nov 11, 2019 at 4:32 PM Tom Stellard via llvm-dev <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> I would like to start using GitHub Actions[1] for CI testing on the release/*
>>> branches.  As far as I know we don't have any buildbots listening to the
>>> release branches, and I think GitHub Actions are a good way for us to quickly
>>> bring-up some CI jobs there.
>>>
>>> My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
>>> One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
>>> Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.
>>>
>>> I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
>>> but in order to get these running in the main repository, I would need to:
>>>
>>> 1. Create a new repository in the LLVM organization called 'actions' for storing some custom
>>> builds steps for our CI jobs (see [4]).
>>> 2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
>>> branch.
>>>
>>> In the future, I would also like to add buil and tests jobs for other sub-projects
>>> once I am able to get those working.
>>>
>>> In addition to being used for post-commit testing, having these CI definitions in the
>>> main tree will make it easier for me (or anyone) to do pre-commit testing for the
>>> release branch in a personal fork.  It will also allow me to experiment with some new
>>> workflows to help make managing the releases much easier.
>>>
>>> I think this will be a good way to test Actions in a low traffic environment to
>>> see if they are something we would want to use for CI on the master branch.
>>>
>>> Given that we are close to the end of the 9.0.1 cycle, unless there are any
>>> strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
>>> usefulness.  Let me know what you think.
>>>
>>> Thanks,
>>> Tom
>>>
>>> [1] https://github.com/features/actions
>>> [2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
>>> [3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
>>> [4] https://github.com/tstellar/actions
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> [hidden email]
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> lldb-dev mailing list
>> [hidden email]
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
> _______________________________________________
> lldb-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
In reply to this post by Tom Stellard via cfe-dev
On Tue, Nov 12, 2019 at 1:32 AM Tom Stellard via lldb-dev
<[hidden email]> wrote:

>
> Hi,
>
> I would like to start using GitHub Actions[1] for CI testing on the release/*
> branches.  As far as I know we don't have any buildbots listening to the
> release branches, and I think GitHub Actions are a good way for us to quickly
> bring-up some CI jobs there.
>
> My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
> One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
> Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.
>
> I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
> but in order to get these running in the main repository, I would need to:
>
> 1. Create a new repository in the LLVM organization called 'actions' for storing some custom
> builds steps for our CI jobs (see [4]).
> 2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
> branch.
>
> In the future, I would also like to add buil and tests jobs for other sub-projects
> once I am able to get those working.
>
> In addition to being used for post-commit testing, having these CI definitions in the
> main tree will make it easier for me (or anyone) to do pre-commit testing for the
> release branch in a personal fork.  It will also allow me to experiment with some new
> workflows to help make managing the releases much easier.
>
> I think this will be a good way to test Actions in a low traffic environment to
> see if they are something we would want to use for CI on the master branch.
>
> Given that we are close to the end of the 9.0.1 cycle, unless there are any
> strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
> usefulness.  Let me know what you think.
>
> Thanks,
> Tom
>
> [1] https://github.com/features/actions
> [2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
> [3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
> [4] https://github.com/tstellar/actions

Sounds great to me!
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
Hey Tom,

That sounds really useful. Would it be possible to include LLDB as
well? We have a subset of tests (unit & lit) that can be run without
Python/SWIG by passing LLDB_DISABLE_PYTHON=ON to CMake.

Thanks,
Jonas

On Tue, Nov 12, 2019 at 2:35 AM Hans Wennborg via cfe-dev
<[hidden email]> wrote:

>
> On Tue, Nov 12, 2019 at 1:32 AM Tom Stellard via lldb-dev
> <[hidden email]> wrote:
> >
> > Hi,
> >
> > I would like to start using GitHub Actions[1] for CI testing on the release/*
> > branches.  As far as I know we don't have any buildbots listening to the
> > release branches, and I think GitHub Actions are a good way for us to quickly
> > bring-up some CI jobs there.
> >
> > My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
> > One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
> > Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.
> >
> > I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
> > but in order to get these running in the main repository, I would need to:
> >
> > 1. Create a new repository in the LLVM organization called 'actions' for storing some custom
> > builds steps for our CI jobs (see [4]).
> > 2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
> > branch.
> >
> > In the future, I would also like to add buil and tests jobs for other sub-projects
> > once I am able to get those working.
> >
> > In addition to being used for post-commit testing, having these CI definitions in the
> > main tree will make it easier for me (or anyone) to do pre-commit testing for the
> > release branch in a personal fork.  It will also allow me to experiment with some new
> > workflows to help make managing the releases much easier.
> >
> > I think this will be a good way to test Actions in a low traffic environment to
> > see if they are something we would want to use for CI on the master branch.
> >
> > Given that we are close to the end of the 9.0.1 cycle, unless there are any
> > strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
> > usefulness.  Let me know what you think.
> >
> > Thanks,
> > Tom
> >
> > [1] https://github.com/features/actions
> > [2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
> > [3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
> > [4] https://github.com/tstellar/actions
>
> Sounds great to me!
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
On 11/12/2019 10:03 AM, Jonas Devlieghere wrote:
> Hey Tom,
>
> That sounds really useful. Would it be possible to include LLDB as
> well? We have a subset of tests (unit & lit) that can be run without
> Python/SWIG by passing LLDB_DISABLE_PYTHON=ON to CMake.
>

I can try to add an lldb builder.  Would you be able to provide me
with the cmake arguments plus a check target where all tests are expected
to pass?

Thanks,
Tom

> Thanks,
> Jonas
>
> On Tue, Nov 12, 2019 at 2:35 AM Hans Wennborg via cfe-dev
> <[hidden email]> wrote:
>>
>> On Tue, Nov 12, 2019 at 1:32 AM Tom Stellard via lldb-dev
>> <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> I would like to start using GitHub Actions[1] for CI testing on the release/*
>>> branches.  As far as I know we don't have any buildbots listening to the
>>> release branches, and I think GitHub Actions are a good way for us to quickly
>>> bring-up some CI jobs there.
>>>
>>> My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
>>> One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
>>> Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.
>>>
>>> I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
>>> but in order to get these running in the main repository, I would need to:
>>>
>>> 1. Create a new repository in the LLVM organization called 'actions' for storing some custom
>>> builds steps for our CI jobs (see [4]).
>>> 2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
>>> branch.
>>>
>>> In the future, I would also like to add buil and tests jobs for other sub-projects
>>> once I am able to get those working.
>>>
>>> In addition to being used for post-commit testing, having these CI definitions in the
>>> main tree will make it easier for me (or anyone) to do pre-commit testing for the
>>> release branch in a personal fork.  It will also allow me to experiment with some new
>>> workflows to help make managing the releases much easier.
>>>
>>> I think this will be a good way to test Actions in a low traffic environment to
>>> see if they are something we would want to use for CI on the master branch.
>>>
>>> Given that we are close to the end of the 9.0.1 cycle, unless there are any
>>> strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
>>> usefulness.  Let me know what you think.
>>>
>>> Thanks,
>>> Tom
>>>
>>> [1] https://github.com/features/actions
>>> [2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
>>> [3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
>>> [4] https://github.com/tstellar/actions
>>
>> Sounds great to me!
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
In reply to this post by Tom Stellard via cfe-dev
+1, I'll be really curious to know how this works out.

Philip

On 11/11/19 4:32 PM, Tom Stellard via cfe-dev wrote:

> Hi,
>
> I would like to start using GitHub Actions[1] for CI testing on the release/*
> branches.  As far as I know we don't have any buildbots listening to the
> release branches, and I think GitHub Actions are a good way for us to quickly
> bring-up some CI jobs there.
>
> My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
> One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
> Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.
>
> I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
> but in order to get these running in the main repository, I would need to:
>
> 1. Create a new repository in the LLVM organization called 'actions' for storing some custom
> builds steps for our CI jobs (see [4]).
> 2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
> branch.
>
> In the future, I would also like to add buil and tests jobs for other sub-projects
> once I am able to get those working.
>
> In addition to being used for post-commit testing, having these CI definitions in the
> main tree will make it easier for me (or anyone) to do pre-commit testing for the
> release branch in a personal fork.  It will also allow me to experiment with some new
> workflows to help make managing the releases much easier.
>
> I think this will be a good way to test Actions in a low traffic environment to
> see if they are something we would want to use for CI on the master branch.
>
> Given that we are close to the end of the 9.0.1 cycle, unless there are any
> strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
> usefulness.  Let me know what you think.
>
> Thanks,
> Tom
>
> [1] https://github.com/features/actions
> [2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
> [3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
> [4] https://github.com/tstellar/actions
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
In reply to this post by Tom Stellard via cfe-dev
On 11/11/2019 04:32 PM, Tom Stellard wrote:

> Hi,
>
> I would like to start using GitHub Actions[1] for CI testing on the release/*
> branches.  As far as I know we don't have any buildbots listening to the
> release branches, and I think GitHub Actions are a good way for us to quickly
> bring-up some CI jobs there.
>
> My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
> One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
> Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.
>
> I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
> but in order to get these running in the main repository, I would need to:
>
> 1. Create a new repository in the LLVM organization called 'actions' for storing some custom
> builds steps for our CI jobs (see [4]).
> 2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
> branch.
>
> In the future, I would also like to add buil and tests jobs for other sub-projects
> once I am able to get those working.
>
> In addition to being used for post-commit testing, having these CI definitions in the
> main tree will make it easier for me (or anyone) to do pre-commit testing for the
> release branch in a personal fork.  It will also allow me to experiment with some new
> workflows to help make managing the releases much easier.
>
> I think this will be a good way to test Actions in a low traffic environment to
> see if they are something we would want to use for CI on the master branch.
>
> Given that we are close to the end of the 9.0.1 cycle, unless there are any
> strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
> usefulness.  Let me know what you think.
>

The CI tests are running now on the stable branch.  Here is the
first run [1].  Passing commits[2] now have a green check mark in
the web UI that you can click on to view the results.

These tests have already caught 1 bug that did not appear in my local
builds, so they are proving to be very useful.

Now that this is in place, I'm going to start looking into some more automation
around backporting fixes to the stable branch.  I think we could do something like
filing backport requests as a GitHub issues and then have that issue automatically
trigger a pull request and do some pre-commit testing.  I'm still trying to learn
what is technically possible with GitHub and if anyone has any ideas or suggestions,
let me know.  I'm hoping to have something in place for the 10.0.1 release cycle.

Thanks,
Tom

[1] https://github.com/llvm/llvm-project/commit/6c86aa62d502ac2df58e207d4792544cc96d79bd/checks?check_suite_id=316648311
[2] https://github.com/llvm/llvm-project/commit/6c86aa62d502ac2df58e207d4792544cc96d79bd

> Thanks,
> Tom
>
> [1] https://github.com/features/actions
> [2] https://github.com/tstellar/llvm-project/commit/952d80e8509ecc95797b2ddbf1af40abad2dcf4e/checks?check_suite_id=305765621
> [3] https://github.com/tstellar/llvm-project/commit/6d74f1b81632ef081dffa1e0c0434f47d4954423/checks?check_suite_id=303074176
> [4] https://github.com/tstellar/actions
>

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Openmp-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
In reply to this post by Tom Stellard via cfe-dev
> Hi Tom,
>
> This sounds very interesting! +1 from me.
> What does this imply for the release testers?
> Would it be possible / desirable for us to add self-hosted runners for
> other architectures? I'm assuming GitHub only provides x86, right? I
> totally missed any details about that in their docs, but it seems to
> be implied here [1]. I'm not saying we should go all-possible-arches
> from the start, we can definitely try it out only for x86 this time
> around if you think that would be the best approach.
>

Nothing changes for release testers at this point.  The main goal here is
to get some post-commit testing so we can catch issues in the branch early.

Longer term I think have self-hosted runners would be great, because it would
allow us to fully automate the testing and uploading we do when a release gets
tagged.

You are correct that GitHub only provides x86 machines, however, they don't
have enough disk space (14GB) to be able to run the test-release.sh script, so
adding x86 self-hosted builders with more disk space would still be very useful.

-Tom
 
>
> [1] https://github.blog/2019-11-05-self-hosted-runners-for-github-actions-is-now-in-beta/
>
> Thanks,
> Diana

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [Openmp-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
Hi Tom,

First of: I'm a big fan of adding more automatic tests to find bugs. Great work!

On the other hand, I think we should consider creating some sort of test strategy for the LLVM project: 
  • What tests do we expect users to run before uploading patches?
  • What tests do we expect users to run before merging?
  • What tests do we run after merging?
  • What failures must be fixed, what failures can be ignored?
  • What do we check for on the build bots?
  • What do we check for on the release branches?
  • What do we check for on the pre-merge tests?
  • Which CI tools do we want to use (github, Jenkins, bulidbot, ...)?
  • What about running clang-tidy and clang-format?
  • What CMake configurations should we check? (Release/Debug, assertions, ...)
  • Do we want to run tests with Sanitizers?
  • Which of these systems do we expect users to monitor?
I suppose it would be good to have a document that summarizes all of this so that we
  1. do not test the same thing twice and
  2. do not miss any important checks.
Does something like this exist? 
Is anyone working on this?

Best,
Christian

On Wed, Nov 20, 2019 at 7:26 AM Tom Stellard via llvm-dev <[hidden email]> wrote:
> Hi Tom,
>
> This sounds very interesting! +1 from me.
> What does this imply for the release testers?
> Would it be possible / desirable for us to add self-hosted runners for
> other architectures? I'm assuming GitHub only provides x86, right? I
> totally missed any details about that in their docs, but it seems to
> be implied here [1]. I'm not saying we should go all-possible-arches
> from the start, we can definitely try it out only for x86 this time
> around if you think that would be the best approach.
>

Nothing changes for release testers at this point.  The main goal here is
to get some post-commit testing so we can catch issues in the branch early.

Longer term I think have self-hosted runners would be great, because it would
allow us to fully automate the testing and uploading we do when a release gets
tagged.

You are correct that GitHub only provides x86 machines, however, they don't
have enough disk space (14GB) to be able to run the test-release.sh script, so
adding x86 self-hosted builders with more disk space would still be very useful.

-Tom

>
> [1] https://github.blog/2019-11-05-self-hosted-runners-for-github-actions-is-now-in-beta/
>
> Thanks,
> Diana

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


--
Best,
Christian

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [Openmp-dev] RFC: Using GitHub Actions for CI testing on the release/* branches

Tom Stellard via cfe-dev
> In addition to being used for post-commit testing, having these CI definitions in the
> main tree will make it easier for me (or anyone) to do pre-commit testing for the
> release branch in a personal fork.

I find this part especially useful. I'm using these actions on a personal branch to validate patches on Linux, OSX and Windows, which gives me more confidence on its portability.

On Thu, Nov 21, 2019 at 10:15 AM Christian Kühnel via llvm-dev <[hidden email]> wrote:
Hi Tom,

First of: I'm a big fan of adding more automatic tests to find bugs. Great work!

On the other hand, I think we should consider creating some sort of test strategy for the LLVM project: 
  • What tests do we expect users to run before uploading patches?
  • What tests do we expect users to run before merging?
  • What tests do we run after merging?
  • What failures must be fixed, what failures can be ignored?
  • What do we check for on the build bots?
  • What do we check for on the release branches?
  • What do we check for on the pre-merge tests?
  • Which CI tools do we want to use (github, Jenkins, bulidbot, ...)?
  • What about running clang-tidy and clang-format?
  • What CMake configurations should we check? (Release/Debug, assertions, ...)
  • Do we want to run tests with Sanitizers?
  • Which of these systems do we expect users to monitor?
I suppose it would be good to have a document that summarizes all of this so that we
  1. do not test the same thing twice and
  2. do not miss any important checks.
Does something like this exist? 
Is anyone working on this?

Best,
Christian

On Wed, Nov 20, 2019 at 7:26 AM Tom Stellard via llvm-dev <[hidden email]> wrote:
> Hi Tom,
>
> This sounds very interesting! +1 from me.
> What does this imply for the release testers?
> Would it be possible / desirable for us to add self-hosted runners for
> other architectures? I'm assuming GitHub only provides x86, right? I
> totally missed any details about that in their docs, but it seems to
> be implied here [1]. I'm not saying we should go all-possible-arches
> from the start, we can definitely try it out only for x86 this time
> around if you think that would be the best approach.
>

Nothing changes for release testers at this point.  The main goal here is
to get some post-commit testing so we can catch issues in the branch early.

Longer term I think have self-hosted runners would be great, because it would
allow us to fully automate the testing and uploading we do when a release gets
tagged.

You are correct that GitHub only provides x86 machines, however, they don't
have enough disk space (14GB) to be able to run the test-release.sh script, so
adding x86 self-hosted builders with more disk space would still be very useful.

-Tom

>
> [1] https://github.blog/2019-11-05-self-hosted-runners-for-github-actions-is-now-in-beta/
>
> Thanks,
> Diana

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


--
Best,
Christian
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev