GN build roundtable summary; adding GN build files to the repo

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
Hi,

first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.

At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].

I had been worried that it would be a lot of work to keep the build files up to date, but I've been using this for all my LLVM/clang/lld development the last 8 months, and it turned out to not be a big problem -- LLVM's build files don't change very often, and GN build files are a pleasure to work with in my opinion.

I gave the lightning talk just to talk about my personal workflow, but there was a lot of interest. We had a roundtable on the next day, and about 20 people said they'd be interested in using this for their development too. The main request was that the .gn files are checked in upstream, so that we can collaborate on keeping them working. Two to three orgs even said they'd be interested in moving their buildbots to GN.

As mentioned at the top, the intention here is not to replace cmake, only to offer an opt-in alternative for people who are interested in it. Keeping the GN build going would be the responsibility of people using it, not of the general LLVM community. If this fails to find use and bitrots, we can easily remove it again.

Are there any concerns with checking in GN files? I've put some initial docs for the GN build at https://reviews.llvm.org/D53944 ; it describes what the GN build is and is not, what its advantages are (speed and easier configurability), and some points about the philosophy behind the LLVM GN build.

If folks are generally ok with GN files living in-tree, I'll start sending patches for gradually adding gn files through the regular review process.

If having a BUILD.gn file in every directory being confusing is a concern, GN has the concept of a "secondary tree" so that all GN files could be below llvm/gn/tree/{llvm,clang,lld,...}.

Cheers,
Nico


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev


On Wed, Oct 31, 2018, 1:18 PM Nico Weber via cfe-dev <[hidden email]> wrote:
Hi,

first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.

At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].


I'm in favor of including the gn build files. 

Aside: build of gn itself failed for me on Ubuntu 14, link errors related to unsupported relocations, AFAICT this is just an incompatibility between the Debian filesystem downloaded and Ubuntu 14.  Could you provide a release tarball of gn for a couple of key platforms?

-Brian

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
On Wed, Oct 31, 2018 at 11:30 AM Brian Cain <[hidden email]> wrote:


On Wed, Oct 31, 2018, 1:18 PM Nico Weber via cfe-dev <[hidden email]> wrote:
Hi,

first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.

At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].


I'm in favor of including the gn build files. 

Aside: build of gn itself failed for me on Ubuntu 14, link errors related to unsupported relocations, AFAICT this is just an incompatibility between the Debian filesystem downloaded and Ubuntu 14.  Could you provide a release tarball of gn for a couple of key platforms?

We provide prebuilts for GN now, see https://gn.googlesource.com/gn/#getting-started 

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev


On Wed, Oct 31, 2018, 1:32 PM Petr Hosek <[hidden email]> wrote:
On Wed, Oct 31, 2018 at 11:30 AM Brian Cain <[hidden email]> wrote:


On Wed, Oct 31, 2018, 1:18 PM Nico Weber via cfe-dev <[hidden email]> wrote:
Hi,

first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.

At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].


I'm in favor of including the gn build files. 

Aside: build of gn itself failed for me on Ubuntu 14, link errors related to unsupported relocations, AFAICT this is just an incompatibility between the Debian filesystem downloaded and Ubuntu 14.  Could you provide a release tarball of gn for a couple of key platforms?

We provide prebuilts for GN now, see https://gn.googlesource.com/gn/#getting-started 

Oh, I see, that works well. Thanks!

-Brian

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
On 31/10/2018 18:18, Nico Weber via cfe-dev wrote:
> Keeping the GN build going would be the responsibility of people using
> it, not of the general LLVM community.

If these files exist in the repo, I could imagine that reviewers will
reject patches that might/likely break the gn files.

I could also imagine that contributors will be pressured to fix breakage
of gn files that they introduce, or commits being reverted etc.

So, I'm not sure that a policy of 'it's ok to break this stuff' will
actually work.

Do you think there will be some point in time that these files will be
out of 'experimental' status?

Thanks,

Stephen.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
Hello!

On 10/31/18 7:18 PM, Nico Weber via llvm-dev wrote:
> At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].
> (...)
> Are there any concerns with checking in GN files? I've put some initial docs for the GN build at https://reviews.llvm.org/D53944 ; it describes what the GN build is and is not, what its advantages are (speed and easier configurability), and some points about the philosophy behind the LLVM GN build.

My personal experience with GN - last time I tested it - was that it was impossible to
build from source as the build procedure was anything but straight-forward and pretty
much architecture-locked in.

And I just tried it again on a POWER machine:

glaubitz@redpanda:/srv/glaubitz/gn$ python build/gen.py
Installing Debian root image from https://commondatastorage.googleapis.com/chrome-linux-sysroot/toolchain/1015a998c2adf188813cca60b558b0ea1a0b6ced/debian_sid_amd64_sysroot.tar.xz
Downloading https://commondatastorage.googleapis.com/chrome-linux-sysroot/toolchain/1015a998c2adf188813cca60b558b0ea1a0b6ced/debian_sid_amd64_sysroot.tar.xz
glaubitz@redpanda:/srv/glaubitz/gn$

Jepp, hasn't changed. If building your build system involves downloading a Debian
chroot with a hard-coded architecture, you're doing something wrong. And from my personal
experience with Google upstream, you can outright forget that they would accept patches
for any target that they are not using themselves.

I really wouldn't recommend using GN. It might be fast, but the whole design seems
to be tailored to Google's own needs.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
On 11/1/18 1:10 AM, John Paul Adrian Glaubitz via llvm-dev wrote:
> Jepp, hasn't changed. If building your build system involves downloading a Debian
> chroot with a hard-coded architecture, you're doing something wrong. And from my personal
> experience with Google upstream, you can outright forget that they would accept patches
> for any target that they are not using themselves.

Oh, and btw, any serious Linux distribution would outright reject to package
something that involves downloading another chroot to be able to build it,
build daemons are - by definition - offline.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
Hi all,

On Oct 31, 2018, at 11:18 AM, Nico Weber via llvm-dev <[hidden email]> wrote:

Hi,

first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.

At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].

I had been worried that it would be a lot of work to keep the build files up to date, but I've been using this for all my LLVM/clang/lld development the last 8 months, and it turned out to not be a big problem -- LLVM's build files don't change very often, and GN build files are a pleasure to work with in my opinion.

I gave the lightning talk just to talk about my personal workflow, but there was a lot of interest. We had a roundtable on the next day, and about 20 people said they'd be interested in using this for their development too. The main request was that the .gn files are checked in upstream, so that we can collaborate on keeping them working. Two to three orgs even said they'd be interested in moving their buildbots to GN.

As mentioned at the top, the intention here is not to replace cmake, only to offer an opt-in alternative for people who are interested in it. Keeping the GN build going would be the responsibility of people using it, not of the general LLVM community. If this fails to find use and bitrots, we can easily remove it again.

Are there any concerns with checking in GN files? I've put some initial docs for the GN build at https://reviews.llvm.org/D53944 ; it describes what the GN build is and is not, what its advantages are (speed and easier configurability), and some points about the philosophy behind the LLVM GN build.

I think it's very valuable that we have one shared build system. If you'd like to check in GN files, my strong preference would be to accompany that with a plan to phase cmake out. 

I really don't mean to be flippant here -- I'm in awe of the work you've already done to set up GN files, and I know transitioning away from cmake would be a massive amount of work. Hear me out :)

My perspective comes from having worked on lldb for a while. lldb has two build systems (xcodebuild & cmake). I suppose opinions differ on whether that works well. Speaking for myself, having two build systems has been a massive source of frustration. I regularly see commits which break one of the two systems because of course they do. No one wants to test their commit a second time against a build system they don't really use. Replicating complex bits of build system logic also is a chore -- I've CC'd Stefan who might be able to say more about that.

My concern is that introducing gn files into llvm will cause a bit of a fracture. If the policy is that cmake users don't have to worry about breaking the gn build, I think gn users would be less inclined to fix the cmake build. If most developers decide to switch to gn, that would leave cmake adopters with a higher (possibly unmanageable) maintenance burden. It's also confusing for new users, as they already have a lot of different ways of checking out and building.

I know your plan is to have the maintenance burden of gn files placed on gn users, and that you haven't experienced an unmanageable number of breaks over the past 8 months. In lldb (a lower volume project), I actually do think the constant build breaks are hard to manage. And I'm just not sure the temptation to only update gn files could be resisted, as "works in my build system" tends to shut down conversations. I'm worried the same thing would happen in llvm.

best,
vedant


If folks are generally ok with GN files living in-tree, I'll start sending patches for gradually adding gn files through the regular review process.

If having a BUILD.gn file in every directory being confusing is a concern, GN has the concept of a "secondary tree" so that all GN files could be below llvm/gn/tree/{llvm,clang,lld,...}.

Cheers,
Nico

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
On 11/1/18 1:22 AM, Vedant Kumar via llvm-dev wrote:
> I think it's very valuable that we have one shared build system. If you'd like to check in GN files, my strong preference would be to accompany that with a plan to phase cmake out. 

Then you can just remove all targets which are not x86 and arm64.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
On Wed, Oct 31, 2018 at 5:15 PM John Paul Adrian Glaubitz
<[hidden email]> wrote:
>
> On 11/1/18 1:10 AM, John Paul Adrian Glaubitz via llvm-dev wrote:
> > Jepp, hasn't changed. If building your build system involves downloading a Debian
> > chroot with a hard-coded architecture, you're doing something wrong. And from my personal
> > experience with Google upstream, you can outright forget that they would accept patches
> > for any target that they are not using themselves.

I'm one of GN maintainers, I'd be happy to review to review and
patches for other systems and architectures. We accepted AIX port a
few months ago:
https://gn.googlesource.com/gn/+/499868c6781dfb87dcefafd8f68dc578f3b70b44

> Oh, and btw, any serious Linux distribution would outright reject to package
> something that involves downloading another chroot to be able to build it,
> build daemons are - by definition - offline.

We're downloading the sysroot we use to build against to avoid
dependencies on anything on the bot itself. I agree with you
completely that this is something that build shouldn't do, and I plan
on moving that step to the bot script.

Until a few months ago, GN was part of Chromium and was relying on its
infrastructure, build and source. We've decided to split GN into a
separate project because it's being adopted by other projects and
users outside of Chromium. We're still dealing with fallout of that
separation and resulting cleanup. I hope that eventually building GN
will be as simple and easy as Ninja itself.

> --
>  .''`.  John Paul Adrian Glaubitz
> : :' :  Debian Developer - [hidden email]
> `. `'   Freie Universitaet Berlin - [hidden email]
>   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913
>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
On 11/1/18 1:29 AM, Petr Hosek wrote:
> I'm one of GN maintainers, I'd be happy to review to review and
> patches for other systems and architectures. We accepted AIX port a
> few months ago:
> https://gn.googlesource.com/gn/+/499868c6781dfb87dcefafd8f68dc578f3b70b44

Fair enough. I just had the experience with SKIA where my patches were
outright rejected with the answer "We don't care about big-endian".

This was also the first time I had contact with GN, I wanted to build
SKIA from source on a non-x86 target and eventually gave up after half
an hour or so.

>> Oh, and btw, any serious Linux distribution would outright reject to package
>> something that involves downloading another chroot to be able to build it,
>> build daemons are - by definition - offline.
>
> We're downloading the sysroot we use to build against to avoid
> dependencies on anything on the bot itself. I agree with you
> completely that this is something that build shouldn't do, and I plan
> on moving that step to the bot script.

Well, there is no need to tie this into the build system directly, that's
what Travis-CI or Jenkins are for. I'm just getting a bad feeling in my
stomach if the solution for building a tool from source is to download
a complete build environment instead of using tools on the target system
to achieve that.

> Until a few months ago, GN was part of Chromium and was relying on its
> infrastructure, build and source. We've decided to split GN into a
> separate project because it's being adopted by other projects and
> users outside of Chromium. We're still dealing with fallout of that
> separation and resulting cleanup. I hope that eventually building GN
> will be as simple and easy as Ninja itself.

Ok, then I hope that LLVM will be buildable with cmake until GN is
mature enough that I can build it from source on any architecture
on Linux without me jumping through loops.

Thanks,
Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev


On Wed, Oct 31, 2018 at 5:22 PM Vedant Kumar <[hidden email]> wrote:
Hi all,

On Oct 31, 2018, at 11:18 AM, Nico Weber via llvm-dev <[hidden email]> wrote:

Hi,

first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.

At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].

I had been worried that it would be a lot of work to keep the build files up to date, but I've been using this for all my LLVM/clang/lld development the last 8 months, and it turned out to not be a big problem -- LLVM's build files don't change very often, and GN build files are a pleasure to work with in my opinion.

I gave the lightning talk just to talk about my personal workflow, but there was a lot of interest. We had a roundtable on the next day, and about 20 people said they'd be interested in using this for their development too. The main request was that the .gn files are checked in upstream, so that we can collaborate on keeping them working. Two to three orgs even said they'd be interested in moving their buildbots to GN.

As mentioned at the top, the intention here is not to replace cmake, only to offer an opt-in alternative for people who are interested in it. Keeping the GN build going would be the responsibility of people using it, not of the general LLVM community. If this fails to find use and bitrots, we can easily remove it again.

Are there any concerns with checking in GN files? I've put some initial docs for the GN build at https://reviews.llvm.org/D53944 ; it describes what the GN build is and is not, what its advantages are (speed and easier configurability), and some points about the philosophy behind the LLVM GN build.

I think it's very valuable that we have one shared build system. If you'd like to check in GN files, my strong preference would be to accompany that with a plan to phase cmake out. 

I really don't mean to be flippant here -- I'm in awe of the work you've already done to set up GN files, and I know transitioning away from cmake would be a massive amount of work. Hear me out :)

My perspective comes from having worked on lldb for a while. lldb has two build systems (xcodebuild & cmake). I suppose opinions differ on whether that works well. Speaking for myself, having two build systems has been a massive source of frustration. I regularly see commits which break one of the two systems because of course they do. No one wants to test their commit a second time against a build system they don't really use. Replicating complex bits of build system logic also is a chore -- I've CC'd Stefan who might be able to say more about that.

My concern is that introducing gn files into llvm will cause a bit of a fracture. If the policy is that cmake users don't have to worry about breaking the gn build, I think gn users would be less inclined to fix the cmake build. If most developers decide to switch to gn, that would leave cmake adopters with a higher (possibly unmanageable) maintenance burden. It's also confusing for new users, as they already have a lot of different ways of checking out and building.

I know your plan is to have the maintenance burden of gn files placed on gn users, and that you haven't experienced an unmanageable number of breaks over the past 8 months. In lldb (a lower volume project), I actually do think the constant build breaks are hard to manage. And I'm just not sure the temptation to only update gn files could be resisted, as "works in my build system" tends to shut down conversations. I'm worried the same thing would happen in llvm.

best,
vedant

Personally I am also skeptical of the value of having 2 different build systems, for many of the same reasons you have elborated here.  One thing I actually find might be worth exploring is the quality of the IDE-generated projects by gn, especially the Xcode projects.  Having a build system (whether it be CMake or gn or something else is irrelevant) that can generate high quality Xcode IDE projects is -- to my understanding -- a pre-requisite for getting LLDB off of the hand-maintained Xcode project.

Would you be willing to try out gn's generated Xcode project (I'm told this is supported, but I haven't tried it since I don't have a Mac) either with LLVM or LLDB (if the current build files support LLDB) and do a usability comparison of the generated projects?

This is the one killer feature that would be a strong win over CMake that we currently have no clear path to ever making headway on.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
On Wed, Oct 31, 2018 at 5:53 PM Zachary Turner <[hidden email]> wrote:

>
>
>
> On Wed, Oct 31, 2018 at 5:22 PM Vedant Kumar <[hidden email]> wrote:
>>
>> Hi all,
>>
>> On Oct 31, 2018, at 11:18 AM, Nico Weber via llvm-dev <[hidden email]> wrote:
>>
>> Hi,
>>
>> first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.
>>
>> At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].
>>
>> I had been worried that it would be a lot of work to keep the build files up to date, but I've been using this for all my LLVM/clang/lld development the last 8 months, and it turned out to not be a big problem -- LLVM's build files don't change very often, and GN build files are a pleasure to work with in my opinion.
>>
>> I gave the lightning talk just to talk about my personal workflow, but there was a lot of interest. We had a roundtable on the next day, and about 20 people said they'd be interested in using this for their development too. The main request was that the .gn files are checked in upstream, so that we can collaborate on keeping them working. Two to three orgs even said they'd be interested in moving their buildbots to GN.
>>
>> As mentioned at the top, the intention here is not to replace cmake, only to offer an opt-in alternative for people who are interested in it. Keeping the GN build going would be the responsibility of people using it, not of the general LLVM community. If this fails to find use and bitrots, we can easily remove it again.
>>
>> Are there any concerns with checking in GN files? I've put some initial docs for the GN build at https://reviews.llvm.org/D53944 ; it describes what the GN build is and is not, what its advantages are (speed and easier configurability), and some points about the philosophy behind the LLVM GN build.
>>
>>
>> I think it's very valuable that we have one shared build system. If you'd like to check in GN files, my strong preference would be to accompany that with a plan to phase cmake out.
>>
>> I really don't mean to be flippant here -- I'm in awe of the work you've already done to set up GN files, and I know transitioning away from cmake would be a massive amount of work. Hear me out :)
>>
>> My perspective comes from having worked on lldb for a while. lldb has two build systems (xcodebuild & cmake). I suppose opinions differ on whether that works well. Speaking for myself, having two build systems has been a massive source of frustration. I regularly see commits which break one of the two systems because of course they do. No one wants to test their commit a second time against a build system they don't really use. Replicating complex bits of build system logic also is a chore -- I've CC'd Stefan who might be able to say more about that.
>>
>> My concern is that introducing gn files into llvm will cause a bit of a fracture. If the policy is that cmake users don't have to worry about breaking the gn build, I think gn users would be less inclined to fix the cmake build. If most developers decide to switch to gn, that would leave cmake adopters with a higher (possibly unmanageable) maintenance burden. It's also confusing for new users, as they already have a lot of different ways of checking out and building.
>>
>> I know your plan is to have the maintenance burden of gn files placed on gn users, and that you haven't experienced an unmanageable number of breaks over the past 8 months. In lldb (a lower volume project), I actually do think the constant build breaks are hard to manage. And I'm just not sure the temptation to only update gn files could be resisted, as "works in my build system" tends to shut down conversations. I'm worried the same thing would happen in llvm.
>>
>> best,
>> vedant
>
>
> Personally I am also skeptical of the value of having 2 different build systems, for many of the same reasons you have elborated here.  One thing I actually find might be worth exploring is the quality of the IDE-generated projects by gn, especially the Xcode projects.  Having a build system (whether it be CMake or gn or something else is irrelevant) that can generate high quality Xcode IDE projects is -- to my understanding -- a pre-requisite for getting LLDB off of the hand-maintained Xcode project.
>
> Would you be willing to try out gn's generated Xcode project (I'm told this is supported, but I haven't tried it since I don't have a Mac) either with LLVM or LLDB (if the current build files support LLDB) and do a usability comparison of the generated projects?
>
> This is the one killer feature that would be a strong win over CMake that we currently have no clear path to ever making headway on.

If you clone Nico's experiment from
https://github.com/nico/llvm-project-20170507/tree/gn, you can try the
following:

gn gen --ide=vs out/vs
gn gen --ide=xcode out/xcode

I'd really like to hear your feedback on the quality of the projects
that GN generates.
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
On 10/31/18 7:18 PM, Nico Weber via cfe-dev wrote:
> cmake is great for many use cases, but in my
> opinion local day-to-day development isn't one of them.

I'm curious - would you please outline the problems you've encountered?

Nikolai
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
On Thu, Nov 1, 2018 at 1:22 AM Vedant Kumar via cfe-dev
<[hidden email]> wrote:
>
> Hi all,
>
>> On Oct 31, 2018, at 11:18 AM, Nico Weber via llvm-dev <[hidden email]> wrote:
>>
>> Hi,
>>
>> first things first: If you're happy with cmake, you can stop reading now.
>>

I'm not, I just put up with it :)

> ...
> I think it's very valuable that we have one shared build system. If you'd like to check in GN files, my strong preference would be to accompany that with a plan to phase cmake out.
>
> I really don't mean to be flippant here -- I'm in awe of the work you've already done to set up GN files, and I know transitioning away from cmake would be a massive amount of work. Hear me out :)
>
> My perspective comes from having worked on lldb for a while. lldb has two build systems (xcodebuild & cmake). I suppose opinions differ on whether that works well. Speaking for myself, having two build systems has been a massive source of frustration. I regularly see commits which break one of the two systems because of course they do. No one wants to test their commit a second time against a build system they don't really use. Replicating complex bits of build system logic also is a chore -- I've CC'd Stefan who might be able to say more about that.
>
> My concern is that introducing gn files into llvm will cause a bit of a fracture. If the policy is that cmake users don't have to worry about breaking the gn build, I think gn users would be less inclined to fix the cmake build. If most developers decide to switch to gn, that would leave cmake adopters with a higher (possibly unmanageable) maintenance burden. It's also confusing for new users, as they already have a lot of different ways of checking out and building.
>
> I know your plan is to have the maintenance burden of gn files placed on gn users, and that you haven't experienced an unmanageable number of breaks over the past 8 months. In lldb (a lower volume project), I actually do think the constant build breaks are hard to manage. And I'm just not sure the temptation to only update gn files could be resisted, as "works in my build system" tends to shut down conversations. I'm worried the same thing would happen in llvm.
>
> best,
> vedant
>
>

Precisely. How would the proposed cmake + gn build system be different
from the (deprecated and then removed) autoconf + cmake ?

Csaba
--
You can get very substantial performance improvements
by not doing the right thing. - Scott Meyers, An Effective C++11/14 Sampler
So if you're looking for a completely portable, 100% standards-conformat way
to get the wrong information: this is what you want. - Scott Meyers (C++TDaWYK)
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
Hello everyone

Vedant, thanks for bringing this to my attention. From my work on LLDB, I agree with all your points.

I think it's very valuable that we have one shared build system.
Most of us don't want to spend their time on build system tasks, right? Supporting a second build system in-tree will inevitably increase our amount of attention on it. We should be pragmatic and not jump on new developments easily.

If you'd like to check in GN files, my strong preference would be to accompany that with a plan to phase cmake out.
So, would it be a good idea to phase out CMake? I rarely meet people who are happy with CMake, but I think the bar is still very high, because:

* The community developed a collective understanding of CMake with all it's strengths and pitfalls. With GN most of us had to learn a new language.
* There is a lot of valuable documentation and tutorials about LLVM and CMake. It will take time and effort for GN to catch up.
* CMake is the de-facto standard for private and cross-corporation C++ projects (and rising [1]).
* Kitware is not a stakeholder in LLVM and CMake not tailored to anyone's specific needs.

[1] Rank 6 fastest growing languages on GitHub: https://octoverse.github.com/projects

lldb has two build systems (xcodebuild & cmake). I suppose opinions differ on whether that works well. Speaking for myself, having two build systems has been a massive source of frustration. I regularly see commits which break one of the two systems because of course they do.
+1 and it's been the same for previous projects I worked on. Is there a good example where multiple build systems actually coexist well?

Replicating complex bits of build system logic also is a chore
Yes, it needs knowledge of the pitfalls for both of them + runnable setups for the various configurations.
Speaking for myself, it takes a lot of time and mood.


Am 31.10.18 um 23:10 schrieb Nicolai Hähnle via llvm-dev:
On 31.10.18 19:18, Nico Weber via llvm-dev wrote:
If having a BUILD.gn file in every directory being confusing is a concern, GN has the concept of a "secondary tree" so that all GN files could be below llvm/gn/tree/{llvm,clang,lld,...}.
So maintain it in a separate repository.
This seems very reasonable as long as GN is not adopted as the one an only.
Also it's the best way to make sure it's fully maintained by GN users.


Am 01.11.18 um 00:28 schrieb Zachary Turner via llvm-dev:
One thing to think about is that generated IDE projects from CMake are less than ideal.  In fact I actually made a post about this a few weeks ago.
Note that IDE vendors are working on direct CMake support as well, e.g:
* VS2017 & Qt Creator: https://blog.kitware.com/cmake-e-server-improves-visual-studio-and-qt-creator-ides/
* CLion: https://www.jetbrains.com/help/clion/project-models.html

Best
Stefan


Am 01.11.18 um 01:22 schrieb Vedant Kumar via cfe-dev:
Hi all,

On Oct 31, 2018, at 11:18 AM, Nico Weber via llvm-dev <[hidden email]> wrote:

Hi,

first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.

At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].

I had been worried that it would be a lot of work to keep the build files up to date, but I've been using this for all my LLVM/clang/lld development the last 8 months, and it turned out to not be a big problem -- LLVM's build files don't change very often, and GN build files are a pleasure to work with in my opinion.

I gave the lightning talk just to talk about my personal workflow, but there was a lot of interest. We had a roundtable on the next day, and about 20 people said they'd be interested in using this for their development too. The main request was that the .gn files are checked in upstream, so that we can collaborate on keeping them working. Two to three orgs even said they'd be interested in moving their buildbots to GN.

As mentioned at the top, the intention here is not to replace cmake, only to offer an opt-in alternative for people who are interested in it. Keeping the GN build going would be the responsibility of people using it, not of the general LLVM community. If this fails to find use and bitrots, we can easily remove it again.

Are there any concerns with checking in GN files? I've put some initial docs for the GN build at https://reviews.llvm.org/D53944 ; it describes what the GN build is and is not, what its advantages are (speed and easier configurability), and some points about the philosophy behind the LLVM GN build.

I think it's very valuable that we have one shared build system. If you'd like to check in GN files, my strong preference would be to accompany that with a plan to phase cmake out. 

I really don't mean to be flippant here -- I'm in awe of the work you've already done to set up GN files, and I know transitioning away from cmake would be a massive amount of work. Hear me out :)

My perspective comes from having worked on lldb for a while. lldb has two build systems (xcodebuild & cmake). I suppose opinions differ on whether that works well. Speaking for myself, having two build systems has been a massive source of frustration. I regularly see commits which break one of the two systems because of course they do. No one wants to test their commit a second time against a build system they don't really use. Replicating complex bits of build system logic also is a chore -- I've CC'd Stefan who might be able to say more about that.

My concern is that introducing gn files into llvm will cause a bit of a fracture. If the policy is that cmake users don't have to worry about breaking the gn build, I think gn users would be less inclined to fix the cmake build. If most developers decide to switch to gn, that would leave cmake adopters with a higher (possibly unmanageable) maintenance burden. It's also confusing for new users, as they already have a lot of different ways of checking out and building.

I know your plan is to have the maintenance burden of gn files placed on gn users, and that you haven't experienced an unmanageable number of breaks over the past 8 months. In lldb (a lower volume project), I actually do think the constant build breaks are hard to manage. And I'm just not sure the temptation to only update gn files could be resisted, as "works in my build system" tends to shut down conversations. I'm worried the same thing would happen in llvm.

best,
vedant


If folks are generally ok with GN files living in-tree, I'll start sending patches for gradually adding gn files through the regular review process.

If having a BUILD.gn file in every directory being confusing is a concern, GN has the concept of a "secondary tree" so that all GN files could be below llvm/gn/tree/{llvm,clang,lld,...}.

Cheers,
Nico

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
Lots of feedback! Sounds like several people are interested in this, which is cool.

Trying to address a few points here:

 > f these files exist in the repo, I could imagine that reviewers will 
> reject patches that might/likely break the gn files.

I will make the language in https://reviews.llvm.org/D53944 stronger. Reviewers should not ask for this. If it helps, we could put the gn build files into a directory called "experimental" or similar.

> Do you think there will be some point in time that these files will be 
> out of 'experimental' status?

Not anytime soon.

> Some folks who don't like GN or don't see the point.

That's fine, see first sentence in my mail :-)

> my strong preference would be to accompany that with a plan to phase cmake out.

I very strongly don't think this should happen. cmake does many things right, for example running on all the platforms under the sun and supporting very long-tail issues. By saying "cmake stays the main build system" there's a long list of things we don't need to worry about. If in a few years GN can build on all the platforms under the sun, everything in LLVM works in the GN build, and everyone happens to use it, we could discuss this at that point, but in my opinion this is extremely unlikely to happen. If this is a requirement, I guarantee this thread will fizzle out. Also, I'm very much not interested in pushing for moving llvm off cmake personally.

> lldb xcode and cmake painful

I think this is in part because it's tricky to update the xcode build files on non-Apple hardware, and because it's agreed upon which of the two is the canonical one. Here, we very explicitly say that cmake is the official build system that's covered by bots, and your patch has to work there, else it gets reverted. And keeping the gn build up to date is solely the responsibility of people using it.

This also gives a good metric on if anyone uses the gn build: If they're bitrotted for long stretches, we know it's time to delete them again.

> I'm curious - would you please outline the problems you've encountered?

Sure. I did that a bit in https://reviews.llvm.org/D53944 (the "Intoduction" section) and in my talk. The issues are:
1. cmake is slow, so it caches things
2. but cache eviction is hard, so it's very hard to change an existing build dir e.g. from debug to release – for many config values you need to create a completely new build dir to be sure the new values are actually used
3. gn has support for building the same target with different toolchains in a single build dir, making it e.g. possible to model bootstrap builds or compiler-rt compiles with the built clang much nicer (I haven't implemented that part yet)

I know I read the following two somewhere but can't find it, so this is an approximate cite:

> the gn bots should be private, not public

I agree that no bots on the official LLVM waterfall (http://lab.llvm.org:8011/console) should use GN. They would be off-waterfall bots that don't email you and that you wouldn't know about if you didn't go looking for them. They wouldn't even run on LLVM's buildbot instance.

> so keep them in a separate repo

Right now, my approach adds the BUILD.gn files in the directory they belong (clang/lib/Support/BUILD.gn, etc). That's not easily possible with a separate repo. Using the secondary tree, this is easier -- but it's still harder than having the files just there, and I don't see which problems are solved by keeping them out of tree.

>  Most of us don't want to spend their time on build system tasks, right?

I do like doing them in gn. I find updating gn build files after syncing very relaxing :-) (And I have a script that helps me, so it really isn't much work -- see gn/sync_source_lists_from_cmake.py in my repo) For example, I'm pretty happy with how the setup for how to select which archs to build turned out in the LLVM GN build.

> Is there a good example where multiple build systems actually coexist well?

webkit has a bunch, but I wouldn't say that works super well. I'm not aware of any project that has tried experimental, unsupported build systems for enthusiasts in addition to a single main supported build system.

--

To reiterate, this is the setup I use for my development. I feel it makes me more happy and productive. Others are interested in it too. I think there's value to be able to experiment in-tree with this since it makes collaboration on the gn files much easier and has zero cost for people not using it. I am not interested in long mailing list discussions about build systems, so I'm very much not interested in proposing to remove cmake. Also, I think requiring that build system experimentation has to have a plan for converging on a single build system again eventually is a basically impossibly high bar.

On Thu, Nov 1, 2018 at 4:44 AM Csaba Raduly <[hidden email]> wrote:
On Thu, Nov 1, 2018 at 1:22 AM Vedant Kumar via cfe-dev
<[hidden email]> wrote:
>
> Hi all,
>
>> On Oct 31, 2018, at 11:18 AM, Nico Weber via llvm-dev <[hidden email]> wrote:
>>
>> Hi,
>>
>> first things first: If you're happy with cmake, you can stop reading now.
>>

I'm not, I just put up with it :)

> ...
> I think it's very valuable that we have one shared build system. If you'd like to check in GN files, my strong preference would be to accompany that with a plan to phase cmake out.
>
> I really don't mean to be flippant here -- I'm in awe of the work you've already done to set up GN files, and I know transitioning away from cmake would be a massive amount of work. Hear me out :)
>
> My perspective comes from having worked on lldb for a while. lldb has two build systems (xcodebuild & cmake). I suppose opinions differ on whether that works well. Speaking for myself, having two build systems has been a massive source of frustration. I regularly see commits which break one of the two systems because of course they do. No one wants to test their commit a second time against a build system they don't really use. Replicating complex bits of build system logic also is a chore -- I've CC'd Stefan who might be able to say more about that.
>
> My concern is that introducing gn files into llvm will cause a bit of a fracture. If the policy is that cmake users don't have to worry about breaking the gn build, I think gn users would be less inclined to fix the cmake build. If most developers decide to switch to gn, that would leave cmake adopters with a higher (possibly unmanageable) maintenance burden. It's also confusing for new users, as they already have a lot of different ways of checking out and building.
>
> I know your plan is to have the maintenance burden of gn files placed on gn users, and that you haven't experienced an unmanageable number of breaks over the past 8 months. In lldb (a lower volume project), I actually do think the constant build breaks are hard to manage. And I'm just not sure the temptation to only update gn files could be resisted, as "works in my build system" tends to shut down conversations. I'm worried the same thing would happen in llvm.
>
> best,
> vedant
>
>

Precisely. How would the proposed cmake + gn build system be different
from the (deprecated and then removed) autoconf + cmake ?

Csaba
--
You can get very substantial performance improvements
by not doing the right thing. - Scott Meyers, An Effective C++11/14 Sampler
So if you're looking for a completely portable, 100% standards-conformat way
to get the wrong information: this is what you want. - Scott Meyers (C++TDaWYK)

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
On Thu, 1 Nov 2018 at 14:14, Nico Weber via cfe-dev
<[hidden email]> wrote:

>
> Lots of feedback! Sounds like several people are interested in this, which is cool.
>
> Trying to address a few points here:
>
>  > f these files exist in the repo, I could imagine that reviewers will
> > reject patches that might/likely break the gn files.
>
> I will make the language in https://reviews.llvm.org/D53944 stronger. Reviewers should not ask for this. If it helps, we could put the gn build files into a directory called "experimental" or similar.
>
> > Do you think there will be some point in time that these files will be
> > out of 'experimental' status?
>
> Not anytime soon.
>
> > Some folks who don't like GN or don't see the point.
>
> That's fine, see first sentence in my mail :-)
>
> > my strong preference would be to accompany that with a plan to phase cmake out.
>
> I very strongly don't think this should happen. cmake does many things right, for example running on all the platforms under the sun and supporting very long-tail issues. By saying "cmake stays the main build system" there's a long list of things we don't need to worry about. If in a few years GN can build on all the platforms under the sun, everything in LLVM works in the GN build, and everyone happens to use it, we could discuss this at that point, but in my opinion this is extremely unlikely to happen. If this is a requirement, I guarantee this thread will fizzle out. Also, I'm very much not interested in pushing for moving llvm off cmake personally.

I think presence in distro repositories is also a barrier. Most
distributions have appropriate cmake and ninja packages, but not gn.
Obviously pre-built binaries help, but it's an extra barrier.

> > the gn bots should be private, not public
>
> I agree that no bots on the official LLVM waterfall (http://lab.llvm.org:8011/console) should use GN. They would be off-waterfall bots that don't email you and that you wouldn't know about if you didn't go looking for them. They wouldn't even run on LLVM's buildbot instance.

There's a "staging" buildbot instance for exactly this sort of thing
http://lab.llvm.org:8014/ - no emails will be generated. Ideally it
would generate emails for the bot owner, but unfortunately neither I
or Dylan McKay have managed to do that.

I'm sorry I missed your lightning talk. I don't have a strong opinion
one way or another on whether gn build files belong in or out-of-tree,
but I appreciate being introduced to it - it looks like it has a lot
to offer.

Best,

Alex
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
Any easy way to do this as some kind of overlay, so they GN files wouldn't live in the LLVM repository, but in a separate one?

(this might avoid some of the community discussion - though would perhaps still likely have the issue I see as most significant: With a sufficient number of developers using GN, the rate of build breaks due to those developers missing CMake file updates might rise to be a bit of a drag on the LLVM project - though I don't think that's likely to ever be a huge deal, just an annoyance)

On Wed, Oct 31, 2018 at 11:19 AM Nico Weber via cfe-dev <[hidden email]> wrote:
Hi,

first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.

At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].

I had been worried that it would be a lot of work to keep the build files up to date, but I've been using this for all my LLVM/clang/lld development the last 8 months, and it turned out to not be a big problem -- LLVM's build files don't change very often, and GN build files are a pleasure to work with in my opinion.

I gave the lightning talk just to talk about my personal workflow, but there was a lot of interest. We had a roundtable on the next day, and about 20 people said they'd be interested in using this for their development too. The main request was that the .gn files are checked in upstream, so that we can collaborate on keeping them working. Two to three orgs even said they'd be interested in moving their buildbots to GN.

As mentioned at the top, the intention here is not to replace cmake, only to offer an opt-in alternative for people who are interested in it. Keeping the GN build going would be the responsibility of people using it, not of the general LLVM community. If this fails to find use and bitrots, we can easily remove it again.

Are there any concerns with checking in GN files? I've put some initial docs for the GN build at https://reviews.llvm.org/D53944 ; it describes what the GN build is and is not, what its advantages are (speed and easier configurability), and some points about the philosophy behind the LLVM GN build.

If folks are generally ok with GN files living in-tree, I'll start sending patches for gradually adding gn files through the regular review process.

If having a BUILD.gn file in every directory being confusing is a concern, GN has the concept of a "secondary tree" so that all GN files could be below llvm/gn/tree/{llvm,clang,lld,...}.

Cheers,
Nico

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] GN build roundtable summary; adding GN build files to the repo

David Blaikie via cfe-dev
I think this is how, after moving to a monorepo, it may be feasible to
get this in a separate fork.

Granted the Git Monorepo + GitHub happens, we can even make it so that
the GN build is a branch on the official git repository, which can
track the mainline development.
On Fri, Nov 2, 2018 at 3:49 AM David Blaikie via llvm-dev
<[hidden email]> wrote:

>
> Any easy way to do this as some kind of overlay, so they GN files wouldn't live in the LLVM repository, but in a separate one?
>
> (this might avoid some of the community discussion - though would perhaps still likely have the issue I see as most significant: With a sufficient number of developers using GN, the rate of build breaks due to those developers missing CMake file updates might rise to be a bit of a drag on the LLVM project - though I don't think that's likely to ever be a huge deal, just an annoyance)
>
> On Wed, Oct 31, 2018 at 11:19 AM Nico Weber via cfe-dev <[hidden email]> wrote:
>>
>> Hi,
>>
>> first things first: If you're happy with cmake, you can stop reading now. Nobody is proposing that LLVM moves off cmake, and nobody is proposing anything that's causing people using cmake more work.
>>
>> At the LLVM conference, I gave a lightning talk [1] about using GN [2] to build LLVM and clang. cmake is great for many use cases, but in my opinion local day-to-day development isn't one of them. So I wrote GN build files for LLVM and clang, enough to make `ninja check-llvm check-clang check-lld` build everything needed for these three test suites and that all tests pass. This works on Linux, Mac, Win hosts targeting X86, ARM, AArch64. You can see them at [3].
>>
>> I had been worried that it would be a lot of work to keep the build files up to date, but I've been using this for all my LLVM/clang/lld development the last 8 months, and it turned out to not be a big problem -- LLVM's build files don't change very often, and GN build files are a pleasure to work with in my opinion.
>>
>> I gave the lightning talk just to talk about my personal workflow, but there was a lot of interest. We had a roundtable on the next day, and about 20 people said they'd be interested in using this for their development too. The main request was that the .gn files are checked in upstream, so that we can collaborate on keeping them working. Two to three orgs even said they'd be interested in moving their buildbots to GN.
>>
>> As mentioned at the top, the intention here is not to replace cmake, only to offer an opt-in alternative for people who are interested in it. Keeping the GN build going would be the responsibility of people using it, not of the general LLVM community. If this fails to find use and bitrots, we can easily remove it again.
>>
>> Are there any concerns with checking in GN files? I've put some initial docs for the GN build at https://reviews.llvm.org/D53944 ; it describes what the GN build is and is not, what its advantages are (speed and easier configurability), and some points about the philosophy behind the LLVM GN build.
>>
>> If folks are generally ok with GN files living in-tree, I'll start sending patches for gradually adding gn files through the regular review process.
>>
>> If having a BUILD.gn file in every directory being confusing is a concern, GN has the concept of a "secondary tree" so that all GN files could be below llvm/gn/tree/{llvm,clang,lld,...}.
>>
>> Cheers,
>> Nico
>>
>> 1: https://llvm.org/devmtg/2018-10/talk-abstracts.html#lt2
>> 2: https://gn.googlesource.com/gn , https://is.gd/gn_intro
>> 3: https://github.com/llvm-project/llvm-project-20170507/compare/master...nico:gn , click "Files Changed" to see the GN files.
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



--
Dean
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
12