Too much time to compile clang. Suggestions for a starter?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Too much time to compile clang. Suggestions for a starter?

Eric Fiselier via cfe-dev

I would like to start "touching" a bit the code, trying to create some functionalities, but I have no idea where to start, and I will need many trials/tests...

So, one thing I was expecting is fast times to try (Modify-Compile-Run).

I did:

make clang
and it needed like 3-4 hours. I am ok with that.

Then, I add some "stupid" statement like 'int a = 3;' somewhere in llvm/tools/clang/lib/Frontend/Rewrite/FixItRewriter.cpp (FixitRewriter::Diag), and it needed:
- 6.57s. to compile the target ('make clangRewriteFrontend').
- 4min 36.36s to link and build clang (`make clang`).
In both cases it spends almost all the time linking (first to a static lib, then to the clang binary itself).

This is really a lot of time to try bit a bit without knowledge of the whole clang, compiling process, etc. Waiting up to is just really time consuming.

What would you do to try the modifications faster?

Any recomendation/blog/guide to follow as a starter trying to modify a bit the codebase of clang?


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Too much time to compile clang. Suggestions for a starter?

Eric Fiselier via cfe-dev
You can use gold <https://en.wikipedia.org/wiki/Gold_(linker)> to link.
To do so, pass -DLLVM_USE_LINKER=gold at CMake configure time.

This is also a good opportunity to point at a minor issue with CMake
configuration, where even after passing the above switch CMake will
still print

-- Linker detection: GNU ld

instead of

-- Linker detection: GNU Gold

I don't know whether it "just works" for others, but
<https://reviews.llvm.org/D38665> (unreviewed) fixes it for me.

On 21.10.2017 11:11, rNoz via cfe-dev wrote:

> I would like to start "touching" a bit the code, trying to create some
> functionalities, but I have no idea where to start, and I will need many
> trials/tests...
>
> So, one thing I was expecting is fast times to try (Modify-Compile-Run).
>
> I did:
>
> make clang
> and it needed like 3-4 hours. I am ok with that.
>
> Then, I add some "stupid" statement like 'int a = 3;' somewhere in
> llvm/tools/clang/lib/Frontend/Rewrite/FixItRewriter.cpp
> (FixitRewriter::Diag), and it needed:
> - *6.57s. to compile the target ('make clangRewriteFrontend').*
> - *4min 36.36s to link and build clang (`make clang`).*
> In both cases it spends almost all the time* linking* (first to a static
> lib, then to the clang binary itself).
>
> This is really a lot of time to try bit a bit without knowledge of the
> whole clang, compiling process, etc. Waiting up to is just really time
> consuming.
>
> What would you do to try the modifications faster?
>
> Any recomendation/blog/guide to follow as a starter trying to modify a
> bit the codebase of clang?
>
>
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
--
Regards,
Oleksii Vilchanskyi
PGP:0x8D3A0E046BDE941F2A53867CE3FD952D48C0B338


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Too much time to compile clang. Suggestions for a starter?

Eric Fiselier via cfe-dev
Did you try to run make in parallel using the -j option? That and linking with Gold (as Oleksii mentioned) should greatly improve build times.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Too much time to compile clang. Suggestions for a starter?

Eric Fiselier via cfe-dev
On Sat, Oct 21, 2017 at 02:30:28PM +0200, Nicolas Lesser via cfe-dev wrote:
> Did you try to run make in parallel using the -j option? That and linking
> with Gold (as Oleksii mentioned) should greatly improve build times.

Especially for incremental builds, ninja tends to be a much better
option.

Joerg
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Too much time to compile clang. Suggestions for a starter?

Eric Fiselier via cfe-dev
In reply to this post by Eric Fiselier via cfe-dev
A few links that Google found in a simple search:
  - this thread:
http://lists.llvm.org/pipermail/llvm-dev/2016-October/106186.html
  - https://llvm.org/devmtg/2015-04/slides/eurollvm-2015-build.pdf and
these two blog posts:
https://blogs.s-osg.org/an-introduction-to-accelerating-your-build-with-clang/ 
and
https://blogs.s-osg.org/a-conclusion-to-accelerating-your-build-with-clang/

I personally took most of the advices from the last blog posts:
  - Ninja, use Clang to build
  - for Debug builds: -DBUILD_SHARED_LIBS=ON -DLLVM_USE_SPLIT_DWARF=ON
-DLLVM_OPTIMIZED_TABLEGEN=ON

Cheers,
Jonas

Am 2017-10-21 05:11, schrieb rNoz via cfe-dev:

> I would like to start "touching" a bit the code, trying to create some
> functionalities, but I have no idea where to start, and I will need
> many trials/tests...
>
> So, one thing I was expecting is fast times to try
> (Modify-Compile-Run).
>
> I did:
>
> make clang
> and it needed like 3-4 hours. I am ok with that.
>
> Then, I add some "stupid" statement like 'int a = 3;' somewhere in
> llvm/tools/clang/lib/Frontend/Rewrite/FixItRewriter.cpp
> (FixitRewriter::Diag), and it needed:
> - 6.57S. TO COMPILE THE TARGET ('MAKE CLANGREWRITEFRONTEND').
> - 4MIN 36.36S TO LINK AND BUILD CLANG (`MAKE CLANG`).
> In both cases it spends almost all the time LINKING (first to a static
> lib, then to the clang binary itself).
>
> This is really a lot of time to try bit a bit without knowledge of the
> whole clang, compiling process, etc. Waiting up to is just really time
> consuming.
>
> What would you do to try the modifications faster?
>
> Any recomendation/blog/guide to follow as a starter trying to modify a
> bit the codebase of clang?
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Too much time to compile clang. Suggestions for a starter?

Eric Fiselier via cfe-dev
Thank you very much, Jonas, Nicolas and Oleksii.

I read those blogs and applied just this:

cp ../../llvm-support/AddLLVM.cmake.gold ../cmake/modules/AddLLVM.cmake
(I modified those lines in the reviews.llvm.org).
CXX=clang++ CC=clang cmake LD=ld.gold -G Ninja .. -DLLVM_USE_LINKER=gold
-DBUILD_SHARED_LIBS=ON -DLLVM_USE_SPLIT_DWARF=ON
-DLLVM_OPTIMIZED_TABLEGEN=ON .. && ninja clang

And with just the same modification (int a = 22; in that class) I get
compiled/linked in JUST 10-3 SECONDS.

Impressive!

Thanks again. This thread can be closed.


On 21/10/17 15:27, Jonas Hahnfeld wrote:

> A few links that Google found in a simple search:
>  - this thread:
> http://lists.llvm.org/pipermail/llvm-dev/2016-October/106186.html
>  - https://llvm.org/devmtg/2015-04/slides/eurollvm-2015-build.pdf and
> these two blog posts:
> https://blogs.s-osg.org/an-introduction-to-accelerating-your-build-with-clang/
> and
> https://blogs.s-osg.org/a-conclusion-to-accelerating-your-build-with-clang/
>
> I personally took most of the advices from the last blog posts:
>  - Ninja, use Clang to build
>  - for Debug builds: -DBUILD_SHARED_LIBS=ON -DLLVM_USE_SPLIT_DWARF=ON
> -DLLVM_OPTIMIZED_TABLEGEN=ON
>
> Cheers,
> Jonas
>
> Am 2017-10-21 05:11, schrieb rNoz via cfe-dev:
>> I would like to start "touching" a bit the code, trying to create some
>> functionalities, but I have no idea where to start, and I will need
>> many trials/tests...
>>
>> So, one thing I was expecting is fast times to try
>> (Modify-Compile-Run).
>>
>> I did:
>>
>> make clang
>> and it needed like 3-4 hours. I am ok with that.
>>
>> Then, I add some "stupid" statement like 'int a = 3;' somewhere in
>> llvm/tools/clang/lib/Frontend/Rewrite/FixItRewriter.cpp
>> (FixitRewriter::Diag), and it needed:
>> - 6.57S. TO COMPILE THE TARGET ('MAKE CLANGREWRITEFRONTEND').
>> - 4MIN 36.36S TO LINK AND BUILD CLANG (`MAKE CLANG`).
>> In both cases it spends almost all the time LINKING (first to a static
>> lib, then to the clang binary itself).
>>
>> This is really a lot of time to try bit a bit without knowledge of the
>> whole clang, compiling process, etc. Waiting up to is just really time
>> consuming.
>>
>> What would you do to try the modifications faster?
>>
>> Any recomendation/blog/guide to follow as a starter trying to modify a
>> bit the codebase of clang?
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Too much time to compile clang. Suggestions for a starter?

Eric Fiselier via cfe-dev
You might also consider building only those backends you are actually using.
E. g.:
cmake -DLLVM_TARGETS_TO_BUILD=X86 ...

More info:
https://llvm.org/docs/CMake.html#llvm-specific-variables

> On Oct 21, 2017, at 4:10 PM, rNoz via cfe-dev <[hidden email]> wrote:
>
> Thank you very much, Jonas, Nicolas and Oleksii.
>
> I read those blogs and applied just this:
>
> cp ../../llvm-support/AddLLVM.cmake.gold ../cmake/modules/AddLLVM.cmake
> (I modified those lines in the reviews.llvm.org).
> CXX=clang++ CC=clang cmake LD=ld.gold -G Ninja .. -DLLVM_USE_LINKER=gold
> -DBUILD_SHARED_LIBS=ON -DLLVM_USE_SPLIT_DWARF=ON
> -DLLVM_OPTIMIZED_TABLEGEN=ON .. && ninja clang
>
> And with just the same modification (int a = 22; in that class) I get
> compiled/linked in JUST 10-3 SECONDS.
>
> Impressive!
>
> Thanks again. This thread can be closed.
>
>
> On 21/10/17 15:27, Jonas Hahnfeld wrote:
>> A few links that Google found in a simple search:
>>  - this thread:
>> http://lists.llvm.org/pipermail/llvm-dev/2016-October/106186.html
>>  - https://llvm.org/devmtg/2015-04/slides/eurollvm-2015-build.pdf and
>> these two blog posts:
>> https://blogs.s-osg.org/an-introduction-to-accelerating-your-build-with-clang/
>> and
>> https://blogs.s-osg.org/a-conclusion-to-accelerating-your-build-with-clang/
>>
>> I personally took most of the advices from the last blog posts:
>>  - Ninja, use Clang to build
>>  - for Debug builds: -DBUILD_SHARED_LIBS=ON -DLLVM_USE_SPLIT_DWARF=ON
>> -DLLVM_OPTIMIZED_TABLEGEN=ON
>>
>> Cheers,
>> Jonas
>>
>> Am 2017-10-21 05:11, schrieb rNoz via cfe-dev:
>>> I would like to start "touching" a bit the code, trying to create some
>>> functionalities, but I have no idea where to start, and I will need
>>> many trials/tests...
>>>
>>> So, one thing I was expecting is fast times to try
>>> (Modify-Compile-Run).
>>>
>>> I did:
>>>
>>> make clang
>>> and it needed like 3-4 hours. I am ok with that.
>>>
>>> Then, I add some "stupid" statement like 'int a = 3;' somewhere in
>>> llvm/tools/clang/lib/Frontend/Rewrite/FixItRewriter.cpp
>>> (FixitRewriter::Diag), and it needed:
>>> - 6.57S. TO COMPILE THE TARGET ('MAKE CLANGREWRITEFRONTEND').
>>> - 4MIN 36.36S TO LINK AND BUILD CLANG (`MAKE CLANG`).
>>> In both cases it spends almost all the time LINKING (first to a static
>>> lib, then to the clang binary itself).
>>>
>>> This is really a lot of time to try bit a bit without knowledge of the
>>> whole clang, compiling process, etc. Waiting up to is just really time
>>> consuming.
>>>
>>> What would you do to try the modifications faster?
>>>
>>> Any recomendation/blog/guide to follow as a starter trying to modify a
>>> bit the codebase of clang?
>>> _______________________________________________
>>> cfe-dev mailing list
>>> [hidden email]
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev