Unusually high ram usage on tests

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Unusually high ram usage on tests

Dimitry Andric via cfe-dev
Hi everyone,
I am running 'ninja check-clang-analysis' on the release build of clang with LLVM_PARALLEL_LINK_JOBS set to 2 and assertions off. I am on Ubuntu 20.04. This is taking up almost 10 gb of memory (6 from ram, 4 from swap). I have been running this command for over a month and I haven't seen this sort of memory usage. Is this normal? Or am I missing something?
Warm regards,
Deep

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Unusually high ram usage on tests

Dimitry Andric via cfe-dev
Use `lld` as a linker, it is superior to `ld` or `gold`.
It consumes less memory and much faster.
Also, prefer dynamic linking for faster development cycles.

Balazs.





From: cfe-dev <[hidden email]> on behalf of Deep Majumder via cfe-dev <[hidden email]>
Sent: Wednesday, March 10, 2021 8:17:38 PM
To: cfe-dev <[hidden email]>
Subject: [cfe-dev] Unusually high ram usage on tests
 
Hi everyone,
I am running 'ninja check-clang-analysis' on the release build of clang with LLVM_PARALLEL_LINK_JOBS set to 2 and assertions off. I am on Ubuntu 20.04. This is taking up almost 10 gb of memory (6 from ram, 4 from swap). I have been running this command for over a month and I haven't seen this sort of memory usage. Is this normal? Or am I missing something?
Warm regards,
Deep

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Unusually high ram usage on tests

Dimitry Andric via cfe-dev
Yup, right you are! I had reinstalled my system and forgotten to install lld. Thanks!

On Thu, Mar 11, 2021 at 12:59 AM Benics Balázs <[hidden email]> wrote:
Use `lld` as a linker, it is superior to `ld` or `gold`.
It consumes less memory and much faster.
Also, prefer dynamic linking for faster development cycles.

Balazs.





From: cfe-dev <[hidden email]> on behalf of Deep Majumder via cfe-dev <[hidden email]>
Sent: Wednesday, March 10, 2021 8:17:38 PM
To: cfe-dev <[hidden email]>
Subject: [cfe-dev] Unusually high ram usage on tests
 
Hi everyone,
I am running 'ninja check-clang-analysis' on the release build of clang with LLVM_PARALLEL_LINK_JOBS set to 2 and assertions off. I am on Ubuntu 20.04. This is taking up almost 10 gb of memory (6 from ram, 4 from swap). I have been running this command for over a month and I haven't seen this sort of memory usage. Is this normal? Or am I missing something?
Warm regards,
Deep

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Unusually high ram usage on tests

Dimitry Andric via cfe-dev
Btw, how does one enable dynamic linking? I couldn't find such an option in CMakeCache.txt.

On Thu, Mar 11, 2021 at 11:09 AM Deep Majumder <[hidden email]> wrote:
Yup, right you are! I had reinstalled my system and forgotten to install lld. Thanks!

On Thu, Mar 11, 2021 at 12:59 AM Benics Balázs <[hidden email]> wrote:
Use `lld` as a linker, it is superior to `ld` or `gold`.
It consumes less memory and much faster.
Also, prefer dynamic linking for faster development cycles.

Balazs.





From: cfe-dev <[hidden email]> on behalf of Deep Majumder via cfe-dev <[hidden email]>
Sent: Wednesday, March 10, 2021 8:17:38 PM
To: cfe-dev <[hidden email]>
Subject: [cfe-dev] Unusually high ram usage on tests
 
Hi everyone,
I am running 'ninja check-clang-analysis' on the release build of clang with LLVM_PARALLEL_LINK_JOBS set to 2 and assertions off. I am on Ubuntu 20.04. This is taking up almost 10 gb of memory (6 from ram, 4 from swap). I have been running this command for over a month and I haven't seen this sort of memory usage. Is this normal? Or am I missing something?
Warm regards,
Deep

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Unusually high ram usage on tests

Dimitry Andric via cfe-dev
sometimes referred to as shared library build in the cmake config, I think

Speeds up building, but slows down test execution, unfortunately - which balance works best for you may vary/be different from what works best for other folks.


On Wed, Mar 10, 2021 at 9:41 PM Deep Majumder via cfe-dev <[hidden email]> wrote:
Btw, how does one enable dynamic linking? I couldn't find such an option in CMakeCache.txt.

On Thu, Mar 11, 2021 at 11:09 AM Deep Majumder <[hidden email]> wrote:
Yup, right you are! I had reinstalled my system and forgotten to install lld. Thanks!

On Thu, Mar 11, 2021 at 12:59 AM Benics Balázs <[hidden email]> wrote:
Use `lld` as a linker, it is superior to `ld` or `gold`.
It consumes less memory and much faster.
Also, prefer dynamic linking for faster development cycles.

Balazs.





From: cfe-dev <[hidden email]> on behalf of Deep Majumder via cfe-dev <[hidden email]>
Sent: Wednesday, March 10, 2021 8:17:38 PM
To: cfe-dev <[hidden email]>
Subject: [cfe-dev] Unusually high ram usage on tests
 
Hi everyone,
I am running 'ninja check-clang-analysis' on the release build of clang with LLVM_PARALLEL_LINK_JOBS set to 2 and assertions off. I am on Ubuntu 20.04. This is taking up almost 10 gb of memory (6 from ram, 4 from swap). I have been running this command for over a month and I haven't seen this sort of memory usage. Is this normal? Or am I missing something?
Warm regards,
Deep
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Unusually high ram usage on tests

Dimitry Andric via cfe-dev
On Wed, Mar 10, 2021 at 10:09 PM David Blaikie via cfe-dev <[hidden email]> wrote:
sometimes referred to as shared library build in the cmake config, I think

Speeds up building, but slows down test execution, unfortunately - which balance works best for you may vary/be different from what works best for other folks.


On Wed, Mar 10, 2021 at 9:41 PM Deep Majumder via cfe-dev <[hidden email]> wrote:
Btw, how does one enable dynamic linking? I couldn't find such an option in CMakeCache.txt.

On Thu, Mar 11, 2021 at 11:09 AM Deep Majumder <[hidden email]> wrote:
Yup, right you are! I had reinstalled my system and forgotten to install lld. Thanks!

On Thu, Mar 11, 2021 at 12:59 AM Benics Balázs <[hidden email]> wrote:
Use `lld` as a linker, it is superior to `ld` or `gold`.
It consumes less memory and much faster.
Also, prefer dynamic linking for faster development cycles.

Balazs.





From: cfe-dev <[hidden email]> on behalf of Deep Majumder via cfe-dev <[hidden email]>
Sent: Wednesday, March 10, 2021 8:17:38 PM
To: cfe-dev <[hidden email]>
Subject: [cfe-dev] Unusually high ram usage on tests
 
Hi everyone,
I am running 'ninja check-clang-analysis' on the release build of clang with LLVM_PARALLEL_LINK_JOBS set to 2 and assertions off. I am on Ubuntu 20.04. This is taking up almost 10 gb of memory (6 from ram, 4 from swap). I have been running this command for over a month and I haven't seen this sort of memory usage. Is this normal? Or am I missing something?
Warm regards,
Deep
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

-DBUILD_SHARED_LIBS=on

The increase on test execution time is huge so I only use it for -DCMAKE_BUILD_TYPE=Debug where I only tend to run a small set of tests during development.

Having both -DBUILD_SHARED_LIBS=on & -DBUILD_SHARED_LIBS=off configurations is handy when you add #include:
-DBUILD_SHARED_LIBS=on on Linux enables -Wl,-z,defs which requires every shared object to have full direct dependencies specified (gold and ld.lld behavior; not in GNU ld),
basically "link what you use" (similar to "include what you use").



_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev