Memory and time consumption for larger files

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Memory and time consumption for larger files

adamhusar
Hello,
   recently I encountered one rather unpleasant issue that
I would like to share with you.
   Me and few colleagues are working on a project where we want to
create a development environment for application-specific processor
processors (web pages, currently not much up-to-date are here:  
http://merlin.fit.vutbr.cz/Lissom/).
One part of this project is a compiler generator.
   To generate instruction selection patterns from our
architecture description language ISAC, one function that describes  
semantics of each
instruction is generated. File that contains these functions is then
compiled and contents of functions are optimized, so I get something quite  
close
to instruction selection patterns.
   For some architectures like ARM, the count of generated functions
is huge (e.g. 50000) and the resulting C file is huge too.

The problem here is that the compilation to LLVM IR using frontend takes
enormous amount of time and memory.

---------------------------------------------------------------------------------

Experiments are shown for C file or size 12 MB, functions have approx. 30  
lines each,
preprocessed file can be downloaded here:  
http://lissom.aps-brno.cz/tmp/clang-large-source.c.zip

   Tests were run on Fedora 11 64-bit, Pentium Quad Core, 2.83GHz, 4GB of  
memory.
Latest llvm and clang from llvm, rev. 99810, configured and compiled with  
--enable-optimized (uses -O2).
clang version 1.0  
(https://llvm.org/svn/llvm-project/cfe/branches/release_26 exported)

   Using GCC, gcc (GCC) 4.4.1 20090725 (Red Hat 4.4.1-2), time is only  
illustrative,
because also compilation into object file is included:
The top memory is just approximation observed from output of the top  
command.


1) g++ -DLISSOM_SEM -O0 -c -o tst.o clang-large-source.c
(time is only illustrative, because object code file is generated)
time: 12m17.064s
top memory approx: 2.6 GB


2) llvm-g++ -DLISSOM_SEM -O0 -c --emit-llvm -o tst.bc clang-large-source.c
time: 6m28.518s
top memory approx: 8 GB

3a) clang -DLISSOM_SEM -DCLANG -c -O0 -o tst.bc clang-large-source.c
time: 11m15.332s
top memory  approx 8 GB


   Resulting file tst.bc with debug info has 250 MB.
Without debug info  (-g0), compilation seems to be even slower, but it was  
maybe because
some swapping collision occurred, I was not patient enough to let it  
finish,
resulting file for llvm-g++ had 181 MB.

   Note also that on a 32-bit machine, the compilation would fail because  
of lack
of memory space.


   If I run then the opt -O3 on bc file generated with debug info,
it also consumes 7GB of memory and finishes in 9 minutes.

   In my opinion, 12 MB of source code is not so much and the compilation  
could be almost
immediate (or at least to be less than one minute), because no  
optimizations are made.
Especially, what i don't understand, is the big difference between code  
size and
the needed memory. After preprocessing, the C file has still 12MB, so  
roughly, each
byte from source file needs 660 bytes in memory, 20 bytes in resulting  
bytecode
and 100 bytes in disassembled bytecode.

-------------------------------------------------------------------------------------

Maybe there could be some batch mode that would parse the file by
smaller pieces, so the top memory usage would be lower.

   If I divide the file into smaller files, compilation takes much less  
time.
The question is, whether it is necessary, for example when -O0
is selected, to keep the whole program representation in memory.

time clang -DLISSOM_SEM -DCLANG -c -O0 -o tst.bc cg_instrsem_incl.c

for 2,5 MB file:
g++ (with obj. code generation): 1m 6s
llvm-g++: 7 s
clang: 2m2.501s

for 1 MB file:

g++ (with obj. code generation): 23 secs
llvm-g++: 2.5 s
clang time: 42 secs

   Here I do not much understand, why is clang so much slower than llvm-g++.
I checked, that it was configured with --enable-optimized more than once
(does this affect also the clang?).
Testing files can be found here:  
http://lissom.aps-brno.cz/tmp/clang_test.zip

------------------------------------------------------------------------------

Probably should this text go into bugzilla, but I thought it would be  
better
that more people would see it and maybe would be interested in the reason,  
why
clang behaves this way.
   Anyway, clang is a great piece of software and I am very looking forward  
to
see it replace gcc frontend with its cryptic error messages.
   However as the abstraction level of program description is
moving higher and higher, I am afraid it will not be uncommon to generate  
such
huge files from other higher-level languages that will use C as some kind  
of
universal assembler (as currently is done with Matlab or
some graphical languages).
   Such high memory and time requirements could pose problem for using  
clang as
compiler for generated C code.


   Or, do you have any ideas, when I would like to use clang, how to
make the compilation faster? (and of course, I already ordered more memory  
for my computer:).
   Also, if anyone would be more interested, what do I need to do with  
these files i need
to compile, you can write me an email.

Have a nice day
   Adam H.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory and time consumption for larger files

Douglas Gregor

On Mar 29, 2010, at 8:37 AM, ihusar wrote:

> 1) g++ -DLISSOM_SEM -O0 -c -o tst.o clang-large-source.c
> (time is only illustrative, because object code file is generated)
> time: 12m17.064s
> top memory approx: 2.6 GB
>
>
> 2) llvm-g++ -DLISSOM_SEM -O0 -c --emit-llvm -o tst.bc clang-large-source.c
> time: 6m28.518s
> top memory approx: 8 GB
>
> 3a) clang -DLISSOM_SEM -DCLANG -c -O0 -o tst.bc clang-large-source.c
> time: 11m15.332s
> top memory  approx 8 GB


When we're testing Clang's performance, we tend to test the different phases of translation to determine where we have a performance problem. For example:

        1) Using the -E option tests preprocessing performance
        2) Using the -fsyntax-only option tests parsing/semantic analysis performance (its time includes (1))
        3) Using the -emit-llvm option tests IR generation performance (its time includes (1) and (2))

llvm-g++ takes the same command-line options, so one can build up a chart fairly quickly to see which phase of Clang's translation is slower than llvm-g++'s corresponding phase. That would tell us where we need to look to start optimizing.

        - Doug
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory and time consumption for larger files, mistake in measurement

adamhusar
In reply to this post by adamhusar
Hello, right after I sent previous message, I realized that I had wrong  
paths set,
sorry for that.
The previous results were for unoptimized clang,
here i fix it and results are for optimized clang.
Now the times match those of llvm-g++, however the huge memory consumption,
especially compared to standard gcc, did not change.
I am not sure if this is still a problem, however, on a 32-bit machine,
it will not be possible to compile the testing code.

As clang was used:
$clang --version
clang version 1.5 (trunk 99809)
Target: x86_64-unknown-linux-gnu

With *** are marked new measurement results.


>
> 1) g++ -DLISSOM_SEM -O0 -c -o tst.o clang-large-source.c
> (time is only illustrative, because object code file is generated)
> time: 12m17.064s
> top memory approx: 2.6 GB
>
>
> 2) llvm-g++ -DLISSOM_SEM -O0 -c --emit-llvm -o tst.bc  
> clang-large-source.c
> time: 6m28.518s
> top memory approx: 8 GB
>

*** 3a) clang -DLISSOM_SEM -DCLANG -c -O0 -o tst.bc clang-large-source.c
*** time: 1m55s (avg. from 3 runs)    (this is almost fine, later I would  
report what will happen on a computer with more memory)
*** top memory  approx 8.5 GB



> for 2,5 MB file:
> g++ (with obj. code generation): 1m 6s
> llvm-g++: 7 s


*** time clang -DLISSOM_SEM -DCLANG -O0 -c -emit-llvm  2_5MB_clang_test.c
*** clang: 8.5 s (avg from 3 runs)


> for 1 MB file:
>
> g++ (with obj. code generation): 23 secs
> llvm-g++: 2.5 s

*** time clang -DLISSOM_SEM -DCLANG -O0 -c -emit-llvm  1MB_clang_test.c
*** clang time: 2.7 secs (avg from 3 runs)



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory and time consumption for larger files, mistake in measurement

Douglas Gregor

On Mar 29, 2010, at 9:10 AM, ihusar wrote:

> Hello, right after I sent previous message, I realized that I had wrong  
> paths set,
> sorry for that.
> The previous results were for unoptimized clang,
> here i fix it and results are for optimized clang.
> Now the times match those of llvm-g++, however the huge memory consumption,
> especially compared to standard gcc, did not change.
> I am not sure if this is still a problem, however, on a 32-bit machine,
> it will not be possible to compile the testing code.

The memory consumption still looks like it could be an issue. However, we still need to know which part of the compiler is requiring so much memory: I'm guessing it's the LLVM back-end, since both llvm-g++ and clang++ are affected (and we've rarely seen Clang itself take more memory than GCC). Could you measure the memory usage difference between using -fsyntax-only and using -emit-llvm?

        - Doug
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory consumption measurement results

adamhusar
On Mon, 29 Mar 2010 18:15:01 +0200, Douglas Gregor <[hidden email]>  
wrote:

>
> On Mar 29, 2010, at 9:10 AM, ihusar wrote:
>
>> Hello, right after I sent previous message, I realized that I had wrong
>> paths set,
>> sorry for that.
>> The previous results were for unoptimized clang,
>> here i fix it and results are for optimized clang.
>> Now the times match those of llvm-g++, however the huge memory  
>> consumption,
>> especially compared to standard gcc, did not change.
>> I am not sure if this is still a problem, however, on a 32-bit machine,
>> it will not be possible to compile the testing code.
>
> The memory consumption still looks like it could be an issue. However,  
> we still need to know which part of the compiler is requiring so much  
> memory: I'm guessing it's the LLVM back-end, since both llvm-g++ and  
> clang++ are affected (and we've rarely seen Clang itself take more  
> memory than GCC). Could you measure the memory usage difference between  
> using -fsyntax-only and using -emit-llvm?
>
> - Doug

Hello,

   so I did some memory consumption measurement and the results are as  
follows:

source file size: 12MB
resulting .bc file with debug info has 250MB

clang: max. 4.8 GB
clang -fsyntax-only: max. ~120 MB
llvm-gcc: max. 4.1 GB
gcc 4.4.1: max 1.4 GB
opt -O3: max. 3.6 GB


source file size: 24MB

clang: max. 7.5 GB, stopped after 1 hour
gcc: after 11 seconds: gcc: Internal error: Segmentation fault (program  
cc1), max. 1.4 GB


Details and graphs showing dependence between time and
memory consumption can be found in http://lissom.aps-brno.cz/tmp/clang.xls

(the values in previous email were wrong, I was adding the virtual memory  
(VIRT) and
real memory (RES) and I did not realize that on by virtual memory is meant  
the whole
address space used by a process).


The testing file contains many simple functions (each has max. 30 lines).
It seems that the high memory consumption may be caused by
the memory LLVM IR representation, because the "opt" has similar  
requirements.
The LLVM IR objects have only virtual destructors, so the virtual function  
tables
should no be large(?), maybe that there are so many pointers between  
instructions,
values etc. (but of course, they are very useful). Is this not caused by  
debug information?
(I have not much inspected this, but using -g0 for clang seemed to have  
not much impact.)

Maybe, is it no necessary for the frontend to hold the whole program in  
memory?
However, it is for sure useful for the optimizer.

This memory consumption also limits the possibilities for paralellization,
if I would like to run two optimizers on one computer at the same time
on two different programs (lets say I would split my testing file into  
two),
because of the memory consumption, most of the time would be spent on
swapping.

There is also a question, whether this is really for clang and llvm an  
issue,
if it is not just my personal problem because I need to compile generated  
files,
anyway, gcc uses almost three times less memory.

Best regards
   Adam Husar




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory consumption measurement results

adamhusar

>
> source file size: 24MB
>
> clang: max. 7.5 GB, stopped after 1 hour
> gcc: after 11 seconds: gcc: Internal error: Segmentation fault (program
> cc1), max. 1.4 GB
>
>
Just a note:
I reported this bug to the gcc bugzilla and it is already fixed.
There was a problem with allocating large data on stack using alloca.
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory consumption measurement results

Eli Friedman
In reply to this post by adamhusar
On Wed, Mar 31, 2010 at 4:33 AM, ihusar <[hidden email]> wrote:
>   so I did some memory consumption measurement and the results are as
> follows:
>
> source file size: 12MB
> resulting .bc file with debug info has 250MB
>
> clang: max. 4.8 GB
> clang -fsyntax-only: max. ~120 MB

Good to know.

> llvm-gcc: max. 4.1 GB
> gcc 4.4.1: max 1.4 GB
> opt -O3: max. 3.6 GB

Can you check the max memory consumption of "opt result.bc
-disable-output" (running opt without any optimization passes)?

-Eli

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory consumption measurement results

Chris Lattner
In reply to this post by adamhusar
On Mar 31, 2010, at 4:33 AM, ihusar wrote:

>   so I did some memory consumption measurement and the results are as  
> follows:
>
> source file size: 12MB
> resulting .bc file with debug info has 250MB
>
> clang: max. 4.8 GB
> clang -fsyntax-only: max. ~120 MB
> llvm-gcc: max. 4.1 GB
> gcc 4.4.1: max 1.4 GB
> opt -O3: max. 3.6 GB
>
>
> source file size: 24MB
>
> clang: max. 7.5 GB, stopped after 1 hour
> gcc: after 11 seconds: gcc: Internal error: Segmentation fault (program  
> cc1), max. 1.4 GB

This is interesting.  I infer that these memory usage numbers are for "clang -O0 -g"?  If so, you might want to try a new build, last night several memory and compile time improvements went in for debug info, and more are coming.

-Chris
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory consumption measurement results

jahanian
In reply to this post by adamhusar

On Mar 31, 2010, at 4:33 AM, ihusar wrote:

> On Mon, 29 Mar 2010 18:15:01 +0200, Douglas Gregor <[hidden email]>
> wrote:
>
>>
>> On Mar 29, 2010, at 9:10 AM, ihusar wrote:
>>
>>> Hello, right after I sent previous message, I realized that I had  
>>> wrong
>>> paths set,
>>> sorry for that.
>>> The previous results were for unoptimized clang,
>>> here i fix it and results are for optimized clang.
>>> Now the times match those of llvm-g++, however the huge memory
>>> consumption,
>>> especially compared to standard gcc, did not change.
>>> I am not sure if this is still a problem, however, on a 32-bit  
>>> machine,
>>> it will not be possible to compile the testing code.
>>
>> The memory consumption still looks like it could be an issue.  
>> However,
>> we still need to know which part of the compiler is requiring so much
>> memory: I'm guessing it's the LLVM back-end, since both llvm-g++ and
>> clang++ are affected (and we've rarely seen Clang itself take more
>> memory than GCC). Could you measure the memory usage difference  
>> between
>> using -fsyntax-only and using -emit-llvm?
>>
>> - Doug
>
> Hello,
>
>   so I did some memory consumption measurement and the results are as
> follows:
>
> source file size: 12MB
> resulting .bc file with debug info has 250MB
>
> clang: max. 4.8 GB
>
> llvm-gcc: max. 4.1 GB

I am curious what these two numbers tell us? Does clang's memory  
consumption is higher than gcc's fe or
clang's output forces more memory consumption on the part of llvm back-
end? I think it is the latter rather
than the former, as we generally consume less memory in clang than  
gcc's fe.

- Fariborz

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory consumption measurement results, for revision 100183

adamhusar
In reply to this post by Chris Lattner
On Wed, 31 Mar 2010 18:33:08 +0200, Chris Lattner <[hidden email]>  
wrote:

> On Mar 31, 2010, at 4:33 AM, ihusar wrote:
>>   so I did some memory consumption measurement and the results are as
>> follows:
>>
>> source file size: 12MB
>> resulting .bc file with debug info has 250MB
>>
>> clang: max. 4.8 GB
>> clang -fsyntax-only: max. ~120 MB
>> llvm-gcc: max. 4.1 GB
>> gcc 4.4.1: max 1.4 GB
>> opt -O3: max. 3.6 GB
>>
>>
>> source file size: 24MB
>>
>> clang: max. 7.5 GB, stopped after 1 hour
>> gcc: after 11 seconds: gcc: Internal error: Segmentation fault (program
>> cc1), max. 1.4 GB
>
> This is interesting.  I infer that these memory usage numbers are for  
> "clang -O0 -g"?  If so, you might want to try a new build, last night  
> several memory and compile time improvements went in for debug info, and  
> more are coming.
>
> -Chris


Hello,
   I made some tests for the latest revision 100183
and the memory consumption has gotten even higher compared to revision  
99809.

For the same 12MB file:

clang (no args): 5GB
clang -O0 -g0: 5GB
clang -O0 -g: 6.3GB, stopped after 33 minutes
opt --disable-output: 3.7 GB
opt -O3 --disable-output: 3.9 GB

Details can be found here:  
http://lissom.aps-brno.cz/tmp/clang_rev100183.xls
and http://lissom.aps-brno.cz/tmp/clang_rev99809.xls

I will probably solve this problem by splitting this large file into
smaller ones and by buying more memory, but I think it would be at
least interesting to see, what is the main reason that the memory  
consumption
is so much higher than in gcc.

Best regards
   Adam
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory consumption measurement results, for revision 100183

Chris Lattner

On Apr 2, 2010, at 6:24 AM, ihusar wrote:

> Hello,
>   I made some tests for the latest revision 100183
> and the memory consumption has gotten even higher compared to revision  
> 99809.
>
> For the same 12MB file:
>
> clang (no args): 5GB
> clang -O0 -g0: 5GB
> clang -O0 -g: 6.3GB, stopped after 33 minutes
> opt --disable-output: 3.7 GB
> opt -O3 --disable-output: 3.9 GB
>
> Details can be found here:  
> http://lissom.aps-brno.cz/tmp/clang_rev100183.xls
> and http://lissom.aps-brno.cz/tmp/clang_rev99809.xls

Please try again post r100261, it should substantially improve the 1.3G of bloat going from -g0 -> -g.

-Chris
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory consumption measurement results, for revision 100620

adamhusar

>
> On Apr 2, 2010, at 6:24 AM, ihusar wrote:
>> Hello,
>>   I made some tests for the latest revision 100183
>> and the memory consumption has gotten even higher compared to revision
>> 99809.
>>
>> For the same 12MB file:
>>
>> clang (no args): 5GB
>> clang -O0 -g0: 5GB
>> clang -O0 -g: 6.3GB, stopped after 33 minutes
>> opt --disable-output: 3.7 GB
>> opt -O3 --disable-output: 3.9 GB
>>
>> Details can be found here:
>> http://lissom.aps-brno.cz/tmp/clang_rev100183.xls
>> and http://lissom.aps-brno.cz/tmp/clang_rev99809.xls
>
> Please try again post r100261, it should substantially improve the 1.3G  
> of bloat going from -g0 -> -g.
>
> -Chris



Hello,

   so I bought some memory, so my computer has 8 GB now and results for a  
quite recent revision 100620
are as follows:

clang -g0 -O0: 5 GB
clang -g1 -O0: 7.6 GB  (for previous results for revision 100183, the test  
was stopped before it ended)
clang -g3 -O0: 7.6 GB

opt --disable-output: 3.8 GB (for file generated with clang -g3)

gcc -g0 -O0: 1.4 GB (object file for x86 was also generated)
gcc -g1 -O0: 1.4 GB
gcc -g3 -O0: 2.0 GB


The memory consumption for large files is much higher than in gcc.
The only serious problem for standard usage I can imagine right now, is  
when "make" tool runs
multiple instances of clang in parallel. Also, for large generated files  
would be the compilation
on standard computer unbearably slow because of swapping, or on a 32-bit  
machine impossible
because of insufficient address space.

Again, details are here:
http://lissom.aps-brno.cz/tmp/clang_rev100620.xls,
http://lissom.aps-brno.cz/tmp/clang_rev100183.xls
and http://lissom.aps-brno.cz/tmp/clang_rev99809.xls.


Best regards,
   Adam
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev