Measuring compilation performance

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Measuring compilation performance

David Blaikie via cfe-dev
I am experiencing, in general, very slow build times. However, I don't
see a way to measure on a per-file basis, the amount of time the
compiler spends compiling code. I think it would be useful to see
statistics on how many times a header file is compiled (collectively,
across all translation units) as well as per-file (H and CPP) how much
time is spent compiling code. This would allow me to find header files
included too often, and possibly fix it by removing includes, forward
declarations, pimpl idiom, etc. Time spent per-file would be nice to
know if, for example, the way I've implemented a template class is
causing long compile times and refactor it to be faster.

Right now I just don't have a way to do this. Are these metrics
inherently provided by clang? And if not, are there external tools to
help diagnose slow compilation? Advice is greatly appreciated.
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compilation performance

David Blaikie via cfe-dev
If you use ninja it has a build log detailing how long each translation
took. (see this: https://github.com/nico/ninjatracing)
Your header requests are not resolveable, because of the copy-paste +
Preprocessor semantics of headers. Thats why C++ is getting modules. The
data is only available for translation units.

Consider using `include-what-you-use` as well, helping with the
include-problems.

Best, Jonas

Am 10.01.19 um 16:05 schrieb Robert Dailey via cfe-dev:

> I am experiencing, in general, very slow build times. However, I don't
> see a way to measure on a per-file basis, the amount of time the
> compiler spends compiling code. I think it would be useful to see
> statistics on how many times a header file is compiled (collectively,
> across all translation units) as well as per-file (H and CPP) how much
> time is spent compiling code. This would allow me to find header files
> included too often, and possibly fix it by removing includes, forward
> declarations, pimpl idiom, etc. Time spent per-file would be nice to
> know if, for example, the way I've implemented a template class is
> causing long compile times and refactor it to be faster.
>
> Right now I just don't have a way to do this. Are these metrics
> inherently provided by clang? And if not, are there external tools to
> help diagnose slow compilation? Advice is greatly appreciated.
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compilation performance

David Blaikie via cfe-dev
Thanks for the advice. Last time I used IWYU, it created more problems
than it solved. It doesn't quite work very well for large code bases.

I guess you could technically perform a per-header-file analysis even
after the preprocessor has gone through files, by keeping track of the
"markers" where that code is concatenated for the actual compile. But
this would be introspection only available to Clang itself I guess,
which would mean a new clang feature.

On Thu, Jan 10, 2019 at 9:35 AM Jonas Toth <[hidden email]> wrote:

>
> If you use ninja it has a build log detailing how long each translation
> took. (see this: https://github.com/nico/ninjatracing)
> Your header requests are not resolveable, because of the copy-paste +
> Preprocessor semantics of headers. Thats why C++ is getting modules. The
> data is only available for translation units.
>
> Consider using `include-what-you-use` as well, helping with the
> include-problems.
>
> Best, Jonas
>
> Am 10.01.19 um 16:05 schrieb Robert Dailey via cfe-dev:
> > I am experiencing, in general, very slow build times. However, I don't
> > see a way to measure on a per-file basis, the amount of time the
> > compiler spends compiling code. I think it would be useful to see
> > statistics on how many times a header file is compiled (collectively,
> > across all translation units) as well as per-file (H and CPP) how much
> > time is spent compiling code. This would allow me to find header files
> > included too often, and possibly fix it by removing includes, forward
> > declarations, pimpl idiom, etc. Time spent per-file would be nice to
> > know if, for example, the way I've implemented a template class is
> > causing long compile times and refactor it to be faster.
> >
> > Right now I just don't have a way to do this. Are these metrics
> > inherently provided by clang? And if not, are there external tools to
> > help diagnose slow compilation? Advice is greatly appreciated.
> > _______________________________________________
> > cfe-dev mailing list
> > [hidden email]
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compilation performance

David Blaikie via cfe-dev
Without delving into Clang's intestines, you could hook a
shell/python/whatever script onto the -E output of all your
compilations (I think CMake creates a .i target for each source file)
and do some counting...
Another non-intrusive (from Clang's perspective) option would be to
strace the file operations and filter it to C++ files. There is an
inner buffer so the same header when included twice isn't opened
twice, but this buffer AFAIK isn't shared between simultaneous
compiler invocations.

Robert Dailey via cfe-dev <[hidden email]> ezt írta (időpont:
2019. jan. 10., Cs, 17:27):

>
> Thanks for the advice. Last time I used IWYU, it created more problems
> than it solved. It doesn't quite work very well for large code bases.
>
> I guess you could technically perform a per-header-file analysis even
> after the preprocessor has gone through files, by keeping track of the
> "markers" where that code is concatenated for the actual compile. But
> this would be introspection only available to Clang itself I guess,
> which would mean a new clang feature.
>
> On Thu, Jan 10, 2019 at 9:35 AM Jonas Toth <[hidden email]> wrote:
> >
> > If you use ninja it has a build log detailing how long each translation
> > took. (see this: https://github.com/nico/ninjatracing)
> > Your header requests are not resolveable, because of the copy-paste +
> > Preprocessor semantics of headers. Thats why C++ is getting modules. The
> > data is only available for translation units.
> >
> > Consider using `include-what-you-use` as well, helping with the
> > include-problems.
> >
> > Best, Jonas
> >
> > Am 10.01.19 um 16:05 schrieb Robert Dailey via cfe-dev:
> > > I am experiencing, in general, very slow build times. However, I don't
> > > see a way to measure on a per-file basis, the amount of time the
> > > compiler spends compiling code. I think it would be useful to see
> > > statistics on how many times a header file is compiled (collectively,
> > > across all translation units) as well as per-file (H and CPP) how much
> > > time is spent compiling code. This would allow me to find header files
> > > included too often, and possibly fix it by removing includes, forward
> > > declarations, pimpl idiom, etc. Time spent per-file would be nice to
> > > know if, for example, the way I've implemented a template class is
> > > causing long compile times and refactor it to be faster.
> > >
> > > Right now I just don't have a way to do this. Are these metrics
> > > inherently provided by clang? And if not, are there external tools to
> > > help diagnose slow compilation? Advice is greatly appreciated.
> > > _______________________________________________
> > > cfe-dev mailing list
> > > [hidden email]
> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev