Windows /bigobj

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Windows /bigobj

Richard Smith via cfe-dev
Hi,

Recently one Windows build bot failed by my commit, because the obj file being generated is too big:

C:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\llvm\tools\clang\unittests\AST\ASTImporterTest.cpp : fatal error C1128: number of sections exceeded object file format limit: compile with /bigobj

Is there an LLVM policy to limit the maximum size of the generated obj files?
If not then I suspect the only solution is to turn on /bigobj on the build bot, right?
(Of course, I could refactor the cpp file to several smaller ones, but how do I know where to cut it?)

Thanks,
Gabor

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows /bigobj

Richard Smith via cfe-dev
On Mon, Dec 17, 2018 at 10:12 AM Gábor Márton via llvm-dev
<[hidden email]> wrote:
>
> Hi,
>
> Recently one Windows build bot failed by my commit, because the obj file being generated is too big:
>
> C:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\llvm\tools\clang\unittests\AST\ASTImporterTest.cpp : fatal error C1128: number of sections exceeded object file format limit: compile with /bigobj
>
> Is there an LLVM policy to limit the maximum size of the generated obj files?

We usually try to limit it in cases where it makes sense to do so.
e.g., for a while we would put in efforts to reduce the number of
template instantiations to help avoid this problem. However, I believe
we only do that up to a point.

> If not then I suspect the only solution is to turn on /bigobj on the build bot, right?

We already required that for some files; we turn on /bigobj support
for several files in Clang. We usually only do this on a file-by-file
basis though, so we can be alerted when new files are causing
problems.

~Aaron

> (Of course, I could refactor the cpp file to several smaller ones, but how do I know where to cut it?)
>
> Thanks,
> Gabor
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows /bigobj

Richard Smith via cfe-dev

Hi Aaron, thanks for your answer.

> We usually try to limit it in cases where it makes sense to do so.
e.g., for a while we would put in efforts to reduce the number of
template instantiations to help avoid this problem. However, I believe
we only do that up to a point.

The corresponding file is a unittest file "ASTImporterTest.cpp" with many gtest macros, which I suspect do quite many template instantiations.
I don't think it would be easy to reduce the size in this case, but I am open to any idea.
Is it okay to connect with the buildbot owner and ask for /bigobj for this file? Who should approve such decisions? 

Thanks,
Gabor


On Mon, Dec 17, 2018 at 4:18 PM Aaron Ballman <[hidden email]> wrote:
On Mon, Dec 17, 2018 at 10:12 AM Gábor Márton via llvm-dev
<[hidden email]> wrote:
>
> Hi,
>
> Recently one Windows build bot failed by my commit, because the obj file being generated is too big:
>
> C:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\llvm\tools\clang\unittests\AST\ASTImporterTest.cpp : fatal error C1128: number of sections exceeded object file format limit: compile with /bigobj
>
> Is there an LLVM policy to limit the maximum size of the generated obj files?

We usually try to limit it in cases where it makes sense to do so.
e.g., for a while we would put in efforts to reduce the number of
template instantiations to help avoid this problem. However, I believe
we only do that up to a point.

> If not then I suspect the only solution is to turn on /bigobj on the build bot, right?

We already required that for some files; we turn on /bigobj support
for several files in Clang. We usually only do this on a file-by-file
basis though, so we can be alerted when new files are causing
problems.

~Aaron

> (Of course, I could refactor the cpp file to several smaller ones, but how do I know where to cut it?)
>
> Thanks,
> Gabor
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Windows /bigobj

Richard Smith via cfe-dev
On Mon, Dec 17, 2018 at 10:48 AM Gábor Márton <[hidden email]> wrote:

> Hi Aaron, thanks for your answer.
>
> > We usually try to limit it in cases where it makes sense to do so.
> e.g., for a while we would put in efforts to reduce the number of
> template instantiations to help avoid this problem. However, I believe
> we only do that up to a point.
>
> The corresponding file is a unittest file "ASTImporterTest.cpp" with many gtest macros, which I suspect do quite many template instantiations.
> I don't think it would be easy to reduce the size in this case, but I am open to any idea.
> Is it okay to connect with the buildbot owner and ask for /bigobj for this file? Who should approve such decisions?

This is done on a file-by-file basis rather than a bot-by-bot basis.
Simon made the recommended changes in r349357 by updating the CMake
file to properly add the flag to just ASTImporterTest.cpp, so
everything should be in a good state now.

~Aaron

>
> Thanks,
> Gabor
>
>
> On Mon, Dec 17, 2018 at 4:18 PM Aaron Ballman <[hidden email]> wrote:
>>
>> On Mon, Dec 17, 2018 at 10:12 AM Gábor Márton via llvm-dev
>> <[hidden email]> wrote:
>> >
>> > Hi,
>> >
>> > Recently one Windows build bot failed by my commit, because the obj file being generated is too big:
>> >
>> > C:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\llvm\tools\clang\unittests\AST\ASTImporterTest.cpp : fatal error C1128: number of sections exceeded object file format limit: compile with /bigobj
>> >
>> > Is there an LLVM policy to limit the maximum size of the generated obj files?
>>
>> We usually try to limit it in cases where it makes sense to do so.
>> e.g., for a while we would put in efforts to reduce the number of
>> template instantiations to help avoid this problem. However, I believe
>> we only do that up to a point.
>>
>> > If not then I suspect the only solution is to turn on /bigobj on the build bot, right?
>>
>> We already required that for some files; we turn on /bigobj support
>> for several files in Clang. We usually only do this on a file-by-file
>> basis though, so we can be alerted when new files are causing
>> problems.
>>
>> ~Aaron
>>
>> > (Of course, I could refactor the cpp file to several smaller ones, but how do I know where to cut it?)
>> >
>> > Thanks,
>> > Gabor
>> > _______________________________________________
>> > LLVM Developers mailing list
>> > [hidden email]
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev