Speed up debug version of LibTooling AST parsing

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Speed up debug version of LibTooling AST parsing

suyash singh via cfe-dev
We are using Clang LibTooling for source-to-source conversion. Size of converted headers is around 10MB.

Release version of our conversion utility works fine, but debug version is spending 20 seconds in clang::ParseAST.

Is there any chance how to speed up parsing, for example using precompiled headers (or storing parsed AST)?

Could you please point me in the right direction?

Best regards
Jan Rysavy
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Speed up debug version of LibTooling AST parsing

suyash singh via cfe-dev
Unless you need the actual Debug build for anything, maybe try sticking to assertions and a release build? (-DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release) Release build is fast, small, and with enough assertions (and maybe putting "llvm/Support/Debug.h" into the file and calling the frontend with -debug?) you could get most of the stuff you need for debugging.

Jan Ryšavý via cfe-dev <[hidden email]> ezt írta (időpont: 2020. febr. 17., H, 10:19):
We are using Clang LibTooling for source-to-source conversion. Size of converted headers is around 10MB.

Release version of our conversion utility works fine, but debug version is spending 20 seconds in clang::ParseAST.

Is there any chance how to speed up parsing, for example using precompiled headers (or storing parsed AST)?

Could you please point me in the right direction?

Best regards
Jan Rysavy
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Speed up debug version of LibTooling AST parsing

suyash singh via cfe-dev
Thank you for this idea. Trouble with LibTooling in Release build is we must use Release build also for our utility (Windows executable), otherwise we are receiving following RTL conflicts:

1>clangAST.lib(DeclarationName.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in CmdLineUtils.obj
1>clangAST.lib(ASTTypeTraits.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in CmdLineUtils.obj
etc.

Of course we can turn off optimizations and debug in Release build...

Best regards
Jan Rysavy

On 17.02.2020 15:13, Whisperity wrote:

> Unless you need the actual Debug build for anything, maybe try sticking to assertions and a release build? (-DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release) Release build is fast, small, and with enough assertions (and maybe putting "llvm/Support/Debug.h" into the file and calling the frontend with -debug?) you could get most of the stuff you need for debugging.
>
> Jan Ryšavý via cfe-dev <[hidden email] <mailto:[hidden email]>> ezt írta (időpont: 2020. febr. 17., H, 10:19):
>
>     We are using Clang LibTooling for source-to-source conversion. Size of converted headers is around 10MB.
>
>     Release version of our conversion utility works fine, but debug version is spending 20 seconds in clang::ParseAST.
>
>     Is there any chance how to speed up parsing, for example using precompiled headers (or storing parsed AST)?
>
>     Could you please point me in the right direction?
>
>     Best regards
>     Jan Rysavy
>     _______________________________________________
>     cfe-dev mailing list
>     [hidden email] <mailto:[hidden email]>
>     https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Speed up debug version of LibTooling AST parsing

suyash singh via cfe-dev
In reply to this post by suyash singh via cfe-dev
On Monday 17 of February 2020, Jan Ryšavý via cfe-dev wrote:

> We are using Clang LibTooling for source-to-source conversion. Size of
> converted headers is around 10MB.
>
> Release version of our conversion utility works fine, but debug version is
> spending 20 seconds in clang::ParseAST.
>
> Is there any chance how to speed up parsing, for example using precompiled
> headers (or storing parsed AST)?
>
> Could you please point me in the right direction?

 You can normally add -include-pch file.pch to the args passed to the tooling
call, we do it in LibreOffice
(https://cgit.freedesktop.org/libreoffice/core/tree/compilerplugins/clang/sharedvisitor/analyzer.cxx?id=9ab2703d5c212746c961dcb238a4c3025d3a2587#n274).

--
 Lubos Lunak
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Speed up debug version of LibTooling AST parsing

suyash singh via cfe-dev
In reply to this post by suyash singh via cfe-dev


On Mon, Feb 17, 2020 at 7:15 AM Jan Ryšavý via cfe-dev <[hidden email]> wrote:
Thank you for this idea. Trouble with LibTooling in Release build is we must use Release build also for our utility (Windows executable), otherwise we are receiving following RTL conflicts:

Why would you not want to build your utility in Release mode in any case where performance/runtime speed of the program matters to you? Perhaps some more context would be helpful to understand the tradeoffs you're trying to make.
 

1>clangAST.lib(DeclarationName.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in CmdLineUtils.obj
1>clangAST.lib(ASTTypeTraits.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in CmdLineUtils.obj
etc.

Of course we can turn off optimizations and debug in Release build...

Best regards
Jan Rysavy

On 17.02.2020 15:13, Whisperity wrote:
> Unless you need the actual Debug build for anything, maybe try sticking to assertions and a release build? (-DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release) Release build is fast, small, and with enough assertions (and maybe putting "llvm/Support/Debug.h" into the file and calling the frontend with -debug?) you could get most of the stuff you need for debugging.
>
> Jan Ryšavý via cfe-dev <[hidden email] <mailto:[hidden email]>> ezt írta (időpont: 2020. febr. 17., H, 10:19):
>
>     We are using Clang LibTooling for source-to-source conversion. Size of converted headers is around 10MB.
>
>     Release version of our conversion utility works fine, but debug version is spending 20 seconds in clang::ParseAST.
>
>     Is there any chance how to speed up parsing, for example using precompiled headers (or storing parsed AST)?
>
>     Could you please point me in the right direction?
>
>     Best regards
>     Jan Rysavy
>     _______________________________________________
>     cfe-dev mailing list
>     [hidden email] <mailto:[hidden email]>
>     https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Speed up debug version of LibTooling AST parsing

suyash singh via cfe-dev
In reply to this post by suyash singh via cfe-dev
Thank you, -include-pch file.pch is exactly what I was looking for!

The debug version now takes 10 seconds instead of 20.

Best regards
Jan Rysavy

On 17.02.2020 18:59, Lubos Lunak via cfe-dev wrote:

> On Monday 17 of February 2020, Jan Ryšavý via cfe-dev wrote:
>> We are using Clang LibTooling for source-to-source conversion. Size of
>> converted headers is around 10MB.
>>
>> Release version of our conversion utility works fine, but debug version is
>> spending 20 seconds in clang::ParseAST.
>>
>> Is there any chance how to speed up parsing, for example using precompiled
>> headers (or storing parsed AST)?
>>
>> Could you please point me in the right direction?
>
>   You can normally add -include-pch file.pch to the args passed to the tooling
> call, we do it in LibreOffice
> (https://cgit.freedesktop.org/libreoffice/core/tree/compilerplugins/clang/sharedvisitor/analyzer.cxx?id=9ab2703d5c212746c961dcb238a4c3025d3a2587#n274).
>
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev