ASTImporter patches and improvements, please help

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

ASTImporter patches and improvements, please help

Oleg Smolsky via cfe-dev
Dear LLDB Developers,

There is an ongoing activity to improve ASTImporter in Clang to make it support
C++ (and C) completely and correctly.  Our team works on cross translation unit
(CTU) static analysis where we use the in-tree Clang static analyzer in
combination with the ASTImporter (via the scan-build.py script or CodeChecker).
In the past 18 months we have committed more than 70 patches to make the
ASTImporter better.  Our primary target is Linux and we do run the LLDB tests
on Linux before we commit.  Unfortunately, quite recently 3 of these patches
broke the LLDB macOS buildbots (green.lab.llvm.org/green/view/LLDB) and this
caused some turmoil.  We are very sorry about this and we are making actions to
avoid such inconveniences in the future: We ordered a Mac Mini, until it
arrives we are using a borrowed Mac Book. We are going to create a CI job which
will execute the macOS LLDB test suite for a specific patch. Besides this, for
every patch we are going to monitor the macOS buildbots once they are
committed.

However, we are experiencing problems both with the buildbots and with the LLDB
tests, therefore we are asking help from the LLDB community in the following:

(1) Apparently, the green lab macOS buildbots are not displayed in the build
bot console (http://lab.llvm.org:8011/console). More importantly they are not
capable of sending emails to the authors of the commit in case of failure.
Would it be possible to setup the these buildbots to send out the emails for
the authors?

(2) We are facing difficulties with the LLDB lit tests both on macOS and on
Linux. E.g. on a fresh macOS mojave install, checking out master LLVM, Clang,
LLDB and then executing ninja check-lldb fails immediately.  On Linux we
experienced that some tests fail because they try to read a non-existent
register. Some tests fail non-deterministically because they can't kill a
process or a timeout is not long enough. Some tests fail because of a linker
error of libcxx. We would like to address all these issues. Would you like to
discuss these issues on lldb-dev or should we create a bugzilla ticket for
each?

(3) ASTImporter unit tests and lit tests in Clang for the LLDB specific usage
are very-very limited.  LLDB uses the so called "minimal" import, but none of
the unit tests exercises that (CTU uses the full import).  We should have a
unit test suite where the LLDB use cases are covered, e.g. after a minimal
import an `importDefinition` is called. Also, a test double which implements
the ExternalASTSource could be used.  I believe it would be possible to cover
with the unit tests all LLDB/macOS related scenarios and these tests could run
on Linux too.  We do something similar in case of windows: we execute all unit
tests with "-fdelayed-template-parsing" and with "-fms-compatibility" too. I
think, the more unit tests we have in Clang the sooner we catch the LLDB
specific importer errors.  I am asking the LLDB community's help here, because
we lack the domain knowledge in LLDB so we don't know what expectations should
we write in each unit test cases.  (About lit tests, there is clang-import-test
but its coverage seems pretty low and I don't know if that really exercises all
LLDB use cases.)

I think that from a better ASTImporter the LLDB debugger can benefit too: the
`expr` command might give better experience by supporting more C++ constructs
and by giving better diagnostics.

Thank you,
Gabor


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: ASTImporter patches and improvements, please help

Oleg Smolsky via cfe-dev
Hi Gábor,

I replied inline.

Am Sa., 1. Dez. 2018 um 17:00 Uhr schrieb Gábor Márton via cfe-dev
<[hidden email]>:

>
> Dear LLDB Developers,
>
> There is an ongoing activity to improve ASTImporter in Clang to make it support
> C++ (and C) completely and correctly.  Our team works on cross translation unit
> (CTU) static analysis where we use the in-tree Clang static analyzer in
> combination with the ASTImporter (via the scan-build.py script or CodeChecker).
> In the past 18 months we have committed more than 70 patches to make the
> ASTImporter better.  Our primary target is Linux and we do run the LLDB tests
> on Linux before we commit.  Unfortunately, quite recently 3 of these patches
> broke the LLDB macOS buildbots (green.lab.llvm.org/green/view/LLDB) and this
> caused some turmoil.  We are very sorry about this and we are making actions to
> avoid such inconveniences in the future: We ordered a Mac Mini, until it
> arrives we are using a borrowed Mac Book. We are going to create a CI job which
> will execute the macOS LLDB test suite for a specific patch. Besides this, for
> every patch we are going to monitor the macOS buildbots once they are
> committed.

Thanks a lot! And no worries about breaking the bots.

> However, we are experiencing problems both with the buildbots and with the LLDB
> tests, therefore we are asking help from the LLDB community in the following:
>
> (1) Apparently, the green lab macOS buildbots are not displayed in the build
> bot console (http://lab.llvm.org:8011/console). More importantly they are not
> capable of sending emails to the authors of the commit in case of failure.
> Would it be possible to setup the these buildbots to send out the emails for
> the authors?
>
> (2) We are facing difficulties with the LLDB lit tests both on macOS and on
> Linux. E.g. on a fresh macOS mojave install, checking out master LLVM, Clang,
> LLDB and then executing ninja check-lldb fails immediately.  On Linux we
> experienced that some tests fail because they try to read a non-existent
> register. Some tests fail non-deterministically because they can't kill a
> process or a timeout is not long enough. Some tests fail because of a linker
> error of libcxx. We would like to address all these issues. Would you like to
> discuss these issues on lldb-dev or should we create a bugzilla ticket for
> each?

I would just post about them on lldb-dev and then I think we can sort
out what are issues with the local setup and what are actual LLDB bugs
that should go to the bugtracker. The LLDB setup needs some small
extra attention that I don't think is really documented anywhere, so
that's a good chance for us to do so.

> (3) ASTImporter unit tests and lit tests in Clang for the LLDB specific usage
> are very-very limited.  LLDB uses the so called "minimal" import, but none of
> the unit tests exercises that (CTU uses the full import).  We should have a
> unit test suite where the LLDB use cases are covered, e.g. after a minimal
> import an `importDefinition` is called. Also, a test double which implements
> the ExternalASTSource could be used.  I believe it would be possible to cover
> with the unit tests all LLDB/macOS related scenarios and these tests could run
> on Linux too.  We do something similar in case of windows: we execute all unit
> tests with "-fdelayed-template-parsing" and with "-fms-compatibility" too. I
> think, the more unit tests we have in Clang the sooner we catch the LLDB
> specific importer errors.  I am asking the LLDB community's help here, because
> we lack the domain knowledge in LLDB so we don't know what expectations should
> we write in each unit test cases.  (About lit tests, there is clang-import-test
> but its coverage seems pretty low and I don't know if that really exercises all
> LLDB use cases.)

Yes, having full coverage of the ASTImporter in Clang would be very
nice. I started writing tests a few months back (with the help of
Aleksei who reviewed all of them!), but I currently don't have enough
time for doing the rest.

The clang-import-test tests are I think a good starting point. IIRC,
they actually do test the minimal import feature (as the first stage
of lazy building the AST). Their coverage should be ok-ish by now, but
it's obviously far from what we should have.

> I think that from a better ASTImporter the LLDB debugger can benefit too: the
> `expr` command might give better experience by supporting more C++ constructs
> and by giving better diagnostics.

Yes, I fully agree.

> Thank you,
> Gabor
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Am Sa., 1. Dez. 2018 um 17:00 Uhr schrieb Gábor Márton via cfe-dev
<[hidden email]>:

>
> Dear LLDB Developers,
>
> There is an ongoing activity to improve ASTImporter in Clang to make it support
> C++ (and C) completely and correctly.  Our team works on cross translation unit
> (CTU) static analysis where we use the in-tree Clang static analyzer in
> combination with the ASTImporter (via the scan-build.py script or CodeChecker).
> In the past 18 months we have committed more than 70 patches to make the
> ASTImporter better.  Our primary target is Linux and we do run the LLDB tests
> on Linux before we commit.  Unfortunately, quite recently 3 of these patches
> broke the LLDB macOS buildbots (green.lab.llvm.org/green/view/LLDB) and this
> caused some turmoil.  We are very sorry about this and we are making actions to
> avoid such inconveniences in the future: We ordered a Mac Mini, until it
> arrives we are using a borrowed Mac Book. We are going to create a CI job which
> will execute the macOS LLDB test suite for a specific patch. Besides this, for
> every patch we are going to monitor the macOS buildbots once they are
> committed.
>
> However, we are experiencing problems both with the buildbots and with the LLDB
> tests, therefore we are asking help from the LLDB community in the following:
>
> (1) Apparently, the green lab macOS buildbots are not displayed in the build
> bot console (http://lab.llvm.org:8011/console). More importantly they are not
> capable of sending emails to the authors of the commit in case of failure.
> Would it be possible to setup the these buildbots to send out the emails for
> the authors?
>
> (2) We are facing difficulties with the LLDB lit tests both on macOS and on
> Linux. E.g. on a fresh macOS mojave install, checking out master LLVM, Clang,
> LLDB and then executing ninja check-lldb fails immediately.  On Linux we
> experienced that some tests fail because they try to read a non-existent
> register. Some tests fail non-deterministically because they can't kill a
> process or a timeout is not long enough. Some tests fail because of a linker
> error of libcxx. We would like to address all these issues. Would you like to
> discuss these issues on lldb-dev or should we create a bugzilla ticket for
> each?
>
> (3) ASTImporter unit tests and lit tests in Clang for the LLDB specific usage
> are very-very limited.  LLDB uses the so called "minimal" import, but none of
> the unit tests exercises that (CTU uses the full import).  We should have a
> unit test suite where the LLDB use cases are covered, e.g. after a minimal
> import an `importDefinition` is called. Also, a test double which implements
> the ExternalASTSource could be used.  I believe it would be possible to cover
> with the unit tests all LLDB/macOS related scenarios and these tests could run
> on Linux too.  We do something similar in case of windows: we execute all unit
> tests with "-fdelayed-template-parsing" and with "-fms-compatibility" too. I
> think, the more unit tests we have in Clang the sooner we catch the LLDB
> specific importer errors.  I am asking the LLDB community's help here, because
> we lack the domain knowledge in LLDB so we don't know what expectations should
> we write in each unit test cases.  (About lit tests, there is clang-import-test
> but its coverage seems pretty low and I don't know if that really exercises all
> LLDB use cases.)
>
> I think that from a better ASTImporter the LLDB debugger can benefit too: the
> `expr` command might give better experience by supporting more C++ constructs
> and by giving better diagnostics.
>
> Thank you,
> Gabor
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: ASTImporter patches and improvements, please help

Oleg Smolsky via cfe-dev
In reply to this post by Oleg Smolsky via cfe-dev
Hi Gabor,

Thank you for taking care of this issue!
I completely agree with what you said. In my opinion, there are also two more things we have to care about.

1. We need to involve LLDB developers into ASTImporter reviews. While we are familiar with the ASTImporter usage in CSA, LLDB developers are much more familiar with the usage of this component in LLDB so their opinion is always appreciated. If there is anybody who can participate in the reviews - please let us know.
2. We have to get more familiar with LLDB ourself. The code related to the ASTImporter looks pretty well isolated so I think there will be no too much problems with it. The build bots breakage already made us take a look on the related components.


01.12.2018 19:00, Gábor Márton via cfe-dev пишет:
Dear LLDB Developers,

There is an ongoing activity to improve ASTImporter in Clang to make it support
C++ (and C) completely and correctly.  Our team works on cross translation unit
(CTU) static analysis where we use the in-tree Clang static analyzer in
combination with the ASTImporter (via the scan-build.py script or CodeChecker).
In the past 18 months we have committed more than 70 patches to make the
ASTImporter better.  Our primary target is Linux and we do run the LLDB tests
on Linux before we commit.  Unfortunately, quite recently 3 of these patches
broke the LLDB macOS buildbots (green.lab.llvm.org/green/view/LLDB) and this
caused some turmoil.  We are very sorry about this and we are making actions to
avoid such inconveniences in the future: We ordered a Mac Mini, until it
arrives we are using a borrowed Mac Book. We are going to create a CI job which
will execute the macOS LLDB test suite for a specific patch. Besides this, for
every patch we are going to monitor the macOS buildbots once they are
committed.

However, we are experiencing problems both with the buildbots and with the LLDB
tests, therefore we are asking help from the LLDB community in the following:

(1) Apparently, the green lab macOS buildbots are not displayed in the build
bot console (http://lab.llvm.org:8011/console). More importantly they are not
capable of sending emails to the authors of the commit in case of failure.
Would it be possible to setup the these buildbots to send out the emails for
the authors?

(2) We are facing difficulties with the LLDB lit tests both on macOS and on
Linux. E.g. on a fresh macOS mojave install, checking out master LLVM, Clang,
LLDB and then executing ninja check-lldb fails immediately.  On Linux we
experienced that some tests fail because they try to read a non-existent
register. Some tests fail non-deterministically because they can't kill a
process or a timeout is not long enough. Some tests fail because of a linker
error of libcxx. We would like to address all these issues. Would you like to
discuss these issues on lldb-dev or should we create a bugzilla ticket for
each?

(3) ASTImporter unit tests and lit tests in Clang for the LLDB specific usage
are very-very limited.  LLDB uses the so called "minimal" import, but none of
the unit tests exercises that (CTU uses the full import).  We should have a
unit test suite where the LLDB use cases are covered, e.g. after a minimal
import an `importDefinition` is called. Also, a test double which implements
the ExternalASTSource could be used.  I believe it would be possible to cover
with the unit tests all LLDB/macOS related scenarios and these tests could run
on Linux too.  We do something similar in case of windows: we execute all unit
tests with "-fdelayed-template-parsing" and with "-fms-compatibility" too. I
think, the more unit tests we have in Clang the sooner we catch the LLDB
specific importer errors.  I am asking the LLDB community's help here, because
we lack the domain knowledge in LLDB so we don't know what expectations should
we write in each unit test cases.  (About lit tests, there is clang-import-test
but its coverage seems pretty low and I don't know if that really exercises all
LLDB use cases.)

I think that from a better ASTImporter the LLDB debugger can benefit too: the
`expr` command might give better experience by supporting more C++ constructs
and by giving better diagnostics.

Thank you,
Gabor


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev