ASTImporter Tutorial/TechTalk and RoundTable

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

ASTImporter Tutorial/TechTalk and RoundTable

Kristof Beyls via cfe-dev
Hi,

I am planning to submit a talk about ASTImporter to the upcoming Dev Meeting at October 22-23, 2019.
I'd like to talk about
- the API, 
- how it is used in CTU analysis and in LLDB,
- internal subtleties and difficulties (Error handling, ExternalASTSource, ...)
The goal would be to attract more developers to use and improve ASTImporter related code, so perhaps this will be a tutorial.

Independently from the talk, I'd like to have a round table discussion if there is enough interest.
Some topics could cover future development ideas and existing problems we have.
Please get back to me if you are interested and think about the topics you have in mind, also don't forget to buy your ticket to the DevMeeting.

Thanks,
Gabor

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: ASTImporter Tutorial/TechTalk and RoundTable

Kristof Beyls via cfe-dev
Hi,

At the upcoming LLVM Dev Conf, we will have a round table discussion for ASTImporter, right after the ASTImporter Tutorial.
The time slot for the round table is Wednesday, Oct 23 2:55-4:00.
I have gathered things about possible future work and improvements, bring your own topic to discuss!

Thanks and see you at the conference,
Gabor

Big stuff
- Error handling: rollback mechanism
- Replace `StructuralEquivalency` with ODRHash
  - Step 0: StructuralEquivalency should not diagnose when called from the
    Importer, diags should be coming from the importer.
- ODRViolation handling
  - Class(Var)TemplateSpecializationDecl: Problem! We can't have more than 1 spec
    (https://reviews.llvm.org/D66999)
  - VarTemplateSpecializationDecl: ODR violation is not even detected
  - Renaming strategy
- Strategies for AccumulateChildErrors
  Clients of the ASTImporter should be able to choose an
  appropriate error handling strategy for their needs.  For instance,
  they may not want to mark an entire namespace as erroneous merely
  because there is an ODR error with two typedefs.  As another example,
  the client may allow EnumConstantDecls with same names but with
  different values in two distinct translation units.

Smaller issues/tasks/FIXMEs (techn debt)
- VisitFunctionDecl:
  - Member function templates are not handled similarly to simple fun
    specializations
  - Merge function definitions of members
  - Merge exception specifications
- Handling of inheritable attributes (https://reviews.llvm.org/D68634)
  - Use PrevDecl in `GetImportedOrCreateDecl` ?
- ObjC/ObjC++ support and stabilization
  - No test cases (not interesting for E///, we don't have objc/c++ code)
- ClassTemplateSpecializationDecl: merge instantiated default arguments,
  exceptions specifications
- Structural Eq:
  - Polluted cache of nonequivalent declarations
  - Some diagnostics are completely missing, this is misleading
- Several minor issues/fixmes with VarTemplateDecl
- Check visibility/linkage for ClassTemplateDecl, VarTemplateDecl
- Fix import of equivalent but repeated FriendDecls
- Handle redecl chain of TypeDefNameDecl
- Add Decls to their context in a unified way, and only if the "From"DC
  contains it (`AddDeclToContexts`)
- VisitVarDecl:
  - Check for ODR error if the two definitions have different initializers?
  - Diagnose ODR error if the two initializers are different
- Remove obsolate FIXMEs and TODOs
- Import default arguments of templates

On Mon, Jul 22, 2019 at 6:28 PM Gábor Márton <[hidden email]> wrote:
Hi,

I am planning to submit a talk about ASTImporter to the upcoming Dev Meeting at October 22-23, 2019.
I'd like to talk about
- the API, 
- how it is used in CTU analysis and in LLDB,
- internal subtleties and difficulties (Error handling, ExternalASTSource, ...)
The goal would be to attract more developers to use and improve ASTImporter related code, so perhaps this will be a tutorial.

Independently from the talk, I'd like to have a round table discussion if there is enough interest.
Some topics could cover future development ideas and existing problems we have.
Please get back to me if you are interested and think about the topics you have in mind, also don't forget to buy your ticket to the DevMeeting.

Thanks,
Gabor

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: ASTImporter Tutorial/TechTalk and RoundTable

Kristof Beyls via cfe-dev
Hi,

Thank you guys for coming to the round table. I am trying to summarize what we were talking about.

- Error handling: rollback mechanism
We agreed that this is something that would be incredibly problematic to implement mostly because the AST is not designed to be trimmed. In normal cases, the AST just grows, but nodes and interconnections between them are never deleted.

- Replace `StructuralEquivalency` with ODRHash
  - Step 0: StructuralEquivalency should not diagnose when called from the
    Importer, diags should be coming from the importer.
We agreed that the class StructuralEquivalenceContext is used for multiple things in the ASTImporter. These are 1) ODR violation discovery 2) identifying the existing part of a redecl chain for a Decl. Actually, ODR checking is based on tokens if we compare definitions, so structural eq is not the best thing to use for that.
We have concerns whether we could use the ODRHash, but we agreed that it would be worth to prototype it.

- ODRViolation handling
  - Class(Var)TemplateSpecializationDecl: Problem! We can't have more than 1 spec
    (https://reviews.llvm.org/D66999)
We agreed that this is fine to be conservative in case of specializations, in these cases we will always use the existing definition and will report back an error. (In case of simple classes and other non-specialization decls ,however, the liberal strategy can be used.)
  - Renaming strategy in case of ODR violation
Raphael is worried about name mangling issues that could cause.

- Strategies for AccumulateChildErrors
  Clients of the ASTImporter should be able to choose an
  appropriate error handling strategy for their needs.  For instance,
  they may not want to mark an entire namespace as erroneous merely
  because there is an ODR error with two typedefs.  As another example,
  the client may allow EnumConstantDecls with same names but with
  different values in two distinct translation units.
We agreed that this is fine to use this approach, probably there is no need to make this behavior configurable.

- VarTemplateSpecializationDecl: ODR violation is not even detected
We didn't have time to discuss this.

Smaller issues/tasks/FIXMEs (techn debt)
- VisitFunctionDecl:
  - Member function templates are not handled similarly to simple fun
    specializations
We agreed that this is a big technical debt
  - Merge function definitions of members
We discussed that out-of class default definitions can cause this situation, and at some time we should finish this.
  - Merge exception specifications

- Handling of inheritable attributes (https://reviews.llvm.org/D68634)
  - Use PrevDecl in `GetImportedOrCreateDecl` ?
We agreed that handling of attributes cannot be done with one patch because there are many different inheritable attributes and each must be handled differently.

- ObjC/ObjC++ support and stabilization
  - No test cases (not interesting for E///, we don't have objc/c++ code)
We agreed that there is a technical debt here, but we did not show much interest in fixing that.

- ClassTemplateSpecializationDecl: merge instantiated default arguments,
  exceptions specifications
We understood the problem (that took like 20 minutes and AST dumping) and recognized that the way we merge initializers of fields is good, and we should continue doing that with default arg and exceptions spec instantiations. 

We did not have time to discuss these other issues:
- Structural Eq:
  - Polluted cache of nonequivalent declarations
  - Some diagnostics are completely missing, this is misleading
- Several minor issues/fixmes with VarTemplateDecl
- Check visibility/linkage for ClassTemplateDecl, VarTemplateDecl
- Fix import of equivalent but repeated FriendDecls
- Handle redecl chain of TypeDefNameDecl
- Add Decls to their context in a unified way, and only if the "From"DC
  contains it (`AddDeclToContexts`)
- VisitVarDecl:
  - Check for ODR error if the two definitions have different initializers?
  - Diagnose ODR error if the two initializers are different
- Remove obsolate FIXMEs and TODOs
- Import default arguments of templates

Cheers,
Gabor

On Thu, Oct 17, 2019 at 4:11 PM Gábor Márton <[hidden email]> wrote:
Hi,

At the upcoming LLVM Dev Conf, we will have a round table discussion for ASTImporter, right after the ASTImporter Tutorial.
The time slot for the round table is Wednesday, Oct 23 2:55-4:00.
I have gathered things about possible future work and improvements, bring your own topic to discuss!

Thanks and see you at the conference,
Gabor

Big stuff
- Error handling: rollback mechanism
- Replace `StructuralEquivalency` with ODRHash
  - Step 0: StructuralEquivalency should not diagnose when called from the
    Importer, diags should be coming from the importer.
- ODRViolation handling
  - Class(Var)TemplateSpecializationDecl: Problem! We can't have more than 1 spec
    (https://reviews.llvm.org/D66999)
  - VarTemplateSpecializationDecl: ODR violation is not even detected
  - Renaming strategy
- Strategies for AccumulateChildErrors
  Clients of the ASTImporter should be able to choose an
  appropriate error handling strategy for their needs.  For instance,
  they may not want to mark an entire namespace as erroneous merely
  because there is an ODR error with two typedefs.  As another example,
  the client may allow EnumConstantDecls with same names but with
  different values in two distinct translation units.

Smaller issues/tasks/FIXMEs (techn debt)
- VisitFunctionDecl:
  - Member function templates are not handled similarly to simple fun
    specializations
  - Merge function definitions of members
  - Merge exception specifications
- Handling of inheritable attributes (https://reviews.llvm.org/D68634)
  - Use PrevDecl in `GetImportedOrCreateDecl` ?
- ObjC/ObjC++ support and stabilization
  - No test cases (not interesting for E///, we don't have objc/c++ code)
- ClassTemplateSpecializationDecl: merge instantiated default arguments,
  exceptions specifications
- Structural Eq:
  - Polluted cache of nonequivalent declarations
  - Some diagnostics are completely missing, this is misleading
- Several minor issues/fixmes with VarTemplateDecl
- Check visibility/linkage for ClassTemplateDecl, VarTemplateDecl
- Fix import of equivalent but repeated FriendDecls
- Handle redecl chain of TypeDefNameDecl
- Add Decls to their context in a unified way, and only if the "From"DC
  contains it (`AddDeclToContexts`)
- VisitVarDecl:
  - Check for ODR error if the two definitions have different initializers?
  - Diagnose ODR error if the two initializers are different
- Remove obsolate FIXMEs and TODOs
- Import default arguments of templates

On Mon, Jul 22, 2019 at 6:28 PM Gábor Márton <[hidden email]> wrote:
Hi,

I am planning to submit a talk about ASTImporter to the upcoming Dev Meeting at October 22-23, 2019.
I'd like to talk about
- the API, 
- how it is used in CTU analysis and in LLDB,
- internal subtleties and difficulties (Error handling, ExternalASTSource, ...)
The goal would be to attract more developers to use and improve ASTImporter related code, so perhaps this will be a tutorial.

Independently from the talk, I'd like to have a round table discussion if there is enough interest.
Some topics could cover future development ideas and existing problems we have.
Please get back to me if you are interested and think about the topics you have in mind, also don't forget to buy your ticket to the DevMeeting.

Thanks,
Gabor

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev