Wrong order of AST traversal?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Wrong order of AST traversal?

Nathan Ridge via cfe-dev
Hello @ all,

I want to annotate some function and generate some additional properties for the annotated functions in an xml document. My problem is, that the AST traversal seems to traverse the attributes of the function after the traversal of the function has been finished. I could not find any valuable resources on this topic by searching via google. So I am addressing my question here.

(Since I do not know if file attachments are allowed I provide the files as comments.)

My test files is: (attributeTest.cpp)

void func1()
__attribute((annotate(R"(Test)"))) {
  int a = 1 + 1;
}

clang-check gives me the correct hierarchy: clang-check --ast-dump attributeTest.cpp
TranslationUnitDecl 0x559b4c7dc598 <<invalid sloc>> <invalid sloc>
|-TypedefDecl 0x559b4c7dce70 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
| `-BuiltinType 0x559b4c7dcb30 '__int128'
|-TypedefDecl 0x559b4c7dced8 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
| `-BuiltinType 0x559b4c7dcb50 'unsigned __int128'
|-TypedefDecl 0x559b4c7dd208 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
| `-RecordType 0x559b4c7dcfb0 '__NSConstantString_tag'
|   `-CXXRecord 0x559b4c7dcf28 '__NSConstantString_tag'
|-TypedefDecl 0x559b4c7dd2a0 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
| `-PointerType 0x559b4c7dd260 'char *'
|   `-BuiltinType 0x559b4c7dc630 'char'
|-TypedefDecl 0x559b4c818958 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag [1]'
| `-ConstantArrayType 0x559b4c818900 '__va_list_tag [1]' 1 
|   `-RecordType 0x559b4c7dd380 '__va_list_tag'
|     `-CXXRecord 0x559b4c7dd2f0 '__va_list_tag'
`-FunctionDecl 0x559b4c818a98 </home/msagebaum/Kaiserslautern/Programms/adDSLParser/temp/attributeTest.cpp:1:1, line:4:1> line:1:6 func1 'void ()'
  |-CompoundStmt 0x559b4c818cb0 <line:2:36, line:4:1>
  | `-DeclStmt 0x559b4c818c98 <line:3:3, col:16>
  |   `-VarDecl 0x559b4c818bd8 <col:3, col:15> col:7 a 'int' cinit
  |     `-BinaryOperator 0x559b4c818c78 <col:11, col:15> 'int' '+'
  |       |-IntegerLiteral 0x559b4c818c38 <col:11> 'int' 1
  |       `-IntegerLiteral 0x559b4c818c58 <col:15> 'int' 1
  `-AnnotateAttr 0x559b4c818b30 <line:2:14, col:32> "Test"

If I now run the file through my AST Visitor: (parserTest.cpp)

#include <iostream>

#include "clang/AST/ASTConsumer.h"
#include "clang/AST/QualTypeNames.h"
#include "clang/AST/RecursiveASTVisitor.h"
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Frontend/FrontendAction.h"
#include "clang/Tooling/CommonOptionsParser.h"
#include "clang/Tooling/Tooling.h"

#include <llvm/Support/CommandLine.h>

using namespace llvm;
using namespace clang;
using namespace clang::tooling;

class XMLGenerator : public RecursiveASTVisitor<XMLGenerator> {
public:

  bool TraverseFunctionDecl(FunctionDecl *declaration) {

    std::cout << "Function start" << std::endl;

    RecursiveASTVisitor<XMLGenerator>::TraverseFunctionDecl(declaration);

    std::cout << "Function end" << std::endl;
    return true;
  }

  bool TraverseAnnotateAttr(AnnotateAttr *a) {
    std::cout << "Annotation start" << std::endl;

    RecursiveASTVisitor<XMLGenerator>::TraverseAnnotateAttr(a);

    std::cout << "Annotation end" << std::endl;
    return true;
  }
};

class XMLGeneratorClassConsumer : public clang::ASTConsumer {
public:
  explicit XMLGeneratorClassConsumer()
    : Visitor() {}

  virtual void HandleTranslationUnit(clang::ASTContext &Context) {
    Visitor.TraverseDecl(Context.getTranslationUnitDecl());
  }
private:
  XMLGenerator Visitor;
};

class XMLGeneratorClassAction : public clang::ASTFrontendAction {
public:
  virtual std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(
    clang::CompilerInstance &Compiler, llvm::StringRef InFile) {
    return std::unique_ptr<clang::ASTConsumer>(
        new XMLGeneratorClassConsumer());
  }
};

static llvm::cl::OptionCategory MyToolCategory("my-tool options");

int main(int argc, const char **argv) {
  CommonOptionsParser OptionsParser(argc, argv, MyToolCategory);
  ClangTool Tool(OptionsParser.getCompilations(),
                 OptionsParser.getSourcePathList());

  return Tool.run(newFrontendActionFactory<XMLGeneratorClassAction>().get());
}

Compilation: (compile.sh)
g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -c parserTest.cpp -o parserTest.o -I/usr/include
g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -o parserTest.exe parserTest.o -L/usr/lib64  -Wl,--start-group -lclangAnalysis -lclangParse -lclangEdit -lclangSema -lclangTooling -lclangDriver -lclangSerialization -lclangFrontend -lclangBasic -lclangASTMatchers -lclangAST -lclangLex -lLLVM-8 -Wl,--end-group -lpthread -lz

The result is:
Function start
Function end
Annotation start
Annotation end

What I would expect is:
Function start
Annotation start
Annotation end
Function end

My questions are now:
Is the order provided by my implementation the correct one or is the expected order the correct one?
Is there a way to change the visitor order such that attributes are visited inside of the function?
If not: Is there a way to get a list of attributes from 'FunctionDecl'? (Could not find any methods in the api https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)

Thanks in advance for any help.

Cheers

Max

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wrong order of AST traversal?

Nathan Ridge via cfe-dev
Hi Max,

You can get the attributes with Decl::attrs() method (also available for FunctionDecl).

On Thu, Jul 18, 2019 at 9:28 AM Max Sagebaum via cfe-dev <[hidden email]> wrote:
Hello @ all,

I want to annotate some function and generate some additional properties for the annotated functions in an xml document. My problem is, that the AST traversal seems to traverse the attributes of the function after the traversal of the function has been finished. I could not find any valuable resources on this topic by searching via google. So I am addressing my question here.

(Since I do not know if file attachments are allowed I provide the files as comments.)

My test files is: (attributeTest.cpp)

void func1()
__attribute((annotate(R"(Test)"))) {
  int a = 1 + 1;
}

clang-check gives me the correct hierarchy: clang-check --ast-dump attributeTest.cpp
TranslationUnitDecl 0x559b4c7dc598 <<invalid sloc>> <invalid sloc>
|-TypedefDecl 0x559b4c7dce70 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
| `-BuiltinType 0x559b4c7dcb30 '__int128'
|-TypedefDecl 0x559b4c7dced8 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
| `-BuiltinType 0x559b4c7dcb50 'unsigned __int128'
|-TypedefDecl 0x559b4c7dd208 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
| `-RecordType 0x559b4c7dcfb0 '__NSConstantString_tag'
|   `-CXXRecord 0x559b4c7dcf28 '__NSConstantString_tag'
|-TypedefDecl 0x559b4c7dd2a0 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
| `-PointerType 0x559b4c7dd260 'char *'
|   `-BuiltinType 0x559b4c7dc630 'char'
|-TypedefDecl 0x559b4c818958 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag [1]'
| `-ConstantArrayType 0x559b4c818900 '__va_list_tag [1]' 1 
|   `-RecordType 0x559b4c7dd380 '__va_list_tag'
|     `-CXXRecord 0x559b4c7dd2f0 '__va_list_tag'
`-FunctionDecl 0x559b4c818a98 </home/msagebaum/Kaiserslautern/Programms/adDSLParser/temp/attributeTest.cpp:1:1, line:4:1> line:1:6 func1 'void ()'
  |-CompoundStmt 0x559b4c818cb0 <line:2:36, line:4:1>
  | `-DeclStmt 0x559b4c818c98 <line:3:3, col:16>
  |   `-VarDecl 0x559b4c818bd8 <col:3, col:15> col:7 a 'int' cinit
  |     `-BinaryOperator 0x559b4c818c78 <col:11, col:15> 'int' '+'
  |       |-IntegerLiteral 0x559b4c818c38 <col:11> 'int' 1
  |       `-IntegerLiteral 0x559b4c818c58 <col:15> 'int' 1
  `-AnnotateAttr 0x559b4c818b30 <line:2:14, col:32> "Test"

If I now run the file through my AST Visitor: (parserTest.cpp)

#include <iostream>

#include "clang/AST/ASTConsumer.h"
#include "clang/AST/QualTypeNames.h"
#include "clang/AST/RecursiveASTVisitor.h"
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Frontend/FrontendAction.h"
#include "clang/Tooling/CommonOptionsParser.h"
#include "clang/Tooling/Tooling.h"

#include <llvm/Support/CommandLine.h>

using namespace llvm;
using namespace clang;
using namespace clang::tooling;

class XMLGenerator : public RecursiveASTVisitor<XMLGenerator> {
public:

  bool TraverseFunctionDecl(FunctionDecl *declaration) {

    std::cout << "Function start" << std::endl;

    RecursiveASTVisitor<XMLGenerator>::TraverseFunctionDecl(declaration);

    std::cout << "Function end" << std::endl;
    return true;
  }

  bool TraverseAnnotateAttr(AnnotateAttr *a) {
    std::cout << "Annotation start" << std::endl;

    RecursiveASTVisitor<XMLGenerator>::TraverseAnnotateAttr(a);

    std::cout << "Annotation end" << std::endl;
    return true;
  }
};

class XMLGeneratorClassConsumer : public clang::ASTConsumer {
public:
  explicit XMLGeneratorClassConsumer()
    : Visitor() {}

  virtual void HandleTranslationUnit(clang::ASTContext &Context) {
    Visitor.TraverseDecl(Context.getTranslationUnitDecl());
  }
private:
  XMLGenerator Visitor;
};

class XMLGeneratorClassAction : public clang::ASTFrontendAction {
public:
  virtual std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(
    clang::CompilerInstance &Compiler, llvm::StringRef InFile) {
    return std::unique_ptr<clang::ASTConsumer>(
        new XMLGeneratorClassConsumer());
  }
};

static llvm::cl::OptionCategory MyToolCategory("my-tool options");

int main(int argc, const char **argv) {
  CommonOptionsParser OptionsParser(argc, argv, MyToolCategory);
  ClangTool Tool(OptionsParser.getCompilations(),
                 OptionsParser.getSourcePathList());

  return Tool.run(newFrontendActionFactory<XMLGeneratorClassAction>().get());
}

Compilation: (compile.sh)
g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -c parserTest.cpp -o parserTest.o -I/usr/include
g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -o parserTest.exe parserTest.o -L/usr/lib64  -Wl,--start-group -lclangAnalysis -lclangParse -lclangEdit -lclangSema -lclangTooling -lclangDriver -lclangSerialization -lclangFrontend -lclangBasic -lclangASTMatchers -lclangAST -lclangLex -lLLVM-8 -Wl,--end-group -lpthread -lz

The result is:
Function start
Function end
Annotation start
Annotation end

What I would expect is:
Function start
Annotation start
Annotation end
Function end

My questions are now:
Is the order provided by my implementation the correct one or is the expected order the correct one?
Is there a way to change the visitor order such that attributes are visited inside of the function?
If not: Is there a way to get a list of attributes from 'FunctionDecl'? (Could not find any methods in the api https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)

Thanks in advance for any help.

Cheers

Max
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


--
Regards,
Ilya Biryukov

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wrong order of AST traversal?

Nathan Ridge via cfe-dev
Hello Ilya,

thank you for the fast reply. I do not know why I missed that.

Nevertheless, is the traversal order correct?

Cheers

Max

On Thu, 2019-07-18 at 09:40 +0200, Ilya Biryukov wrote:
Hi Max,

You can get the attributes with Decl::attrs() method (also available for FunctionDecl).

On Thu, Jul 18, 2019 at 9:28 AM Max Sagebaum via cfe-dev <[hidden email]> wrote:
Hello @ all,

I want to annotate some function and generate some additional properties for the annotated functions in an xml document. My problem is, that the AST traversal seems to traverse the attributes of the function after the traversal of the function has been finished. I could not find any valuable resources on this topic by searching via google. So I am addressing my question here.

(Since I do not know if file attachments are allowed I provide the files as comments.)

My test files is: (attributeTest.cpp)

void func1()
__attribute((annotate(R"(Test)"))) {
  int a = 1 + 1;
}

clang-check gives me the correct hierarchy: clang-check --ast-dump attributeTest.cpp
TranslationUnitDecl 0x559b4c7dc598 <<invalid sloc>> <invalid sloc>
|-TypedefDecl 0x559b4c7dce70 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
| `-BuiltinType 0x559b4c7dcb30 '__int128'
|-TypedefDecl 0x559b4c7dced8 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
| `-BuiltinType 0x559b4c7dcb50 'unsigned __int128'
|-TypedefDecl 0x559b4c7dd208 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
| `-RecordType 0x559b4c7dcfb0 '__NSConstantString_tag'
|   `-CXXRecord 0x559b4c7dcf28 '__NSConstantString_tag'
|-TypedefDecl 0x559b4c7dd2a0 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
| `-PointerType 0x559b4c7dd260 'char *'
|   `-BuiltinType 0x559b4c7dc630 'char'
|-TypedefDecl 0x559b4c818958 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag [1]'
| `-ConstantArrayType 0x559b4c818900 '__va_list_tag [1]' 1 
|   `-RecordType 0x559b4c7dd380 '__va_list_tag'
|     `-CXXRecord 0x559b4c7dd2f0 '__va_list_tag'
`-FunctionDecl 0x559b4c818a98 </home/msagebaum/Kaiserslautern/Programms/adDSLParser/temp/attributeTest.cpp:1:1, line:4:1> line:1:6 func1 'void ()'
  |-CompoundStmt 0x559b4c818cb0 <line:2:36, line:4:1>
  | `-DeclStmt 0x559b4c818c98 <line:3:3, col:16>
  |   `-VarDecl 0x559b4c818bd8 <col:3, col:15> col:7 a 'int' cinit
  |     `-BinaryOperator 0x559b4c818c78 <col:11, col:15> 'int' '+'
  |       |-IntegerLiteral 0x559b4c818c38 <col:11> 'int' 1
  |       `-IntegerLiteral 0x559b4c818c58 <col:15> 'int' 1
  `-AnnotateAttr 0x559b4c818b30 <line:2:14, col:32> "Test"

If I now run the file through my AST Visitor: (parserTest.cpp)

#include <iostream>

#include "clang/AST/ASTConsumer.h"
#include "clang/AST/QualTypeNames.h"
#include "clang/AST/RecursiveASTVisitor.h"
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Frontend/FrontendAction.h"
#include "clang/Tooling/CommonOptionsParser.h"
#include "clang/Tooling/Tooling.h"

#include <llvm/Support/CommandLine.h>

using namespace llvm;
using namespace clang;
using namespace clang::tooling;

class XMLGenerator : public RecursiveASTVisitor<XMLGenerator> {
public:

  bool TraverseFunctionDecl(FunctionDecl *declaration) {

    std::cout << "Function start" << std::endl;

    RecursiveASTVisitor<XMLGenerator>::TraverseFunctionDecl(declaration);

    std::cout << "Function end" << std::endl;
    return true;
  }

  bool TraverseAnnotateAttr(AnnotateAttr *a) {
    std::cout << "Annotation start" << std::endl;

    RecursiveASTVisitor<XMLGenerator>::TraverseAnnotateAttr(a);

    std::cout << "Annotation end" << std::endl;
    return true;
  }
};

class XMLGeneratorClassConsumer : public clang::ASTConsumer {
public:
  explicit XMLGeneratorClassConsumer()
    : Visitor() {}

  virtual void HandleTranslationUnit(clang::ASTContext &Context) {
    Visitor.TraverseDecl(Context.getTranslationUnitDecl());
  }
private:
  XMLGenerator Visitor;
};

class XMLGeneratorClassAction : public clang::ASTFrontendAction {
public:
  virtual std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(
    clang::CompilerInstance &Compiler, llvm::StringRef InFile) {
    return std::unique_ptr<clang::ASTConsumer>(
        new XMLGeneratorClassConsumer());
  }
};

static llvm::cl::OptionCategory MyToolCategory("my-tool options");

int main(int argc, const char **argv) {
  CommonOptionsParser OptionsParser(argc, argv, MyToolCategory);
  ClangTool Tool(OptionsParser.getCompilations(),
                 OptionsParser.getSourcePathList());

  return Tool.run(newFrontendActionFactory<XMLGeneratorClassAction>().get());
}

Compilation: (compile.sh)
g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -c parserTest.cpp -o parserTest.o -I/usr/include
g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -o parserTest.exe parserTest.o -L/usr/lib64  -Wl,--start-group -lclangAnalysis -lclangParse -lclangEdit -lclangSema -lclangTooling -lclangDriver -lclangSerialization -lclangFrontend -lclangBasic -lclangASTMatchers -lclangAST -lclangLex -lLLVM-8 -Wl,--end-group -lpthread -lz

The result is:
Function start
Function end
Annotation start
Annotation end

What I would expect is:
Function start
Annotation start
Annotation end
Function end

My questions are now:
Is the order provided by my implementation the correct one or is the expected order the correct one?
Is there a way to change the visitor order such that attributes are visited inside of the function?
If not: Is there a way to get a list of attributes from 'FunctionDecl'? (Could not find any methods in the api https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)

Thanks in advance for any help.

Cheers

Max
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


-- 
Dr. Max Sagebaum

Chair for Scientific Computing,
TU Kaiserslautern,
Bldg/Geb 34, Paul-Ehrlich-Strasse,
67663 Kaiserslautern, Germany

Phone: +49 (0)631 205 5638
Fax:   +49 (0)631 205 3056







_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wrong order of AST traversal?

Nathan Ridge via cfe-dev

I believe it is intentional to have that traversal order.  If we were to visit the attributes before the functiondecl itself, they would be meaningless without the context of where they are attached (Which *Attr doesn’t contain). 

 

The argument that COULD be made is that it makes more sense to be traversed before the function body however.

 

From: cfe-dev [mailto:[hidden email]] On Behalf Of Max Sagebaum via cfe-dev
Sent: Thursday, July 18, 2019 1:36 AM
To: [hidden email]
Cc: [hidden email]
Subject: Re: [cfe-dev] Wrong order of AST traversal?

 

Hello Ilya,

 

thank you for the fast reply. I do not know why I missed that.

 

Nevertheless, is the traversal order correct?

 

Cheers

 

Max

 

On Thu, 2019-07-18 at 09:40 +0200, Ilya Biryukov wrote:

Hi Max,

 

You can get the attributes with Decl::attrs() method (also available for FunctionDecl).

 

On Thu, Jul 18, 2019 at 9:28 AM Max Sagebaum via cfe-dev <[hidden email]> wrote:

Hello @ all,

 

I want to annotate some function and generate some additional properties for the annotated functions in an xml document. My problem is, that the AST traversal seems to traverse the attributes of the function after the traversal of the function has been finished. I could not find any valuable resources on this topic by searching via google. So I am addressing my question here.

 

(Since I do not know if file attachments are allowed I provide the files as comments.)

 

My test files is: (attributeTest.cpp)

 

void func1()

__attribute((annotate(R"(Test)"))) {

  int a = 1 + 1;

}

 

clang-check gives me the correct hierarchy: clang-check --ast-dump attributeTest.cpp

TranslationUnitDecl 0x559b4c7dc598 <<invalid sloc>> <invalid sloc>

|-TypedefDecl 0x559b4c7dce70 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'

| `-BuiltinType 0x559b4c7dcb30 '__int128'

|-TypedefDecl 0x559b4c7dced8 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'

| `-BuiltinType 0x559b4c7dcb50 'unsigned __int128'

|-TypedefDecl 0x559b4c7dd208 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'

| `-RecordType 0x559b4c7dcfb0 '__NSConstantString_tag'

|   `-CXXRecord 0x559b4c7dcf28 '__NSConstantString_tag'

|-TypedefDecl 0x559b4c7dd2a0 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'

| `-PointerType 0x559b4c7dd260 'char *'

|   `-BuiltinType 0x559b4c7dc630 'char'

|-TypedefDecl 0x559b4c818958 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag [1]'

| `-ConstantArrayType 0x559b4c818900 '__va_list_tag [1]' 1 

|   `-RecordType 0x559b4c7dd380 '__va_list_tag'

|     `-CXXRecord 0x559b4c7dd2f0 '__va_list_tag'

`-FunctionDecl 0x559b4c818a98 </home/msagebaum/Kaiserslautern/Programms/adDSLParser/temp/attributeTest.cpp:1:1, line:4:1> line:1:6 func1 'void ()'

  |-CompoundStmt 0x559b4c818cb0 <line:2:36, line:4:1>

  | `-DeclStmt 0x559b4c818c98 <line:3:3, col:16>

  |   `-VarDecl 0x559b4c818bd8 <col:3, col:15> col:7 a 'int' cinit

  |     `-BinaryOperator 0x559b4c818c78 <col:11, col:15> 'int' '+'

  |       |-IntegerLiteral 0x559b4c818c38 <col:11> 'int' 1

  |       `-IntegerLiteral 0x559b4c818c58 <col:15> 'int' 1

  `-AnnotateAttr 0x559b4c818b30 <line:2:14, col:32> "Test"

 

If I now run the file through my AST Visitor: (parserTest.cpp)

 

#include <iostream>

 

#include "clang/AST/ASTConsumer.h"

#include "clang/AST/QualTypeNames.h"

#include "clang/AST/RecursiveASTVisitor.h"

#include "clang/Frontend/CompilerInstance.h"

#include "clang/Frontend/FrontendAction.h"

#include "clang/Tooling/CommonOptionsParser.h"

#include "clang/Tooling/Tooling.h"

 

#include <llvm/Support/CommandLine.h>

 

using namespace llvm;

using namespace clang;

using namespace clang::tooling;

 

class XMLGenerator : public RecursiveASTVisitor<XMLGenerator> {

public:

 

  bool TraverseFunctionDecl(FunctionDecl *declaration) {

 

    std::cout << "Function start" << std::endl;

 

    RecursiveASTVisitor<XMLGenerator>::TraverseFunctionDecl(declaration);

 

    std::cout << "Function end" << std::endl;

    return true;

  }

 

  bool TraverseAnnotateAttr(AnnotateAttr *a) {

    std::cout << "Annotation start" << std::endl;

 

    RecursiveASTVisitor<XMLGenerator>::TraverseAnnotateAttr(a);

 

    std::cout << "Annotation end" << std::endl;

    return true;

  }

};

 

class XMLGeneratorClassConsumer : public clang::ASTConsumer {

public:

  explicit XMLGeneratorClassConsumer()

    : Visitor() {}

 

  virtual void HandleTranslationUnit(clang::ASTContext &Context) {

    Visitor.TraverseDecl(Context.getTranslationUnitDecl());

  }

private:

  XMLGenerator Visitor;

};

 

class XMLGeneratorClassAction : public clang::ASTFrontendAction {

public:

  virtual std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(

    clang::CompilerInstance &Compiler, llvm::StringRef InFile) {

    return std::unique_ptr<clang::ASTConsumer>(

        new XMLGeneratorClassConsumer());

  }

};

 

static llvm::cl::OptionCategory MyToolCategory("my-tool options");

 

int main(int argc, const char **argv) {

  CommonOptionsParser OptionsParser(argc, argv, MyToolCategory);

  ClangTool Tool(OptionsParser.getCompilations(),

                 OptionsParser.getSourcePathList());

 

  return Tool.run(newFrontendActionFactory<XMLGeneratorClassAction>().get());

}

 

Compilation: (compile.sh)

g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -c parserTest.cpp -o parserTest.o -I/usr/include

g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -o parserTest.exe parserTest.o -L/usr/lib64  -Wl,--start-group -lclangAnalysis -lclangParse -lclangEdit -lclangSema -lclangTooling -lclangDriver -lclangSerialization -lclangFrontend -lclangBasic -lclangASTMatchers -lclangAST -lclangLex -lLLVM-8 -Wl,--end-group -lpthread -lz

 

The result is:

Function start

Function end

Annotation start

Annotation end

 

What I would expect is:

Function start

Annotation start

Annotation end

Function end

 

My questions are now:

Is the order provided by my implementation the correct one or is the expected order the correct one?

Is there a way to change the visitor order such that attributes are visited inside of the function?

If not: Is there a way to get a list of attributes from 'FunctionDecl'? (Could not find any methods in the api https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)

 

Thanks in advance for any help.

 

Cheers

 

Max

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


 

-- 

Dr. Max Sagebaum

 

Chair for Scientific Computing,

TU Kaiserslautern,

Bldg/Geb 34, Paul-Ehrlich-Strasse,

67663 Kaiserslautern, Germany

 

Phone: +49 (0)631 205 5638

Fax:   +49 (0)631 205 3056

 

 

 

 

 

 


_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wrong order of AST traversal?

Nathan Ridge via cfe-dev
In reply to this post by Nathan Ridge via cfe-dev
I would agree the traversal order is wrong, attributes are logically attached to a function, i.e. they are "child nodes".
Therefore, they should be traversed alongside others child nodes (function body, type, etc).


On Thu, Jul 18, 2019 at 10:35 AM Max Sagebaum <[hidden email]> wrote:
Hello Ilya,

thank you for the fast reply. I do not know why I missed that.

Nevertheless, is the traversal order correct?

Cheers

Max

On Thu, 2019-07-18 at 09:40 +0200, Ilya Biryukov wrote:
Hi Max,

You can get the attributes with Decl::attrs() method (also available for FunctionDecl).

On Thu, Jul 18, 2019 at 9:28 AM Max Sagebaum via cfe-dev <[hidden email]> wrote:
Hello @ all,

I want to annotate some function and generate some additional properties for the annotated functions in an xml document. My problem is, that the AST traversal seems to traverse the attributes of the function after the traversal of the function has been finished. I could not find any valuable resources on this topic by searching via google. So I am addressing my question here.

(Since I do not know if file attachments are allowed I provide the files as comments.)

My test files is: (attributeTest.cpp)

void func1()
__attribute((annotate(R"(Test)"))) {
  int a = 1 + 1;
}

clang-check gives me the correct hierarchy: clang-check --ast-dump attributeTest.cpp
TranslationUnitDecl 0x559b4c7dc598 <<invalid sloc>> <invalid sloc>
|-TypedefDecl 0x559b4c7dce70 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
| `-BuiltinType 0x559b4c7dcb30 '__int128'
|-TypedefDecl 0x559b4c7dced8 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
| `-BuiltinType 0x559b4c7dcb50 'unsigned __int128'
|-TypedefDecl 0x559b4c7dd208 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
| `-RecordType 0x559b4c7dcfb0 '__NSConstantString_tag'
|   `-CXXRecord 0x559b4c7dcf28 '__NSConstantString_tag'
|-TypedefDecl 0x559b4c7dd2a0 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
| `-PointerType 0x559b4c7dd260 'char *'
|   `-BuiltinType 0x559b4c7dc630 'char'
|-TypedefDecl 0x559b4c818958 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag [1]'
| `-ConstantArrayType 0x559b4c818900 '__va_list_tag [1]' 1 
|   `-RecordType 0x559b4c7dd380 '__va_list_tag'
|     `-CXXRecord 0x559b4c7dd2f0 '__va_list_tag'
`-FunctionDecl 0x559b4c818a98 </home/msagebaum/Kaiserslautern/Programms/adDSLParser/temp/attributeTest.cpp:1:1, line:4:1> line:1:6 func1 'void ()'
  |-CompoundStmt 0x559b4c818cb0 <line:2:36, line:4:1>
  | `-DeclStmt 0x559b4c818c98 <line:3:3, col:16>
  |   `-VarDecl 0x559b4c818bd8 <col:3, col:15> col:7 a 'int' cinit
  |     `-BinaryOperator 0x559b4c818c78 <col:11, col:15> 'int' '+'
  |       |-IntegerLiteral 0x559b4c818c38 <col:11> 'int' 1
  |       `-IntegerLiteral 0x559b4c818c58 <col:15> 'int' 1
  `-AnnotateAttr 0x559b4c818b30 <line:2:14, col:32> "Test"

If I now run the file through my AST Visitor: (parserTest.cpp)

#include <iostream>

#include "clang/AST/ASTConsumer.h"
#include "clang/AST/QualTypeNames.h"
#include "clang/AST/RecursiveASTVisitor.h"
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Frontend/FrontendAction.h"
#include "clang/Tooling/CommonOptionsParser.h"
#include "clang/Tooling/Tooling.h"

#include <llvm/Support/CommandLine.h>

using namespace llvm;
using namespace clang;
using namespace clang::tooling;

class XMLGenerator : public RecursiveASTVisitor<XMLGenerator> {
public:

  bool TraverseFunctionDecl(FunctionDecl *declaration) {

    std::cout << "Function start" << std::endl;

    RecursiveASTVisitor<XMLGenerator>::TraverseFunctionDecl(declaration);

    std::cout << "Function end" << std::endl;
    return true;
  }

  bool TraverseAnnotateAttr(AnnotateAttr *a) {
    std::cout << "Annotation start" << std::endl;

    RecursiveASTVisitor<XMLGenerator>::TraverseAnnotateAttr(a);

    std::cout << "Annotation end" << std::endl;
    return true;
  }
};

class XMLGeneratorClassConsumer : public clang::ASTConsumer {
public:
  explicit XMLGeneratorClassConsumer()
    : Visitor() {}

  virtual void HandleTranslationUnit(clang::ASTContext &Context) {
    Visitor.TraverseDecl(Context.getTranslationUnitDecl());
  }
private:
  XMLGenerator Visitor;
};

class XMLGeneratorClassAction : public clang::ASTFrontendAction {
public:
  virtual std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(
    clang::CompilerInstance &Compiler, llvm::StringRef InFile) {
    return std::unique_ptr<clang::ASTConsumer>(
        new XMLGeneratorClassConsumer());
  }
};

static llvm::cl::OptionCategory MyToolCategory("my-tool options");

int main(int argc, const char **argv) {
  CommonOptionsParser OptionsParser(argc, argv, MyToolCategory);
  ClangTool Tool(OptionsParser.getCompilations(),
                 OptionsParser.getSourcePathList());

  return Tool.run(newFrontendActionFactory<XMLGeneratorClassAction>().get());
}

Compilation: (compile.sh)
g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -c parserTest.cpp -o parserTest.o -I/usr/include
g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -o parserTest.exe parserTest.o -L/usr/lib64  -Wl,--start-group -lclangAnalysis -lclangParse -lclangEdit -lclangSema -lclangTooling -lclangDriver -lclangSerialization -lclangFrontend -lclangBasic -lclangASTMatchers -lclangAST -lclangLex -lLLVM-8 -Wl,--end-group -lpthread -lz

The result is:
Function start
Function end
Annotation start
Annotation end

What I would expect is:
Function start
Annotation start
Annotation end
Function end

My questions are now:
Is the order provided by my implementation the correct one or is the expected order the correct one?
Is there a way to change the visitor order such that attributes are visited inside of the function?
If not: Is there a way to get a list of attributes from 'FunctionDecl'? (Could not find any methods in the api https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)

Thanks in advance for any help.

Cheers

Max
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


-- 
Dr. Max Sagebaum

Chair for Scientific Computing,
TU Kaiserslautern,
Bldg/Geb 34, Paul-Ehrlich-Strasse,
67663 Kaiserslautern, Germany

Phone: +49 (0)631 205 5638
Fax:   +49 (0)631 205 3056








--
Regards,
Ilya Biryukov

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wrong order of AST traversal?

Nathan Ridge via cfe-dev
I looked quickly at RecursiveASTVisitor.h, and it looks like the
challenge is that attributes can be attached to other decls as well.

Attrs are traversed generically for all Decl after any dynamic-type traversal:
https://github.com/llvm/llvm-project/blob/master/clang/include/clang/AST/RecursiveASTVisitor.h#L707

So I think in order to change this, attribute traversal would need to
be custom-implemented for every kind of Decl.

- Kim

On Thu, Jul 18, 2019 at 10:54 AM Ilya Biryukov via cfe-dev
<[hidden email]> wrote:

>
> I would agree the traversal order is wrong, attributes are logically attached to a function, i.e. they are "child nodes".
> Therefore, they should be traversed alongside others child nodes (function body, type, etc).
>
>
> On Thu, Jul 18, 2019 at 10:35 AM Max Sagebaum <[hidden email]> wrote:
>>
>> Hello Ilya,
>>
>> thank you for the fast reply. I do not know why I missed that.
>>
>> Nevertheless, is the traversal order correct?
>>
>> Cheers
>>
>> Max
>>
>> On Thu, 2019-07-18 at 09:40 +0200, Ilya Biryukov wrote:
>>
>> Hi Max,
>>
>> You can get the attributes with Decl::attrs() method (also available for FunctionDecl).
>>
>> On Thu, Jul 18, 2019 at 9:28 AM Max Sagebaum via cfe-dev <[hidden email]> wrote:
>>
>> Hello @ all,
>>
>> I want to annotate some function and generate some additional properties for the annotated functions in an xml document. My problem is, that the AST traversal seems to traverse the attributes of the function after the traversal of the function has been finished. I could not find any valuable resources on this topic by searching via google. So I am addressing my question here.
>>
>> (Since I do not know if file attachments are allowed I provide the files as comments.)
>>
>> My test files is: (attributeTest.cpp)
>>
>> void func1()
>> __attribute((annotate(R"(Test)"))) {
>>   int a = 1 + 1;
>> }
>>
>>
>> clang-check gives me the correct hierarchy: clang-check --ast-dump attributeTest.cpp
>>
>> TranslationUnitDecl 0x559b4c7dc598 <<invalid sloc>> <invalid sloc>
>> |-TypedefDecl 0x559b4c7dce70 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
>> | `-BuiltinType 0x559b4c7dcb30 '__int128'
>> |-TypedefDecl 0x559b4c7dced8 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
>> | `-BuiltinType 0x559b4c7dcb50 'unsigned __int128'
>> |-TypedefDecl 0x559b4c7dd208 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
>> | `-RecordType 0x559b4c7dcfb0 '__NSConstantString_tag'
>> |   `-CXXRecord 0x559b4c7dcf28 '__NSConstantString_tag'
>> |-TypedefDecl 0x559b4c7dd2a0 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
>> | `-PointerType 0x559b4c7dd260 'char *'
>> |   `-BuiltinType 0x559b4c7dc630 'char'
>> |-TypedefDecl 0x559b4c818958 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag [1]'
>> | `-ConstantArrayType 0x559b4c818900 '__va_list_tag [1]' 1
>> |   `-RecordType 0x559b4c7dd380 '__va_list_tag'
>> |     `-CXXRecord 0x559b4c7dd2f0 '__va_list_tag'
>> `-FunctionDecl 0x559b4c818a98 </home/msagebaum/Kaiserslautern/Programms/adDSLParser/temp/attributeTest.cpp:1:1, line:4:1> line:1:6 func1 'void ()'
>>   |-CompoundStmt 0x559b4c818cb0 <line:2:36, line:4:1>
>>   | `-DeclStmt 0x559b4c818c98 <line:3:3, col:16>
>>   |   `-VarDecl 0x559b4c818bd8 <col:3, col:15> col:7 a 'int' cinit
>>   |     `-BinaryOperator 0x559b4c818c78 <col:11, col:15> 'int' '+'
>>   |       |-IntegerLiteral 0x559b4c818c38 <col:11> 'int' 1
>>   |       `-IntegerLiteral 0x559b4c818c58 <col:15> 'int' 1
>>   `-AnnotateAttr 0x559b4c818b30 <line:2:14, col:32> "Test"
>>
>>
>> If I now run the file through my AST Visitor: (parserTest.cpp)
>>
>> #include <iostream>
>>
>> #include "clang/AST/ASTConsumer.h"
>> #include "clang/AST/QualTypeNames.h"
>> #include "clang/AST/RecursiveASTVisitor.h"
>> #include "clang/Frontend/CompilerInstance.h"
>> #include "clang/Frontend/FrontendAction.h"
>> #include "clang/Tooling/CommonOptionsParser.h"
>> #include "clang/Tooling/Tooling.h"
>>
>> #include <llvm/Support/CommandLine.h>
>>
>> using namespace llvm;
>> using namespace clang;
>> using namespace clang::tooling;
>>
>> class XMLGenerator : public RecursiveASTVisitor<XMLGenerator> {
>> public:
>>
>>   bool TraverseFunctionDecl(FunctionDecl *declaration) {
>>
>>     std::cout << "Function start" << std::endl;
>>
>>     RecursiveASTVisitor<XMLGenerator>::TraverseFunctionDecl(declaration);
>>
>>     std::cout << "Function end" << std::endl;
>>     return true;
>>   }
>>
>>   bool TraverseAnnotateAttr(AnnotateAttr *a) {
>>     std::cout << "Annotation start" << std::endl;
>>
>>     RecursiveASTVisitor<XMLGenerator>::TraverseAnnotateAttr(a);
>>
>>     std::cout << "Annotation end" << std::endl;
>>     return true;
>>   }
>> };
>>
>> class XMLGeneratorClassConsumer : public clang::ASTConsumer {
>> public:
>>   explicit XMLGeneratorClassConsumer()
>>     : Visitor() {}
>>
>>   virtual void HandleTranslationUnit(clang::ASTContext &Context) {
>>     Visitor.TraverseDecl(Context.getTranslationUnitDecl());
>>   }
>> private:
>>   XMLGenerator Visitor;
>> };
>>
>> class XMLGeneratorClassAction : public clang::ASTFrontendAction {
>> public:
>>   virtual std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(
>>     clang::CompilerInstance &Compiler, llvm::StringRef InFile) {
>>     return std::unique_ptr<clang::ASTConsumer>(
>>         new XMLGeneratorClassConsumer());
>>   }
>> };
>>
>> static llvm::cl::OptionCategory MyToolCategory("my-tool options");
>>
>> int main(int argc, const char **argv) {
>>   CommonOptionsParser OptionsParser(argc, argv, MyToolCategory);
>>   ClangTool Tool(OptionsParser.getCompilations(),
>>                  OptionsParser.getSourcePathList());
>>
>>   return Tool.run(newFrontendActionFactory<XMLGeneratorClassAction>().get());
>> }
>>
>>
>> Compilation: (compile.sh)
>>
>> g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -c parserTest.cpp -o parserTest.o -I/usr/include
>> g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -o parserTest.exe parserTest.o -L/usr/lib64  -Wl,--start-group -lclangAnalysis -lclangParse -lclangEdit -lclangSema -lclangTooling -lclangDriver -lclangSerialization -lclangFrontend -lclangBasic -lclangASTMatchers -lclangAST -lclangLex -lLLVM-8 -Wl,--end-group -lpthread -lz
>>
>>
>> The result is:
>>
>> Function start
>> Function end
>> Annotation start
>> Annotation end
>>
>>
>> What I would expect is:
>>
>> Function start
>> Annotation start
>> Annotation end
>> Function end
>>
>>
>> My questions are now:
>> Is the order provided by my implementation the correct one or is the expected order the correct one?
>> Is there a way to change the visitor order such that attributes are visited inside of the function?
>> If not: Is there a way to get a list of attributes from 'FunctionDecl'? (Could not find any methods in the api https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)
>>
>> Thanks in advance for any help.
>>
>> Cheers
>>
>> Max
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>>
>>
>> --
>>
>> Dr. Max Sagebaum
>>
>> Chair for Scientific Computing,
>> TU Kaiserslautern,
>> Bldg/Geb 34, Paul-Ehrlich-Strasse,
>> 67663 Kaiserslautern, Germany
>>
>> Phone: +49 (0)631 205 5638
>> Fax:   +49 (0)631 205 3056
>> Email: [hidden email]
>> URL:   www.scicomp.uni-kl.de
>>
>>
>>
>>
>>
>>
>
>
> --
> Regards,
> Ilya Biryukov
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wrong order of AST traversal?

Nathan Ridge via cfe-dev
We already have a DEF_TRAVERSE_DECL macro that contains the boilerplate common to all Decl traversals, moving it there seems to give the desired behavior.
I've sent https://reviews.llvm.org/D64907 for review, let's see whether Richard thinks it's a good idea.


On Thu, Jul 18, 2019 at 11:01 AM Kim Gräsman <[hidden email]> wrote:
I looked quickly at RecursiveASTVisitor.h, and it looks like the
challenge is that attributes can be attached to other decls as well.

Attrs are traversed generically for all Decl after any dynamic-type traversal:
https://github.com/llvm/llvm-project/blob/master/clang/include/clang/AST/RecursiveASTVisitor.h#L707

So I think in order to change this, attribute traversal would need to
be custom-implemented for every kind of Decl.

- Kim

On Thu, Jul 18, 2019 at 10:54 AM Ilya Biryukov via cfe-dev
<[hidden email]> wrote:
>
> I would agree the traversal order is wrong, attributes are logically attached to a function, i.e. they are "child nodes".
> Therefore, they should be traversed alongside others child nodes (function body, type, etc).
>
>
> On Thu, Jul 18, 2019 at 10:35 AM Max Sagebaum <[hidden email]> wrote:
>>
>> Hello Ilya,
>>
>> thank you for the fast reply. I do not know why I missed that.
>>
>> Nevertheless, is the traversal order correct?
>>
>> Cheers
>>
>> Max
>>
>> On Thu, 2019-07-18 at 09:40 +0200, Ilya Biryukov wrote:
>>
>> Hi Max,
>>
>> You can get the attributes with Decl::attrs() method (also available for FunctionDecl).
>>
>> On Thu, Jul 18, 2019 at 9:28 AM Max Sagebaum via cfe-dev <[hidden email]> wrote:
>>
>> Hello @ all,
>>
>> I want to annotate some function and generate some additional properties for the annotated functions in an xml document. My problem is, that the AST traversal seems to traverse the attributes of the function after the traversal of the function has been finished. I could not find any valuable resources on this topic by searching via google. So I am addressing my question here.
>>
>> (Since I do not know if file attachments are allowed I provide the files as comments.)
>>
>> My test files is: (attributeTest.cpp)
>>
>> void func1()
>> __attribute((annotate(R"(Test)"))) {
>>   int a = 1 + 1;
>> }
>>
>>
>> clang-check gives me the correct hierarchy: clang-check --ast-dump attributeTest.cpp
>>
>> TranslationUnitDecl 0x559b4c7dc598 <<invalid sloc>> <invalid sloc>
>> |-TypedefDecl 0x559b4c7dce70 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
>> | `-BuiltinType 0x559b4c7dcb30 '__int128'
>> |-TypedefDecl 0x559b4c7dced8 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
>> | `-BuiltinType 0x559b4c7dcb50 'unsigned __int128'
>> |-TypedefDecl 0x559b4c7dd208 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
>> | `-RecordType 0x559b4c7dcfb0 '__NSConstantString_tag'
>> |   `-CXXRecord 0x559b4c7dcf28 '__NSConstantString_tag'
>> |-TypedefDecl 0x559b4c7dd2a0 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
>> | `-PointerType 0x559b4c7dd260 'char *'
>> |   `-BuiltinType 0x559b4c7dc630 'char'
>> |-TypedefDecl 0x559b4c818958 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag [1]'
>> | `-ConstantArrayType 0x559b4c818900 '__va_list_tag [1]' 1
>> |   `-RecordType 0x559b4c7dd380 '__va_list_tag'
>> |     `-CXXRecord 0x559b4c7dd2f0 '__va_list_tag'
>> `-FunctionDecl 0x559b4c818a98 </home/msagebaum/Kaiserslautern/Programms/adDSLParser/temp/attributeTest.cpp:1:1, line:4:1> line:1:6 func1 'void ()'
>>   |-CompoundStmt 0x559b4c818cb0 <line:2:36, line:4:1>
>>   | `-DeclStmt 0x559b4c818c98 <line:3:3, col:16>
>>   |   `-VarDecl 0x559b4c818bd8 <col:3, col:15> col:7 a 'int' cinit
>>   |     `-BinaryOperator 0x559b4c818c78 <col:11, col:15> 'int' '+'
>>   |       |-IntegerLiteral 0x559b4c818c38 <col:11> 'int' 1
>>   |       `-IntegerLiteral 0x559b4c818c58 <col:15> 'int' 1
>>   `-AnnotateAttr 0x559b4c818b30 <line:2:14, col:32> "Test"
>>
>>
>> If I now run the file through my AST Visitor: (parserTest.cpp)
>>
>> #include <iostream>
>>
>> #include "clang/AST/ASTConsumer.h"
>> #include "clang/AST/QualTypeNames.h"
>> #include "clang/AST/RecursiveASTVisitor.h"
>> #include "clang/Frontend/CompilerInstance.h"
>> #include "clang/Frontend/FrontendAction.h"
>> #include "clang/Tooling/CommonOptionsParser.h"
>> #include "clang/Tooling/Tooling.h"
>>
>> #include <llvm/Support/CommandLine.h>
>>
>> using namespace llvm;
>> using namespace clang;
>> using namespace clang::tooling;
>>
>> class XMLGenerator : public RecursiveASTVisitor<XMLGenerator> {
>> public:
>>
>>   bool TraverseFunctionDecl(FunctionDecl *declaration) {
>>
>>     std::cout << "Function start" << std::endl;
>>
>>     RecursiveASTVisitor<XMLGenerator>::TraverseFunctionDecl(declaration);
>>
>>     std::cout << "Function end" << std::endl;
>>     return true;
>>   }
>>
>>   bool TraverseAnnotateAttr(AnnotateAttr *a) {
>>     std::cout << "Annotation start" << std::endl;
>>
>>     RecursiveASTVisitor<XMLGenerator>::TraverseAnnotateAttr(a);
>>
>>     std::cout << "Annotation end" << std::endl;
>>     return true;
>>   }
>> };
>>
>> class XMLGeneratorClassConsumer : public clang::ASTConsumer {
>> public:
>>   explicit XMLGeneratorClassConsumer()
>>     : Visitor() {}
>>
>>   virtual void HandleTranslationUnit(clang::ASTContext &Context) {
>>     Visitor.TraverseDecl(Context.getTranslationUnitDecl());
>>   }
>> private:
>>   XMLGenerator Visitor;
>> };
>>
>> class XMLGeneratorClassAction : public clang::ASTFrontendAction {
>> public:
>>   virtual std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(
>>     clang::CompilerInstance &Compiler, llvm::StringRef InFile) {
>>     return std::unique_ptr<clang::ASTConsumer>(
>>         new XMLGeneratorClassConsumer());
>>   }
>> };
>>
>> static llvm::cl::OptionCategory MyToolCategory("my-tool options");
>>
>> int main(int argc, const char **argv) {
>>   CommonOptionsParser OptionsParser(argc, argv, MyToolCategory);
>>   ClangTool Tool(OptionsParser.getCompilations(),
>>                  OptionsParser.getSourcePathList());
>>
>>   return Tool.run(newFrontendActionFactory<XMLGeneratorClassAction>().get());
>> }
>>
>>
>> Compilation: (compile.sh)
>>
>> g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -c parserTest.cpp -o parserTest.o -I/usr/include
>> g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -o parserTest.exe parserTest.o -L/usr/lib64  -Wl,--start-group -lclangAnalysis -lclangParse -lclangEdit -lclangSema -lclangTooling -lclangDriver -lclangSerialization -lclangFrontend -lclangBasic -lclangASTMatchers -lclangAST -lclangLex -lLLVM-8 -Wl,--end-group -lpthread -lz
>>
>>
>> The result is:
>>
>> Function start
>> Function end
>> Annotation start
>> Annotation end
>>
>>
>> What I would expect is:
>>
>> Function start
>> Annotation start
>> Annotation end
>> Function end
>>
>>
>> My questions are now:
>> Is the order provided by my implementation the correct one or is the expected order the correct one?
>> Is there a way to change the visitor order such that attributes are visited inside of the function?
>> If not: Is there a way to get a list of attributes from 'FunctionDecl'? (Could not find any methods in the api https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)
>>
>> Thanks in advance for any help.
>>
>> Cheers
>>
>> Max
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>>
>>
>> --
>>
>> Dr. Max Sagebaum
>>
>> Chair for Scientific Computing,
>> TU Kaiserslautern,
>> Bldg/Geb 34, Paul-Ehrlich-Strasse,
>> 67663 Kaiserslautern, Germany
>>
>> Phone: +49 (0)631 205 5638
>> Fax:   +49 (0)631 205 3056
>> Email: [hidden email]
>> URL:   www.scicomp.uni-kl.de
>>
>>
>>
>>
>>
>>
>
>
> --
> Regards,
> Ilya Biryukov
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


--
Regards,
Ilya Biryukov

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wrong order of AST traversal?

Nathan Ridge via cfe-dev
Nice, I hadn't seen that!

- Kim

On Thu, Jul 18, 2019 at 11:40 AM Ilya Biryukov <[hidden email]> wrote:

>
> We already have a DEF_TRAVERSE_DECL macro that contains the boilerplate common to all Decl traversals, moving it there seems to give the desired behavior.
> I've sent https://reviews.llvm.org/D64907 for review, let's see whether Richard thinks it's a good idea.
>
>
> On Thu, Jul 18, 2019 at 11:01 AM Kim Gräsman <[hidden email]> wrote:
>>
>> I looked quickly at RecursiveASTVisitor.h, and it looks like the
>> challenge is that attributes can be attached to other decls as well.
>>
>> Attrs are traversed generically for all Decl after any dynamic-type traversal:
>> https://github.com/llvm/llvm-project/blob/master/clang/include/clang/AST/RecursiveASTVisitor.h#L707
>>
>> So I think in order to change this, attribute traversal would need to
>> be custom-implemented for every kind of Decl.
>>
>> - Kim
>>
>> On Thu, Jul 18, 2019 at 10:54 AM Ilya Biryukov via cfe-dev
>> <[hidden email]> wrote:
>> >
>> > I would agree the traversal order is wrong, attributes are logically attached to a function, i.e. they are "child nodes".
>> > Therefore, they should be traversed alongside others child nodes (function body, type, etc).
>> >
>> >
>> > On Thu, Jul 18, 2019 at 10:35 AM Max Sagebaum <[hidden email]> wrote:
>> >>
>> >> Hello Ilya,
>> >>
>> >> thank you for the fast reply. I do not know why I missed that.
>> >>
>> >> Nevertheless, is the traversal order correct?
>> >>
>> >> Cheers
>> >>
>> >> Max
>> >>
>> >> On Thu, 2019-07-18 at 09:40 +0200, Ilya Biryukov wrote:
>> >>
>> >> Hi Max,
>> >>
>> >> You can get the attributes with Decl::attrs() method (also available for FunctionDecl).
>> >>
>> >> On Thu, Jul 18, 2019 at 9:28 AM Max Sagebaum via cfe-dev <[hidden email]> wrote:
>> >>
>> >> Hello @ all,
>> >>
>> >> I want to annotate some function and generate some additional properties for the annotated functions in an xml document. My problem is, that the AST traversal seems to traverse the attributes of the function after the traversal of the function has been finished. I could not find any valuable resources on this topic by searching via google. So I am addressing my question here.
>> >>
>> >> (Since I do not know if file attachments are allowed I provide the files as comments.)
>> >>
>> >> My test files is: (attributeTest.cpp)
>> >>
>> >> void func1()
>> >> __attribute((annotate(R"(Test)"))) {
>> >>   int a = 1 + 1;
>> >> }
>> >>
>> >>
>> >> clang-check gives me the correct hierarchy: clang-check --ast-dump attributeTest.cpp
>> >>
>> >> TranslationUnitDecl 0x559b4c7dc598 <<invalid sloc>> <invalid sloc>
>> >> |-TypedefDecl 0x559b4c7dce70 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
>> >> | `-BuiltinType 0x559b4c7dcb30 '__int128'
>> >> |-TypedefDecl 0x559b4c7dced8 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
>> >> | `-BuiltinType 0x559b4c7dcb50 'unsigned __int128'
>> >> |-TypedefDecl 0x559b4c7dd208 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
>> >> | `-RecordType 0x559b4c7dcfb0 '__NSConstantString_tag'
>> >> |   `-CXXRecord 0x559b4c7dcf28 '__NSConstantString_tag'
>> >> |-TypedefDecl 0x559b4c7dd2a0 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
>> >> | `-PointerType 0x559b4c7dd260 'char *'
>> >> |   `-BuiltinType 0x559b4c7dc630 'char'
>> >> |-TypedefDecl 0x559b4c818958 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag [1]'
>> >> | `-ConstantArrayType 0x559b4c818900 '__va_list_tag [1]' 1
>> >> |   `-RecordType 0x559b4c7dd380 '__va_list_tag'
>> >> |     `-CXXRecord 0x559b4c7dd2f0 '__va_list_tag'
>> >> `-FunctionDecl 0x559b4c818a98 </home/msagebaum/Kaiserslautern/Programms/adDSLParser/temp/attributeTest.cpp:1:1, line:4:1> line:1:6 func1 'void ()'
>> >>   |-CompoundStmt 0x559b4c818cb0 <line:2:36, line:4:1>
>> >>   | `-DeclStmt 0x559b4c818c98 <line:3:3, col:16>
>> >>   |   `-VarDecl 0x559b4c818bd8 <col:3, col:15> col:7 a 'int' cinit
>> >>   |     `-BinaryOperator 0x559b4c818c78 <col:11, col:15> 'int' '+'
>> >>   |       |-IntegerLiteral 0x559b4c818c38 <col:11> 'int' 1
>> >>   |       `-IntegerLiteral 0x559b4c818c58 <col:15> 'int' 1
>> >>   `-AnnotateAttr 0x559b4c818b30 <line:2:14, col:32> "Test"
>> >>
>> >>
>> >> If I now run the file through my AST Visitor: (parserTest.cpp)
>> >>
>> >> #include <iostream>
>> >>
>> >> #include "clang/AST/ASTConsumer.h"
>> >> #include "clang/AST/QualTypeNames.h"
>> >> #include "clang/AST/RecursiveASTVisitor.h"
>> >> #include "clang/Frontend/CompilerInstance.h"
>> >> #include "clang/Frontend/FrontendAction.h"
>> >> #include "clang/Tooling/CommonOptionsParser.h"
>> >> #include "clang/Tooling/Tooling.h"
>> >>
>> >> #include <llvm/Support/CommandLine.h>
>> >>
>> >> using namespace llvm;
>> >> using namespace clang;
>> >> using namespace clang::tooling;
>> >>
>> >> class XMLGenerator : public RecursiveASTVisitor<XMLGenerator> {
>> >> public:
>> >>
>> >>   bool TraverseFunctionDecl(FunctionDecl *declaration) {
>> >>
>> >>     std::cout << "Function start" << std::endl;
>> >>
>> >>     RecursiveASTVisitor<XMLGenerator>::TraverseFunctionDecl(declaration);
>> >>
>> >>     std::cout << "Function end" << std::endl;
>> >>     return true;
>> >>   }
>> >>
>> >>   bool TraverseAnnotateAttr(AnnotateAttr *a) {
>> >>     std::cout << "Annotation start" << std::endl;
>> >>
>> >>     RecursiveASTVisitor<XMLGenerator>::TraverseAnnotateAttr(a);
>> >>
>> >>     std::cout << "Annotation end" << std::endl;
>> >>     return true;
>> >>   }
>> >> };
>> >>
>> >> class XMLGeneratorClassConsumer : public clang::ASTConsumer {
>> >> public:
>> >>   explicit XMLGeneratorClassConsumer()
>> >>     : Visitor() {}
>> >>
>> >>   virtual void HandleTranslationUnit(clang::ASTContext &Context) {
>> >>     Visitor.TraverseDecl(Context.getTranslationUnitDecl());
>> >>   }
>> >> private:
>> >>   XMLGenerator Visitor;
>> >> };
>> >>
>> >> class XMLGeneratorClassAction : public clang::ASTFrontendAction {
>> >> public:
>> >>   virtual std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(
>> >>     clang::CompilerInstance &Compiler, llvm::StringRef InFile) {
>> >>     return std::unique_ptr<clang::ASTConsumer>(
>> >>         new XMLGeneratorClassConsumer());
>> >>   }
>> >> };
>> >>
>> >> static llvm::cl::OptionCategory MyToolCategory("my-tool options");
>> >>
>> >> int main(int argc, const char **argv) {
>> >>   CommonOptionsParser OptionsParser(argc, argv, MyToolCategory);
>> >>   ClangTool Tool(OptionsParser.getCompilations(),
>> >>                  OptionsParser.getSourcePathList());
>> >>
>> >>   return Tool.run(newFrontendActionFactory<XMLGeneratorClassAction>().get());
>> >> }
>> >>
>> >>
>> >> Compilation: (compile.sh)
>> >>
>> >> g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -c parserTest.cpp -o parserTest.o -I/usr/include
>> >> g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -o parserTest.exe parserTest.o -L/usr/lib64  -Wl,--start-group -lclangAnalysis -lclangParse -lclangEdit -lclangSema -lclangTooling -lclangDriver -lclangSerialization -lclangFrontend -lclangBasic -lclangASTMatchers -lclangAST -lclangLex -lLLVM-8 -Wl,--end-group -lpthread -lz
>> >>
>> >>
>> >> The result is:
>> >>
>> >> Function start
>> >> Function end
>> >> Annotation start
>> >> Annotation end
>> >>
>> >>
>> >> What I would expect is:
>> >>
>> >> Function start
>> >> Annotation start
>> >> Annotation end
>> >> Function end
>> >>
>> >>
>> >> My questions are now:
>> >> Is the order provided by my implementation the correct one or is the expected order the correct one?
>> >> Is there a way to change the visitor order such that attributes are visited inside of the function?
>> >> If not: Is there a way to get a list of attributes from 'FunctionDecl'? (Could not find any methods in the api https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)
>> >>
>> >> Thanks in advance for any help.
>> >>
>> >> Cheers
>> >>
>> >> Max
>> >> _______________________________________________
>> >> cfe-dev mailing list
>> >> [hidden email]
>> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>> >>
>> >>
>> >>
>> >> --
>> >>
>> >> Dr. Max Sagebaum
>> >>
>> >> Chair for Scientific Computing,
>> >> TU Kaiserslautern,
>> >> Bldg/Geb 34, Paul-Ehrlich-Strasse,
>> >> 67663 Kaiserslautern, Germany
>> >>
>> >> Phone: +49 (0)631 205 5638
>> >> Fax:   +49 (0)631 205 3056
>> >> Email: [hidden email]
>> >> URL:   www.scicomp.uni-kl.de
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > Regards,
>> > Ilya Biryukov
>> > _______________________________________________
>> > cfe-dev mailing list
>> > [hidden email]
>> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
>
> --
> Regards,
> Ilya Biryukov
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev