Compile to ast and llvm bytecode using Clang API

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Compile to ast and llvm bytecode using Clang API

Oleg Smolsky via cfe-dev
I have a requirement to parse a .cpp file to a .ast file and a .bc file.
I have the parsing, with filtering of the AST working using the the
tooling tempalte on this page:
https://clang.llvm.org/docs/RAVFrontendAction.html. I also need to
generate LLVM code to write out to a .bc file. I tried using :

tool.run (newFrontEndActionFactory<EmitBCAction>().get())

where tool is an instance of ClangTool but received an error saying "not
enough position command line arguments specified". Any ideas? Are there
any examples I can look at?

Regards,

Michael Collison


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Compile to ast and llvm bytecode using Clang API

Oleg Smolsky via cfe-dev
Michael,

The easiest way to use Clang API to perform "standard" compiler tasks is through the Driver class.
Basically what you need to do is setup a command line invocation, using the flags from a compilation database or adding your own, build a Compilation object and execute it through the driver.

For instance:

std::string CommandLine = "clang++ -Wall -O3 -emit-llvm -o test.bc test.cpp";
clang::DiagnosticOptions* DiagOptions = new DiagnosticOptions();
clang::DiagnosticConsumer *DiagClient = new TextDiagnosticPrinter(llvm::errs(), DiagOptions);
llvm::IntrusiveRefCntPtr<clang::DiagnosticIDs> DiagID(new clang::DiagnosticIDs());
clang::DiagnosticsEngine Diags(DiagID, DiagOptions, DiagClient);
clang::driver::Driver TheDriver("clang++", llvm::sys::getDefaultTargetTriple(), Diags);
const std::unique_ptr<driver::Compilation> compilation(TheDriver.BuildCompilation(CommandLine));
if (!compilation) { llvm::errs() << "failed to build compilation object!\n"; }
llvm::SmallVector<std::pair<int, const driver::Command*>,10> failingCommands;
int res = TheDriver.ExecuteCompilation(*compilation, failingCommands);
if (res < 0) {
  ProcessingFailed = true;
  for (const std::pair<int, const driver::Command*>& p : failingCommands) {
    if (p.first) {
      TheDriver.generateCompilationDiagnostics(*compilation, *p.second);
    }
  }
}


On Sun, Nov 11, 2018 at 6:53 PM Michael Collison via cfe-dev <[hidden email]> wrote:
I have a requirement to parse a .cpp file to a .ast file and a .bc file.
I have the parsing, with filtering of the AST working using the the
tooling tempalte on this page:
https://clang.llvm.org/docs/RAVFrontendAction.html. I also need to
generate LLVM code to write out to a .bc file. I tried using :

tool.run (newFrontEndActionFactory<EmitBCAction>().get())

where tool is an instance of ClangTool but received an error saying "not
enough position command line arguments specified". Any ideas? Are there
any examples I can look at?

Regards,

Michael Collison


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


--
João Paulo L. de Carvalho
Computer Science |  IC-UNICAMP | Campinas , SP - Brazil

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Compile to ast and llvm bytecode using Clang API

Oleg Smolsky via cfe-dev

João,

Thank you for your reply. I t appears from your response that I need to make "two passes" over the input file: one to generate my modified ast output file using FrontEndAction and a second pass using the code template your provided to generate the .bc file.

On 11/11/2018 2:33 PM, João Paulo Labegalini de Carvalho wrote:
Michael,

The easiest way to use Clang API to perform "standard" compiler tasks is through the Driver class.
Basically what you need to do is setup a command line invocation, using the flags from a compilation database or adding your own, build a Compilation object and execute it through the driver.

For instance:

std::string CommandLine = "clang++ -Wall -O3 -emit-llvm -o test.bc test.cpp";
clang::DiagnosticOptions* DiagOptions = new DiagnosticOptions();
clang::DiagnosticConsumer *DiagClient = new TextDiagnosticPrinter(llvm::errs(), DiagOptions);
llvm::IntrusiveRefCntPtr<clang::DiagnosticIDs> DiagID(new clang::DiagnosticIDs());
clang::DiagnosticsEngine Diags(DiagID, DiagOptions, DiagClient);
clang::driver::Driver TheDriver("clang++", llvm::sys::getDefaultTargetTriple(), Diags);
const std::unique_ptr<driver::Compilation> compilation(TheDriver.BuildCompilation(CommandLine));
if (!compilation) { llvm::errs() << "failed to build compilation object!\n"; }
llvm::SmallVector<std::pair<int, const driver::Command*>,10> failingCommands;
int res = TheDriver.ExecuteCompilation(*compilation, failingCommands);
if (res < 0) {
  ProcessingFailed = true;
  for (const std::pair<int, const driver::Command*>& p : failingCommands) {
    if (p.first) {
      TheDriver.generateCompilationDiagnostics(*compilation, *p.second);
    }
  }
}


On Sun, Nov 11, 2018 at 6:53 PM Michael Collison via cfe-dev <[hidden email]> wrote:
I have a requirement to parse a .cpp file to a .ast file and a .bc file.
I have the parsing, with filtering of the AST working using the the
tooling tempalte on this page:
https://clang.llvm.org/docs/RAVFrontendAction.html. I also need to
generate LLVM code to write out to a .bc file. I tried using :

tool.run (newFrontEndActionFactory<EmitBCAction>().get())

where tool is an instance of ClangTool but received an error saying "not
enough position command line arguments specified". Any ideas? Are there
any examples I can look at?

Regards,

Michael Collison


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


--
João Paulo L. de Carvalho
Computer Science |  IC-UNICAMP | Campinas , SP - Brazil

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Compile to ast and llvm bytecode using Clang API

Oleg Smolsky via cfe-dev
Michael,

Exactly. I do not know if there is another way. I have a tool which runs, for each source file, 3 AST Actions and generates a .bc using the code I provided.

Something i did not try was to map a source file into a virtual one via ClangTool::mapVirtualFile. It seems that you can do it only once, then run your AST Actions and finally generate code. But I am new to clang, so I might be wrong.

On Sun, Nov 11, 2018 at 7:48 PM Michael Collison <[hidden email]> wrote:

João,

Thank you for your reply. I t appears from your response that I need to make "two passes" over the input file: one to generate my modified ast output file using FrontEndAction and a second pass using the code template your provided to generate the .bc file.

On 11/11/2018 2:33 PM, João Paulo Labegalini de Carvalho wrote:
Michael,

The easiest way to use Clang API to perform "standard" compiler tasks is through the Driver class.
Basically what you need to do is setup a command line invocation, using the flags from a compilation database or adding your own, build a Compilation object and execute it through the driver.

For instance:

std::string CommandLine = "clang++ -Wall -O3 -emit-llvm -o test.bc test.cpp";
clang::DiagnosticOptions* DiagOptions = new DiagnosticOptions();
clang::DiagnosticConsumer *DiagClient = new TextDiagnosticPrinter(llvm::errs(), DiagOptions);
llvm::IntrusiveRefCntPtr<clang::DiagnosticIDs> DiagID(new clang::DiagnosticIDs());
clang::DiagnosticsEngine Diags(DiagID, DiagOptions, DiagClient);
clang::driver::Driver TheDriver("clang++", llvm::sys::getDefaultTargetTriple(), Diags);
const std::unique_ptr<driver::Compilation> compilation(TheDriver.BuildCompilation(CommandLine));
if (!compilation) { llvm::errs() << "failed to build compilation object!\n"; }
llvm::SmallVector<std::pair<int, const driver::Command*>,10> failingCommands;
int res = TheDriver.ExecuteCompilation(*compilation, failingCommands);
if (res < 0) {
  ProcessingFailed = true;
  for (const std::pair<int, const driver::Command*>& p : failingCommands) {
    if (p.first) {
      TheDriver.generateCompilationDiagnostics(*compilation, *p.second);
    }
  }
}


On Sun, Nov 11, 2018 at 6:53 PM Michael Collison via cfe-dev <[hidden email]> wrote:
I have a requirement to parse a .cpp file to a .ast file and a .bc file.
I have the parsing, with filtering of the AST working using the the
tooling tempalte on this page:
https://clang.llvm.org/docs/RAVFrontendAction.html. I also need to
generate LLVM code to write out to a .bc file. I tried using :

tool.run (newFrontEndActionFactory<EmitBCAction>().get())

where tool is an instance of ClangTool but received an error saying "not
enough position command line arguments specified". Any ideas? Are there
any examples I can look at?

Regards,

Michael Collison


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


--
João Paulo L. de Carvalho
Computer Science |  IC-UNICAMP | Campinas , SP - Brazil


--
João Paulo L. de Carvalho
Computer Science |  IC-UNICAMP | Campinas , SP - Brazil

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Compile to ast and llvm bytecode using Clang API

Oleg Smolsky via cfe-dev
In reply to this post by Oleg Smolsky via cfe-dev

João,

I just got back around to trying the example program. I had to make a few changes to get the code to compile. When I try the program it says it is unable to find clang++ which is definitely on my system. Do I need to hard code the path to clang++?

On 11/11/18 2:33 PM, João Paulo Labegalini de Carvalho wrote:
Michael,

The easiest way to use Clang API to perform "standard" compiler tasks is through the Driver class.
Basically what you need to do is setup a command line invocation, using the flags from a compilation database or adding your own, build a Compilation object and execute it through the driver.

For instance:

std::string CommandLine = "clang++ -Wall -O3 -emit-llvm -o test.bc test.cpp";
clang::DiagnosticOptions* DiagOptions = new DiagnosticOptions();
clang::DiagnosticConsumer *DiagClient = new TextDiagnosticPrinter(llvm::errs(), DiagOptions);
llvm::IntrusiveRefCntPtr<clang::DiagnosticIDs> DiagID(new clang::DiagnosticIDs());
clang::DiagnosticsEngine Diags(DiagID, DiagOptions, DiagClient);
clang::driver::Driver TheDriver("clang++", llvm::sys::getDefaultTargetTriple(), Diags);
const std::unique_ptr<driver::Compilation> compilation(TheDriver.BuildCompilation(CommandLine));
if (!compilation) { llvm::errs() << "failed to build compilation object!\n"; }
llvm::SmallVector<std::pair<int, const driver::Command*>,10> failingCommands;
int res = TheDriver.ExecuteCompilation(*compilation, failingCommands);
if (res < 0) {
  ProcessingFailed = true;
  for (const std::pair<int, const driver::Command*>& p : failingCommands) {
    if (p.first) {
      TheDriver.generateCompilationDiagnostics(*compilation, *p.second);
    }
  }
}


On Sun, Nov 11, 2018 at 6:53 PM Michael Collison via cfe-dev <[hidden email]> wrote:
I have a requirement to parse a .cpp file to a .ast file and a .bc file.
I have the parsing, with filtering of the AST working using the the
tooling tempalte on this page:
https://clang.llvm.org/docs/RAVFrontendAction.html. I also need to
generate LLVM code to write out to a .bc file. I tried using :

tool.run (newFrontEndActionFactory<EmitBCAction>().get())

where tool is an instance of ClangTool but received an error saying "not
enough position command line arguments specified". Any ideas? Are there
any examples I can look at?

Regards,

Michael Collison


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


--
João Paulo L. de Carvalho
Computer Science |  IC-UNICAMP | Campinas , SP - Brazil

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Compile to ast and llvm bytecode using Clang API

Oleg Smolsky via cfe-dev

I just got back around to trying the example program. I had to make a few changes to get the code to compile.

You are right, after I have sent you the code I realized it was not correct. The CommandLine you pass to Driver::BuildCompilation must be a ArrayRef not a string. Sorry for the confusion. 
 

When I try the program it says it is unable to find clang++ which is definitely on my system. Do I need to hard code the path to clang++?

Yes, the path for clang/llvm tools must be absolute paths. Thats why I find it easier to re-use the command line from the compilation database. In order to generate the compile_commands.json (a.k.a. compilation database), you just need to pass -DCMAKE_EXPORT_COMPILE_COMMANDS to cmake while configuring the build of your project. After that, you can retrieve the compile command for a specific file using the tooling::CompilationDatabase::getCompileCommmands() method.

 

On 11/11/18 2:33 PM, João Paulo Labegalini de Carvalho wrote:
Michael,

The easiest way to use Clang API to perform "standard" compiler tasks is through the Driver class.
Basically what you need to do is setup a command line invocation, using the flags from a compilation database or adding your own, build a Compilation object and execute it through the driver.

For instance:

std::string CommandLine = "clang++ -Wall -O3 -emit-llvm -o test.bc test.cpp";
clang::DiagnosticOptions* DiagOptions = new DiagnosticOptions();
clang::DiagnosticConsumer *DiagClient = new TextDiagnosticPrinter(llvm::errs(), DiagOptions);
llvm::IntrusiveRefCntPtr<clang::DiagnosticIDs> DiagID(new clang::DiagnosticIDs());
clang::DiagnosticsEngine Diags(DiagID, DiagOptions, DiagClient);
clang::driver::Driver TheDriver("clang++", llvm::sys::getDefaultTargetTriple(), Diags);
const std::unique_ptr<driver::Compilation> compilation(TheDriver.BuildCompilation(CommandLine));
if (!compilation) { llvm::errs() << "failed to build compilation object!\n"; }
llvm::SmallVector<std::pair<int, const driver::Command*>,10> failingCommands;
int res = TheDriver.ExecuteCompilation(*compilation, failingCommands);
if (res < 0) {
  ProcessingFailed = true;
  for (const std::pair<int, const driver::Command*>& p : failingCommands) {
    if (p.first) {
      TheDriver.generateCompilationDiagnostics(*compilation, *p.second);
    }
  }
}


On Sun, Nov 11, 2018 at 6:53 PM Michael Collison via cfe-dev <[hidden email]> wrote:
I have a requirement to parse a .cpp file to a .ast file and a .bc file.
I have the parsing, with filtering of the AST working using the the
tooling tempalte on this page:
https://clang.llvm.org/docs/RAVFrontendAction.html. I also need to
generate LLVM code to write out to a .bc file. I tried using :

tool.run (newFrontEndActionFactory<EmitBCAction>().get())

where tool is an instance of ClangTool but received an error saying "not
enough position command line arguments specified". Any ideas? Are there
any examples I can look at?

Regards,

Michael Collison


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


--
João Paulo L. de Carvalho
Computer Science |  IC-UNICAMP | Campinas , SP - Brazil


--
João Paulo L. de Carvalho
Computer Science |  IC-UNICAMP | Campinas , SP - Brazil

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev