Clang PGO and LLVM PGO commandline interface

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Clang PGO and LLVM PGO commandline interface

Louis Dionne via cfe-dev
Hello all,
  A while ago, I implemented frontend-based PGO in LDC (very similar to Clang's) and now I'd like to add IR-based PGO. I want the commandline interface to be the same (or very similar) to Clang's, but I am having trouble finding the current state of Clang's PGO commandline flags (and their meaning related to Clang/LLVM-IR PGO).

Can you give me a pointer to where I can find this, or perhaps give me a brief overview of the current flags and how to select whether to use frontend-based or IR-based PGO ?

Thanks a lot,
  Johan

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Clang PGO and LLVM PGO commandline interface

Louis Dionne via cfe-dev
https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization is the documentation for Clang's PGO.  It doesn't explicitly state which flags map to frontend and IR-based instrumentation.

* -fprofile-generate and -fprofile-use are for IR-based instrumentation and profile use.
* -fprofile-instr-generate and -fprofile-instr-use are for frontend-based instrumentation and profile use.
* -fprofile-sample-use is to use sparse profiles from hardware-counter-based profiles.



On Wed, Dec 27, 2017 at 2:37 PM, Johan Engelen via cfe-dev <[hidden email]> wrote:
Hello all,
  A while ago, I implemented frontend-based PGO in LDC (very similar to Clang's) and now I'd like to add IR-based PGO. I want the commandline interface to be the same (or very similar) to Clang's, but I am having trouble finding the current state of Clang's PGO commandline flags (and their meaning related to Clang/LLVM-IR PGO).

Can you give me a pointer to where I can find this, or perhaps give me a brief overview of the current flags and how to select whether to use frontend-based or IR-based PGO ?

Thanks a lot,
  Johan

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Clang PGO and LLVM PGO commandline interface

Louis Dionne via cfe-dev
-fprofile-use and -fprofile-instr-use can actually be used interchangeably for front-end and IR based PGO.  The difference is that -fprofile-use=<> also takes a directory as the argument.


David

On Wed, Dec 27, 2017 at 7:46 PM, Pirama Arumuga Nainar via cfe-dev <[hidden email]> wrote:
https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization is the documentation for Clang's PGO.  It doesn't explicitly state which flags map to frontend and IR-based instrumentation.

* -fprofile-generate and -fprofile-use are for IR-based instrumentation and profile use.
* -fprofile-instr-generate and -fprofile-instr-use are for frontend-based instrumentation and profile use.
* -fprofile-sample-use is to use sparse profiles from hardware-counter-based profiles.



On Wed, Dec 27, 2017 at 2:37 PM, Johan Engelen via cfe-dev <[hidden email]> wrote:
Hello all,
  A while ago, I implemented frontend-based PGO in LDC (very similar to Clang's) and now I'd like to add IR-based PGO. I want the commandline interface to be the same (or very similar) to Clang's, but I am having trouble finding the current state of Clang's PGO commandline flags (and their meaning related to Clang/LLVM-IR PGO).

Can you give me a pointer to where I can find this, or perhaps give me a brief overview of the current flags and how to select whether to use frontend-based or IR-based PGO ?

Thanks a lot,
  Johan

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Clang PGO and LLVM PGO commandline interface

Louis Dionne via cfe-dev
Hi David and Pirama,     (first sent off-list, but resending to list)
  Thank you for your quick responses. I've now successfully added IR-based PGO to LDC; it's almost trivial. The only thing needed (on LLVM side) was
```
// For generating pgo info
    builder.EnablePGOInstrGen = true;
    builder.PGOInstrGen = <outputfilename>;
```

```
// For using pgo profile
    builder.PGOInstrUse = <inputfilename>;
````
where `builder` is the PassManagerBuilder.

This works well on Linux and macOS, however for a Windows target triple, it does not work. The "instrprof" pass fails in that case.
I have attached the failing testcase, that fails with "opt -instrprof crash.ll":
```
>  opt crash.ll -instrprof

comdat global value has private linkage
[1 x i64]* @__profc_luke
comdat global value has private linkage
[1 x i64]* @__profc_cold
comdat global value has private linkage
[1 x i64]* @__profc_main
comdat global value has private linkage
[2 x i64]* @__profc__Dmain
LLVM ERROR: Broken module found, compilation aborted!
```
I've obtained the IR by compiling the D code with `-print-before-all`, and crash.ll is the IR before the instrprof pass.

Now, before I file this as a bug, I want to make sure that I didn't do anything wrong in setting up IR-based PGO. Are there any other steps needed? FYI: we don't use a special pipeline setup, just the default what LLVM gives us: https://github.com/ldc-developers/ldc/blob/2645ae8645c70d4dc01988f85482d1b85f82fc14/gen/optimizer.cpp#L235-L331 .
(somehow "opt -instr-prof-gen" does work for Windows...)

As Pirama suggested, I tested with Clang and there it does work with a Windows triple. (https://godbolt.org/g/h5CMJx)

Thanks a lot,
Kind regards,
  Johan

On Thu, Dec 28, 2017 at 7:02 AM, Xinliang David Li <[hidden email]> wrote:
-fprofile-use and -fprofile-instr-use can actually be used interchangeably for front-end and IR based PGO.  The difference is that -fprofile-use=<> also takes a directory as the argument.


David

On Wed, Dec 27, 2017 at 7:46 PM, Pirama Arumuga Nainar via cfe-dev <[hidden email]> wrote:
https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization is the documentation for Clang's PGO.  It doesn't explicitly state which flags map to frontend and IR-based instrumentation.

* -fprofile-generate and -fprofile-use are for IR-based instrumentation and profile use.
* -fprofile-instr-generate and -fprofile-instr-use are for frontend-based instrumentation and profile use.
* -fprofile-sample-use is to use sparse profiles from hardware-counter-based profiles.



On Wed, Dec 27, 2017 at 2:37 PM, Johan Engelen via cfe-dev <[hidden email]> wrote:
Hello all,
  A while ago, I implemented frontend-based PGO in LDC (very similar to Clang's) and now I'd like to add IR-based PGO. I want the commandline interface to be the same (or very similar) to Clang's, but I am having trouble finding the current state of Clang's PGO commandline flags (and their meaning related to Clang/LLVM-IR PGO).

Can you give me a pointer to where I can find this, or perhaps give me a brief overview of the current flags and how to select whether to use frontend-based or IR-based PGO ?

Thanks a lot,
  Johan

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

crash.ll (8K) Download Attachment