[XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

Jacob Carlborg via cfe-dev
TL;DR: Adding the [[clang::xray_{always,never}_instrument]] attribute in the declaration/definition of a function is cumbersome for guaranteed instrumentation (or non-instrumentation) of specific functions. We'd like to make the imbuing of this particular attribute a command-line controllable option.

Background
==========

XRay is a function call tracing system that's implemented in LLVM. One part of this system involves enabling users to use a source-level attribute to mark that a function is always instrumented with XRay when building with the `-fxray-instrument` flag enabled. While this is sufficient for some users, it's really inconvenient to have to manually add this attribute to functions that need to be marked with this attribute. Sometimes the decision on whether a function must always be instrumented isn't always known a-priori and could be the result of (or driver for) experimental methods (i.e. building with all functions instrumented, and in next iterations pruning the list down or changing thresholds, etc).

Proposal
=======

We would like to add two flags (`-fxray-always-instrument=` and `-fxray-never-instrument=`) to clang that takes filenames processed similar to how the SanitizerSpecialCaseList [0] feature is handled. We apply first the blacklist, then the whitelist to determine whether a certain functions should be either never instrumented (in the blacklist) or always instrumented (in the whitelist). This allows users to define sets of functions that should be either always or never instrumented with XRay through command-line options.

These lists will only take effect when `-fxray-instrument` is defined when invoking Clang. Currently, we determine whether instrumentation is necessary based on the combination of attributes and whether the instruction count threshold is met (see more at http://llvm.org/docs/XRay.html).

The intent is to have a single class (XRayFunctionFilter) which maintains the special case lists to determine whether to always instrument a function or never instrument a function. We add an instance of class to the ASTContext, and when code-genning function declarations that fall in whitelist will get the "xray_always_instrument" attribute, and if they fall in the blacklist (and not the whitelist) will get the "xray_never_instrument" attribute. The strawman API for determining whether to always or never instrument is as follows:

class XRayFunctionFilter {
public:
  enum class ImbueAttribute {
    NONE, ALWAYS, NEVER, …
  }

  ImbueAttribute shouldImbueAttribute(StringRef FunctionName) const;
private:
  ...
};

An open question is what to do to functions that have the attribute explicitly defined in the source -- whether the whitelist/blacklist overrides those. I'm inclined to think that we should preserve the source-level attribute, but am willing to go either way.

We also have work on-going to enable logging some arguments to function calls (https://reviews.llvm.org/D29704). In these cases we can use the whitelist to also say whether to imbue the attribute using the categories feature in the SanitizerSpecialCaseList file format.

Does the community think this is something worth exploring?

A "natural" extension of this feature would be to just support compile-time attribution of any attribute to any function/class/<insert construct here>. While that might be a more general approach, it seems a bit more invasive and larger a scope than the targeted proposal being made. We aren't opposed per-se to doing this general approach, and believe that the targeted approach we're proposing might inform the design of the more general approach later on.

Thoughts?

[0] http://clang.llvm.org/docs/SanitizerSpecialCaseList.html

-- Dean

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

Jacob Carlborg via cfe-dev
Just as an update, I have a patch under review for this functionality: https://reviews.llvm.org/D30388

Thoughts would be most appreciated.

Cheers

> On 23 Feb 2017, at 12:18, Dean Michael Berris <[hidden email]> wrote:
>
> TL;DR: Adding the [[clang::xray_{always,never}_instrument]] attribute in the declaration/definition of a function is cumbersome for guaranteed instrumentation (or non-instrumentation) of specific functions. We'd like to make the imbuing of this particular attribute a command-line controllable option.
>
> Background
> ==========
>
> XRay is a function call tracing system that's implemented in LLVM. One part of this system involves enabling users to use a source-level attribute to mark that a function is always instrumented with XRay when building with the `-fxray-instrument` flag enabled. While this is sufficient for some users, it's really inconvenient to have to manually add this attribute to functions that need to be marked with this attribute. Sometimes the decision on whether a function must always be instrumented isn't always known a-priori and could be the result of (or driver for) experimental methods (i.e. building with all functions instrumented, and in next iterations pruning the list down or changing thresholds, etc).
>
> Proposal
> =======
>
> We would like to add two flags (`-fxray-always-instrument=` and `-fxray-never-instrument=`) to clang that takes filenames processed similar to how the SanitizerSpecialCaseList [0] feature is handled. We apply first the blacklist, then the whitelist to determine whether a certain functions should be either never instrumented (in the blacklist) or always instrumented (in the whitelist). This allows users to define sets of functions that should be either always or never instrumented with XRay through command-line options.
>
> These lists will only take effect when `-fxray-instrument` is defined when invoking Clang. Currently, we determine whether instrumentation is necessary based on the combination of attributes and whether the instruction count threshold is met (see more at http://llvm.org/docs/XRay.html).
>
> The intent is to have a single class (XRayFunctionFilter) which maintains the special case lists to determine whether to always instrument a function or never instrument a function. We add an instance of class to the ASTContext, and when code-genning function declarations that fall in whitelist will get the "xray_always_instrument" attribute, and if they fall in the blacklist (and not the whitelist) will get the "xray_never_instrument" attribute. The strawman API for determining whether to always or never instrument is as follows:
>
> class XRayFunctionFilter {
> public:
>  enum class ImbueAttribute {
>    NONE, ALWAYS, NEVER, …
>  }
>
>  ImbueAttribute shouldImbueAttribute(StringRef FunctionName) const;
> private:
>  ...
> };
>
> An open question is what to do to functions that have the attribute explicitly defined in the source -- whether the whitelist/blacklist overrides those. I'm inclined to think that we should preserve the source-level attribute, but am willing to go either way.
>
> We also have work on-going to enable logging some arguments to function calls (https://reviews.llvm.org/D29704). In these cases we can use the whitelist to also say whether to imbue the attribute using the categories feature in the SanitizerSpecialCaseList file format.
>
> Does the community think this is something worth exploring?
>
> A "natural" extension of this feature would be to just support compile-time attribution of any attribute to any function/class/<insert construct here>. While that might be a more general approach, it seems a bit more invasive and larger a scope than the targeted proposal being made. We aren't opposed per-se to doing this general approach, and believe that the targeted approach we're proposing might inform the design of the more general approach later on.
>
> Thoughts?
>
> [0] http://clang.llvm.org/docs/SanitizerSpecialCaseList.html
>
> -- Dean
>

-- Dean

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

Jacob Carlborg via cfe-dev
In reply to this post by Jacob Carlborg via cfe-dev
This sounds totally reasonable.

I have vague memories that there was some demand for a generic way to slap attributes on decls without changing the source code, but I don't remember who wanted that. I thought it had something to do with static analysis, so maybe Jordan or Anna know?

It feels premature to me to try to build out that functionality when we already have special case lists implemented for such a similar use case (sanitizer blacklists), so I'm in favor of moving forward with those.

On Wed, Feb 22, 2017 at 5:18 PM, Dean Michael Berris via cfe-dev <[hidden email]> wrote:
TL;DR: Adding the [[clang::xray_{always,never}_instrument]] attribute in the declaration/definition of a function is cumbersome for guaranteed instrumentation (or non-instrumentation) of specific functions. We'd like to make the imbuing of this particular attribute a command-line controllable option.

Background
==========

XRay is a function call tracing system that's implemented in LLVM. One part of this system involves enabling users to use a source-level attribute to mark that a function is always instrumented with XRay when building with the `-fxray-instrument` flag enabled. While this is sufficient for some users, it's really inconvenient to have to manually add this attribute to functions that need to be marked with this attribute. Sometimes the decision on whether a function must always be instrumented isn't always known a-priori and could be the result of (or driver for) experimental methods (i.e. building with all functions instrumented, and in next iterations pruning the list down or changing thresholds, etc).

Proposal
=======

We would like to add two flags (`-fxray-always-instrument=` and `-fxray-never-instrument=`) to clang that takes filenames processed similar to how the SanitizerSpecialCaseList [0] feature is handled. We apply first the blacklist, then the whitelist to determine whether a certain functions should be either never instrumented (in the blacklist) or always instrumented (in the whitelist). This allows users to define sets of functions that should be either always or never instrumented with XRay through command-line options.

These lists will only take effect when `-fxray-instrument` is defined when invoking Clang. Currently, we determine whether instrumentation is necessary based on the combination of attributes and whether the instruction count threshold is met (see more at http://llvm.org/docs/XRay.html).

The intent is to have a single class (XRayFunctionFilter) which maintains the special case lists to determine whether to always instrument a function or never instrument a function. We add an instance of class to the ASTContext, and when code-genning function declarations that fall in whitelist will get the "xray_always_instrument" attribute, and if they fall in the blacklist (and not the whitelist) will get the "xray_never_instrument" attribute. The strawman API for determining whether to always or never instrument is as follows:

class XRayFunctionFilter {
public:
  enum class ImbueAttribute {
    NONE, ALWAYS, NEVER, …
  }

  ImbueAttribute shouldImbueAttribute(StringRef FunctionName) const;
private:
  ...
};

An open question is what to do to functions that have the attribute explicitly defined in the source -- whether the whitelist/blacklist overrides those. I'm inclined to think that we should preserve the source-level attribute, but am willing to go either way.

We also have work on-going to enable logging some arguments to function calls (https://reviews.llvm.org/D29704). In these cases we can use the whitelist to also say whether to imbue the attribute using the categories feature in the SanitizerSpecialCaseList file format.

Does the community think this is something worth exploring?

A "natural" extension of this feature would be to just support compile-time attribution of any attribute to any function/class/<insert construct here>. While that might be a more general approach, it seems a bit more invasive and larger a scope than the targeted proposal being made. We aren't opposed per-se to doing this general approach, and believe that the targeted approach we're proposing might inform the design of the more general approach later on.

Thoughts?

[0] http://clang.llvm.org/docs/SanitizerSpecialCaseList.html

-- Dean

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

Jacob Carlborg via cfe-dev


On Wed, Mar 1, 2017 at 2:25 PM, Reid Kleckner <[hidden email]> wrote:
This sounds totally reasonable.

I have vague memories that there was some demand for a generic way to slap attributes on decls without changing the source code, but I don't remember who wanted that. I thought it had something to do with static analysis, so maybe Jordan or Anna know?

Correct. We have implemented such system for clang. We call it API Notes. It currently lives in the out-of-tree clang used by the Swift compiler (http://github.com/apple/swift-clang). It is not complete, for example, it lacks C++ support, but we've been using it in production for several years now.

Doug has sent out an email about this and other out-of-tree changes trying to figure out if the community has interest in these additions. Here is the thread, it mainly discusses API Notes:

 
Cheers,
Anna

It feels premature to me to try to build out that functionality when we already have special case lists implemented for such a similar use case (sanitizer blacklists), so I'm in favor of moving forward with those.

On Wed, Feb 22, 2017 at 5:18 PM, Dean Michael Berris via cfe-dev <[hidden email]> wrote:
TL;DR: Adding the [[clang::xray_{always,never}_instrument]] attribute in the declaration/definition of a function is cumbersome for guaranteed instrumentation (or non-instrumentation) of specific functions. We'd like to make the imbuing of this particular attribute a command-line controllable option.

Background
==========

XRay is a function call tracing system that's implemented in LLVM. One part of this system involves enabling users to use a source-level attribute to mark that a function is always instrumented with XRay when building with the `-fxray-instrument` flag enabled. While this is sufficient for some users, it's really inconvenient to have to manually add this attribute to functions that need to be marked with this attribute. Sometimes the decision on whether a function must always be instrumented isn't always known a-priori and could be the result of (or driver for) experimental methods (i.e. building with all functions instrumented, and in next iterations pruning the list down or changing thresholds, etc).

Proposal
=======

We would like to add two flags (`-fxray-always-instrument=` and `-fxray-never-instrument=`) to clang that takes filenames processed similar to how the SanitizerSpecialCaseList [0] feature is handled. We apply first the blacklist, then the whitelist to determine whether a certain functions should be either never instrumented (in the blacklist) or always instrumented (in the whitelist). This allows users to define sets of functions that should be either always or never instrumented with XRay through command-line options.

These lists will only take effect when `-fxray-instrument` is defined when invoking Clang. Currently, we determine whether instrumentation is necessary based on the combination of attributes and whether the instruction count threshold is met (see more at http://llvm.org/docs/XRay.html).

The intent is to have a single class (XRayFunctionFilter) which maintains the special case lists to determine whether to always instrument a function or never instrument a function. We add an instance of class to the ASTContext, and when code-genning function declarations that fall in whitelist will get the "xray_always_instrument" attribute, and if they fall in the blacklist (and not the whitelist) will get the "xray_never_instrument" attribute. The strawman API for determining whether to always or never instrument is as follows:

class XRayFunctionFilter {
public:
  enum class ImbueAttribute {
    NONE, ALWAYS, NEVER, …
  }

  ImbueAttribute shouldImbueAttribute(StringRef FunctionName) const;
private:
  ...
};

An open question is what to do to functions that have the attribute explicitly defined in the source -- whether the whitelist/blacklist overrides those. I'm inclined to think that we should preserve the source-level attribute, but am willing to go either way.

We also have work on-going to enable logging some arguments to function calls (https://reviews.llvm.org/D29704). In these cases we can use the whitelist to also say whether to imbue the attribute using the categories feature in the SanitizerSpecialCaseList file format.

Does the community think this is something worth exploring?

A "natural" extension of this feature would be to just support compile-time attribution of any attribute to any function/class/<insert construct here>. While that might be a more general approach, it seems a bit more invasive and larger a scope than the targeted proposal being made. We aren't opposed per-se to doing this general approach, and believe that the targeted approach we're proposing might inform the design of the more general approach later on.

Thoughts?

[0] http://clang.llvm.org/docs/SanitizerSpecialCaseList.html

-- Dean

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

Jacob Carlborg via cfe-dev
In reply to this post by Jacob Carlborg via cfe-dev

> On 2 Mar 2017, at 09:25, Reid Kleckner <[hidden email]> wrote:
>
> This sounds totally reasonable.
>
> I have vague memories that there was some demand for a generic way to slap attributes on decls without changing the source code, but I don't remember who wanted that. I thought it had something to do with static analysis, so maybe Jordan or Anna know?
>
> It feels premature to me to try to build out that functionality when we already have special case lists implemented for such a similar use case (sanitizer blacklists), so I'm in favor of moving forward with those.
>

Thanks Reid!

I agree, I think we can build upon this later on in case we want a more generic utility for imbuing attributes through a side channel.

-- Dean

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

Jacob Carlborg via cfe-dev
In reply to this post by Jacob Carlborg via cfe-dev

> On 2 Mar 2017, at 09:40, Anna Zaks <[hidden email]> wrote:
>
>
>
> On Wed, Mar 1, 2017 at 2:25 PM, Reid Kleckner <[hidden email]> wrote:
> This sounds totally reasonable.
>
> I have vague memories that there was some demand for a generic way to slap attributes on decls without changing the source code, but I don't remember who wanted that. I thought it had something to do with static analysis, so maybe Jordan or Anna know?
>
> Correct. We have implemented such system for clang. We call it API Notes. It currently lives in the out-of-tree clang used by the Swift compiler (http://github.com/apple/swift-clang). It is not complete, for example, it lacks C++ support, but we've been using it in production for several years now.
>
> Doug has sent out an email about this and other out-of-tree changes trying to figure out if the community has interest in these additions. Here is the thread, it mainly discusses API Notes:
>
> http://lists.llvm.org/pipermail/cfe-dev/2015-December/046335.html

Thanks for the pointer Anna -- I'll go have a read about this.

Maybe later on we can generalise based on experience from swift to inform the design/use-cases if this will ever be done in clang.

Cheers

-- Dean

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

Jacob Carlborg via cfe-dev


On Thu, Mar 2, 2017 at 4:55 AM, Dean Michael Berris <[hidden email]> wrote:

> On 2 Mar 2017, at 09:40, Anna Zaks <[hidden email]> wrote:
>
>
>
> On Wed, Mar 1, 2017 at 2:25 PM, Reid Kleckner <[hidden email]> wrote:
> This sounds totally reasonable.
>
> I have vague memories that there was some demand for a generic way to slap attributes on decls without changing the source code, but I don't remember who wanted that. I thought it had something to do with static analysis, so maybe Jordan or Anna know?
>
> Correct. We have implemented such system for clang. We call it API Notes. It currently lives in the out-of-tree clang used by the Swift compiler (http://github.com/apple/swift-clang). It is not complete, for example, it lacks C++ support, but we've been using it in production for several years now.
>
> Doug has sent out an email about this and other out-of-tree changes trying to figure out if the community has interest in these additions. Here is the thread, it mainly discusses API Notes:
>
> http://lists.llvm.org/pipermail/cfe-dev/2015-December/046335.html

Thanks for the pointer Anna -- I'll go have a read about this.

Maybe later on we can generalise based on experience from swift to inform the design/use-cases if this will ever be done in clang.


Just to be 100% clear, API Notes is a clang feature it's used by the Swift project to support interoperability between Swift and C/ObjC. (For example, to store knowledge about how C APIs should be imported into Swift.) The Swift project uses a clone of llvm/clang. The clone is auto-synced with top-of-tree llvm/clang but contains several features that are currently only used by the Swift project.

Here is the description of API Notes from Doug's email:
API notes solve a not-uncommon problem: we invent some new Clang attribute that would be beneficial to add to some declarations in system headers (e.g., adding a ‘noreturn’ attribute to the C ‘exit’ function), but we can’t go around and fix all of the system headers everywhere. With API notes, we can write a separate YAML file that states that we want to add ‘noreturn’ to the ‘exit’ function: when we feed that YAML file into Clang as part of normal compilation (via a command-line option), Clang will add ‘noreturn’ to the ‘exit’ function when it parses the declaration of ‘exit’. Personally, I don’t like API notes—even with our optimizations, it’s inefficient in compile time and it takes the “truth” out of the headers—but I can see the wider use cases. If the Clang community wants this feature, I can prepare a proper proposal; if not, we’ll keep this code in the Swift clone of Clang and delete it if Swift ever stops needing it.
I think API notes can be used by other clients in clang, such a the static analyzer. It seems that it would be directly applicable to the scenario you describe as well. If so, I would propose to merge API Notes into mainline clang.

Even though Doug mentioned (in Dec 2015) that the Swift clone might delete this functionality in the future if API Notes are not needed any more, in practice, we see that the Swift project use cases for API Notes are expanding not shrinking.
Cheers

-- Dean



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

Jacob Carlborg via cfe-dev

> On 3 Mar 2017, at 04:14, Anna Zaks <[hidden email]> wrote:
>
> Just to be 100% clear, API Notes is a clang feature it's used by the Swift project to support interoperability between Swift and C/ObjC. (For example, to store knowledge about how C APIs should be imported into Swift.) The Swift project uses a clone of llvm/clang. The clone is auto-synced with top-of-tree llvm/clang but contains several features that are currently only used by the Swift project.
>

I think this is what I meant -- if they're in the clone of clang used in the Swift project, if (or when?) it makes it back upstream we can think about maybe also using it. I think it's very similar to what the sanitisers and XRay need for imbuing attributes with the special case list, without having to use YAML or something more structured than the simple text files.

> Here is the description of API Notes from Doug's email:
> API notes solve a not-uncommon problem: we invent some new Clang attribute that would be beneficial to add to some declarations in system headers (e.g., adding a ‘noreturn’ attribute to the C ‘exit’ function), but we can’t go around and fix all of the system headers everywhere. With API notes, we can write a separate YAML file that states that we want to add ‘noreturn’ to the ‘exit’ function: when we feed that YAML file into Clang as part of normal compilation (via a command-line option), Clang will add ‘noreturn’ to the ‘exit’ function when it parses the declaration of ‘exit’. Personally, I don’t like API notes—even with our optimizations, it’s inefficient in compile time and it takes the “truth” out of the headers—but I can see the wider use cases. If the Clang community wants this feature, I can prepare a proper proposal; if not, we’ll keep this code in the Swift clone of Clang and delete it if Swift ever stops needing it.
> I think API notes can be used by other clients in clang, such a the static analyzer. It seems that it would be directly applicable to the scenario you describe as well. If so, I would propose to merge API Notes into mainline clang.
>
> Even though Doug mentioned (in Dec 2015) that the Swift clone might delete this functionality in the future if API Notes are not needed any more, in practice, we see that the Swift project use cases for API Notes are expanding not shrinking.

That's encouraging!

I could also imagine a means of doing a pass at the LLVM level that might be used to imbue attributes based on information gained in other passes (or even externally). One thing that some users of XRay have asked is whether it's possible to statically do a walk of the call graph and selectively do instrumentation -- i.e. remove instrumentation from other parts of the binary and only focus on the code paths that we can prove statically will be calling a specific set of functions. This could work just on the LLVM level, but would also work on the front-end in cases where the choice between "always" and "never" instrument are being made at that level.

Anyway, API Notes sounds like an interesting approach for the generic case. I'm sure other projects would find interesting uses of this when it makes it in clang proper. :)

Cheers

-- Dean

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [XRay] RFC: Adding -fxray-{always, never}-instrument=... to Clang

Jacob Carlborg via cfe-dev


On Fri, Mar 3, 2017 at 3:21 AM, Dean Michael Berris <[hidden email]> wrote:

> On 3 Mar 2017, at 04:14, Anna Zaks <[hidden email]> wrote:
>
> Just to be 100% clear, API Notes is a clang feature it's used by the Swift project to support interoperability between Swift and C/ObjC. (For example, to store knowledge about how C APIs should be imported into Swift.) The Swift project uses a clone of llvm/clang. The clone is auto-synced with top-of-tree llvm/clang but contains several features that are currently only used by the Swift project.
>

I think this is what I meant -- if they're in the clone of clang used in the Swift project, if (or when?) it makes it back upstream we can think about maybe also using it. I think it's very similar to what the sanitisers and XRay need for imbuing attributes with the special case list, without having to use YAML or something more structured than the simple text files.

> Here is the description of API Notes from Doug's email:
> API notes solve a not-uncommon problem: we invent some new Clang attribute that would be beneficial to add to some declarations in system headers (e.g., adding a ‘noreturn’ attribute to the C ‘exit’ function), but we can’t go around and fix all of the system headers everywhere. With API notes, we can write a separate YAML file that states that we want to add ‘noreturn’ to the ‘exit’ function: when we feed that YAML file into Clang as part of normal compilation (via a command-line option), Clang will add ‘noreturn’ to the ‘exit’ function when it parses the declaration of ‘exit’. Personally, I don’t like API notes—even with our optimizations, it’s inefficient in compile time and it takes the “truth” out of the headers—but I can see the wider use cases. If the Clang community wants this feature, I can prepare a proper proposal; if not, we’ll keep this code in the Swift clone of Clang and delete it if Swift ever stops needing it.
> I think API notes can be used by other clients in clang, such a the static analyzer. It seems that it would be directly applicable to the scenario you describe as well. If so, I would propose to merge API Notes into mainline clang.
>
> Even though Doug mentioned (in Dec 2015) that the Swift clone might delete this functionality in the future if API Notes are not needed any more, in practice, we see that the Swift project use cases for API Notes are expanding not shrinking.

That's encouraging!

I could also imagine a means of doing a pass at the LLVM level that might be used to imbue attributes based on information gained in other passes (or even externally). One thing that some users of XRay have asked is whether it's possible to statically do a walk of the call graph and selectively do instrumentation -- i.e. remove instrumentation from other parts of the binary and only focus on the code paths that we can prove statically will be calling a specific set of functions. This could work just on the LLVM level, but would also work on the front-end in cases where the choice between "always" and "never" instrument are being made at that level.

This sounds very useful! For example, it could allow sanitizers or fuzzing to focus on newly updated code or code viewed as important for other reasons.
 

Anyway, API Notes sounds like an interesting approach for the generic case. I'm sure other projects would find interesting uses of this when it makes it in clang proper. :)

If/When you think API Notes will be useful for your cause, let us know. We can upstream them at any time. One of the main reasons they are not upstreamed already is that there are no users who are ready to immediately utilize them. (The static analyzer could definitely benefit from them, but no-one is actively working on using external annotations in the analyzer right now.)
 
 
Cheers

-- Dean



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Loading...