How to use clang-tidy for smaller projects

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

How to use clang-tidy for smaller projects

Kristof Beyls via cfe-dev

All,

tl;dr is there a way to use clang-tidy for small projects without requiring users to rebuild most of clang/llvm provided versions match?

Let me begin by offering my gratitude to the Clang/LLVM community for the excellent C/C++ tools you all develop and we all benefit from.

I'm a grad student who works on a number of small libraries that use C/C++, and occasionally me or my colleagues change the APIs for those libraries, and would like to provide tools to update to the new APIs to our users.  As I understand there are two main options:

1. Write a new custom clang-tool to do the migrations

2. Reuse clang-tidy and write a new tidy-module or add our check to an existing module.

The former has some disadvantages: it means that users now need to run our tool and clang-tidy to get all of the fixes they need. Additionally what we are doing with the migrations are essentially what clang-tidy was designed to do.

The latter addresses these challenges, but presents a few more. Now users have to compile and install a custom version of clang-tidy in order to use our checks, and for various reasons the user probably will compile and install a fair bit of the clang/llvm ecosystem just to try our migration modules. LLVM/clang/clang-tools-extra are not small projects and take a while to compile and occupy substantial disk space once compiled for a task like this.

I considered trying to write a module external to the LLVM/Clang source tree and just link to the appropriate libraries in as I have done with clang-tools in the past, but clang-tidy uses several implementation specific headers that are not available outside the clang-tools-extra source code repository (i.e. ClangTidy.h).  I recognize that this is not the [recommended method for developing clang-tidy modules][1] so it doesn't surprise me that this does not work.

However, I think there is a use case here.  Not every project is backed by Google, Intel, Apple, Sony, or another large company with an extensive distributed build environment where rebuilding a large amount of clang/llvm is more feasible.  Not every clang-tidy module needs to be up-streamed to clang-tidy especially those for small libraries like mine.

I admit it is quite possible I am missing something, but I think there should be a way to make this easier for small projects to use.  I've read the provided documentation several times, and I've not found it.

I would appreciate your time and consideration as you consider this request.

Respectfully,

Robert Underwood


[1]: https://clang.llvm.org/extra/clang-tidy/Contributing.html


_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to use clang-tidy for smaller projects

Kristof Beyls via cfe-dev
I wonder if enough of clang tidy functionality is exposed through libclang and python that you could do some clang-tidy-lite work with python tooling?

I had some success extending libclang to power a language bindings generator (like SWIG)

On 26 Oct 2019, at 14:13, Robert Underwood via cfe-dev <[hidden email]> wrote:



All,

tl;dr is there a way to use clang-tidy for small projects without requiring users to rebuild most of clang/llvm provided versions match?

Let me begin by offering my gratitude to the Clang/LLVM community for the excellent C/C++ tools you all develop and we all benefit from.

I'm a grad student who works on a number of small libraries that use C/C++, and occasionally me or my colleagues change the APIs for those libraries, and would like to provide tools to update to the new APIs to our users.  As I understand there are two main options:

1. Write a new custom clang-tool to do the migrations

2. Reuse clang-tidy and write a new tidy-module or add our check to an existing module.

The former has some disadvantages: it means that users now need to run our tool and clang-tidy to get all of the fixes they need. Additionally what we are doing with the migrations are essentially what clang-tidy was designed to do.

The latter addresses these challenges, but presents a few more. Now users have to compile and install a custom version of clang-tidy in order to use our checks, and for various reasons the user probably will compile and install a fair bit of the clang/llvm ecosystem just to try our migration modules. LLVM/clang/clang-tools-extra are not small projects and take a while to compile and occupy substantial disk space once compiled for a task like this.

I considered trying to write a module external to the LLVM/Clang source tree and just link to the appropriate libraries in as I have done with clang-tools in the past, but clang-tidy uses several implementation specific headers that are not available outside the clang-tools-extra source code repository (i.e. ClangTidy.h).  I recognize that this is not the [recommended method for developing clang-tidy modules][1] so it doesn't surprise me that this does not work.

However, I think there is a use case here.  Not every project is backed by Google, Intel, Apple, Sony, or another large company with an extensive distributed build environment where rebuilding a large amount of clang/llvm is more feasible.  Not every clang-tidy module needs to be up-streamed to clang-tidy especially those for small libraries like mine.

I admit it is quite possible I am missing something, but I think there should be a way to make this easier for small projects to use.  I've read the provided documentation several times, and I've not found it.

I would appreciate your time and consideration as you consider this request.

Respectfully,

Robert Underwood


[1]: https://clang.llvm.org/extra/clang-tidy/Contributing.html

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to use clang-tidy for smaller projects

Kristof Beyls via cfe-dev
In reply to this post by Kristof Beyls via cfe-dev
On 26/10/2019 14:13, Robert Underwood via cfe-dev wrote:
>
> I would appreciate your time and consideration as you consider this
> request.

This bug covers adding a plugin extension interface to clang-tidy:

  https://bugs.llvm.org//show_bug.cgi?id=32739

Aside from that, it is possible to write/generate bindings for the two
steps of clang-tidy (AST Matching followed by Rewriting). You can see an
exploration of that here:

  https://steveire.wordpress.com/2019/04/30/the-future-of-ast-matching

Thanks,

Stephen

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to use clang-tidy for smaller projects

Kristof Beyls via cfe-dev
In reply to this post by Kristof Beyls via cfe-dev
Stephen,

Thanks for your detailed response.

On 27/10/2019, 22:33, Stephen Kelly via cfe-dev wrote:
>This bug covers adding a plugin extension interface to clang-tidy:
> https://bugs.llvm.org//show_bug.cgi?id=32739

Alexander Kornienko's comment about "statically-linked-out-of-tree
checks" seems to be what I am looking for,  but that isn't really
documented anywhere that I can find.  I'm just not sure where to start
on this since several of the headers are internal to
clang-tools-extra.

I also recall that its possible to compile each of the llvm projects
separately, but I am unsure how to configure CMake the use the system
installed version of the libraries, or  even if my distribution
installs enough components to do that.  I use CMake from time to time,
but I have to admit I am a little intimidated by the 1330 different
CMake settings in the HEAD branch.  I also am pretty sure how to use
the system libraries for clang-tools-extra isn't documented anywhere
either.  That would get me most of the way towards the kind of
solution that I am looking for.

> Aside from that, it is possible to write/generate bindings for the two
> steps of clang-tidy (AST Matching followed by Rewriting). You can see an
> exploration of that here:
> https://steveire.wordpress.com/2019/04/30/the-future-of-ast-matching

These tools in your talk are amazing!  I would love to see them merged
upstream (both to LLVM and to CompilerExplorer).

Respectfully,
Robert Underwood
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to use clang-tidy for smaller projects

Kristof Beyls via cfe-dev
In reply to this post by Kristof Beyls via cfe-dev
Jonathan,

On Sat, Oct 26, 2019 at 3:06 PM Jonathan Coe <[hidden email]> wrote:
>
> I wonder if enough of clang tidy functionality is exposed through libclang and python that you could do some clang-tidy-lite work with python tooling?

libclang can do some of what I am looking for, but I would need to
reinvent the wheel for several of the tools provided by clang-tidy
such as re-writing support, compilation database support, astMatcher
support.  Not that I couldn't do that in a pinch, It would be
potentially substantially less dev-time to just use libtooling and
endure the long compile times.

Yes, I could use visitors to do most of what I want, but the matcher
API is substantially more intuitive to me.

Respectfully,
Robert
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to use clang-tidy for smaller projects

Kristof Beyls via cfe-dev
In reply to this post by Kristof Beyls via cfe-dev
Hi Robert,

On Mon, Oct 28, 2019 at 08:56:31AM -0500, Robert Underwood via cfe-dev <[hidden email]> wrote:
> Alexander Kornienko's comment about "statically-linked-out-of-tree
> checks" seems to be what I am looking for,  but that isn't really
> documented anywhere that I can find.  I'm just not sure where to start
> on this since several of the headers are internal to
> clang-tools-extra.

My understanding is that creating a libtooling-based standalone tool
that can analyze the AST and emit diagnostics / fixits is not something
that requires clang-tools-extra headers. See the tool template here:

https://github.com/llvm/llvm-project/tree/master/clang-tools-extra/tool-template

If that's too minimal, I while ago I also wrote a toy tool to play with
this, and came up with

https://github.com/vmiklos/vmexam/blob/master/llvm/libtooling/readability-avoid-auto.cxx

That just follows in-tree tools, but is a self-contained example in ~100
lines of code, including AST matchers and emitting fixits (see the
Makefile for an end-to-end demo).

Regards,

Miklos

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

signature.asc (201 bytes) Download Attachment