Using clang::tooling::RefactoringTool on cross-compiled code with different /usr/include

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Using clang::tooling::RefactoringTool on cross-compiled code with different /usr/include

Tom Stellard via cfe-dev
Hello everyone! It seems that my previous email got tumbled away as a html attachment, so I've removed all the formating in hope that helps.

I am writing a tool using clang's  tooling::RefactoringTool  and want to apply it to ARM targeted sources. Those rely on their own sysroot to compile, which is incomplete and basically contains only lib and usr directories with proprietary stuff. Its  /usr/include  headers also differ from my system ones - so when parsing my sources with clang tool I get a lot of parsing errors related to header ordering, etc. And of course, with vendor-provided gcc4.8 everything is fine, which led me to a conclusion there's something wrong with how I use clang.

I've tried to surpass the problem by copying my  /lib/clang/5.0.1/include  to this sysroot (so clang builtin headers would work correctly with new include & lib) and changing my  compile_commands.json  to use  -nostdinc++ -isystem /arm-hisiv300-linux/target/usr/include -isystem /arm-hisiv300-linux/gcc/include  - so clang wouldn't rely on system headers.

Unforntunately it doesn't fool clang - it still tries to search for some headers in my host system:

$ arm-hisiv300-linux/bin/my-tool src/grabcommon/rtp_assemblers.cpp
In file included from src/grabcommon/rtp_assemblers.cpp:1:
...
In file included from /arm-hisiv300-linux/target/usr/include/limits.h:125: <- here #include_next is used to access compiler-specific <limits.h>
In file included from /arm-hisiv300-linux/bin/../lib/clang/5.0.1/include/limits.h:37: <- here clang #include_next <limits.h>
>>> /usr/include/limits.h:145:5 <<< error: function-like macro '__GLIBC_USE' is not defined <- here I'd expect /arm-hisiv300-linux/target/usr/include/limits.h, but get system one
#if __GLIBC_USE (IEC_60559_BFP_EXT)

I believe I have to pass "sysroot" directory as an argument to my tool and somehow construct RefactoringTool / CompilationDatabase instance so it take advantage of needed include directory, but how? Or am I entirely wrong and this calls for drastic measures like recompiling clang/LLVM for my needs? I am aware that I can  mount --bind my/usr/include /usr/include  (it works!), or put everything to some arm-based distro and use qemu, so instead of cross-compiling I just compile in place. However my intention is that the tool could be used as simple as  $ my-tool --sysroot /dir file.cpp . So what are my options?
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using clang::tooling::RefactoringTool on cross-compiled code with different /usr/include

Tom Stellard via cfe-dev
just in case, 
I'm wondering if your compilation commands explicitly contain 
--sysroot or -isysroot, i.e. see 
(these are gcc docs, but clang supports these options (since clang tries to be a drop-in replacement for gcc).
Kind regards,
Alexander Shaposhnikov

On Fri, Mar 23, 2018 at 12:52 AM, Иванов Игорь via cfe-dev <[hidden email]> wrote:
Hello everyone! It seems that my previous email got tumbled away as a html attachment, so I've removed all the formating in hope that helps.

I am writing a tool using clang's  tooling::RefactoringTool  and want to apply it to ARM targeted sources. Those rely on their own sysroot to compile, which is incomplete and basically contains only lib and usr directories with proprietary stuff. Its  /usr/include  headers also differ from my system ones - so when parsing my sources with clang tool I get a lot of parsing errors related to header ordering, etc. And of course, with vendor-provided gcc4.8 everything is fine, which led me to a conclusion there's something wrong with how I use clang.

I've tried to surpass the problem by copying my  /lib/clang/5.0.1/include  to this sysroot (so clang builtin headers would work correctly with new include & lib) and changing my  compile_commands.json  to use  -nostdinc++ -isystem /arm-hisiv300-linux/target/usr/include -isystem /arm-hisiv300-linux/gcc/include  - so clang wouldn't rely on system headers.

Unforntunately it doesn't fool clang - it still tries to search for some headers in my host system:

$ arm-hisiv300-linux/bin/my-tool src/grabcommon/rtp_assemblers.cpp
In file included from src/grabcommon/rtp_assemblers.cpp:1:
...
In file included from /arm-hisiv300-linux/target/usr/include/limits.h:125: <- here #include_next is used to access compiler-specific <limits.h>
In file included from /arm-hisiv300-linux/bin/../lib/clang/5.0.1/include/limits.h:37: <- here clang #include_next <limits.h>
>>> /usr/include/limits.h:145:5 <<< error: function-like macro '__GLIBC_USE' is not defined <- here I'd expect /arm-hisiv300-linux/target/usr/include/limits.h, but get system one
#if __GLIBC_USE (IEC_60559_BFP_EXT)

I believe I have to pass "sysroot" directory as an argument to my tool and somehow construct RefactoringTool / CompilationDatabase instance so it take advantage of needed include directory, but how? Or am I entirely wrong and this calls for drastic measures like recompiling clang/LLVM for my needs? I am aware that I can  mount --bind my/usr/include /usr/include  (it works!), or put everything to some arm-based distro and use qemu, so instead of cross-compiling I just compile in place. However my intention is that the tool could be used as simple as  $ my-tool --sysroot /dir file.cpp . So what are my options?
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using clang::tooling::RefactoringTool on cross-compiled code with different /usr/include

Tom Stellard via cfe-dev
No, compilation commands contain only -isystem (to 3rd party libs) and -I (to project include) directories, aside from compilation flags.
 
I've seen --sysroot option available when compiling with clang, however didn't yet find a clue how to pass it to libTooling internals.
 
23.03.2018, 11:02, "Александр Шапошников" <[hidden email]>:
just in case, 
I'm wondering if your compilation commands explicitly contain 
--sysroot or -isysroot, i.e. see 
(these are gcc docs, but clang supports these options (since clang tries to be a drop-in replacement for gcc).
Kind regards,
Alexander Shaposhnikov
 
On Fri, Mar 23, 2018 at 12:52 AM, Иванов Игорь via cfe-dev <[hidden email]> wrote:
Hello everyone! It seems that my previous email got tumbled away as a html attachment, so I've removed all the formating in hope that helps.

I am writing a tool using clang's  tooling::RefactoringTool  and want to apply it to ARM targeted sources. Those rely on their own sysroot to compile, which is incomplete and basically contains only lib and usr directories with proprietary stuff. Its  /usr/include  headers also differ from my system ones - so when parsing my sources with clang tool I get a lot of parsing errors related to header ordering, etc. And of course, with vendor-provided gcc4.8 everything is fine, which led me to a conclusion there's something wrong with how I use clang.

I've tried to surpass the problem by copying my  /lib/clang/5.0.1/include  to this sysroot (so clang builtin headers would work correctly with new include & lib) and changing my  compile_commands.json  to use  -nostdinc++ -isystem /arm-hisiv300-linux/target/usr/include -isystem /arm-hisiv300-linux/gcc/include  - so clang wouldn't rely on system headers.

Unforntunately it doesn't fool clang - it still tries to search for some headers in my host system:

$ arm-hisiv300-linux/bin/my-tool src/grabcommon/rtp_assemblers.cpp
In file included from src/grabcommon/rtp_assemblers.cpp:1:
...
In file included from /arm-hisiv300-linux/target/usr/include/limits.h:125: <- here #include_next is used to access compiler-specific <limits.h>
In file included from /arm-hisiv300-linux/bin/../lib/clang/5.0.1/include/limits.h:37: <- here clang #include_next <limits.h>
>>> /usr/include/limits.h:145:5 <<< error: function-like macro '__GLIBC_USE' is not defined <- here I'd expect /arm-hisiv300-linux/target/usr/include/limits.h, but get system one
#if __GLIBC_USE (IEC_60559_BFP_EXT)

I believe I have to pass "sysroot" directory as an argument to my tool and somehow construct RefactoringTool / CompilationDatabase instance so it take advantage of needed include directory, but how? Or am I entirely wrong and this calls for drastic measures like recompiling clang/LLVM for my needs? I am aware that I can  mount --bind my/usr/include /usr/include  (it works!), or put everything to some arm-based distro and use qemu, so instead of cross-compiling I just compile in place. However my intention is that the tool could be used as simple as  $ my-tool --sysroot /dir file.cpp . So what are my options?
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using clang::tooling::RefactoringTool on cross-compiled code with different /usr/include

Tom Stellard via cfe-dev
In reply to this post by Tom Stellard via cfe-dev
On 23/03/18 09:21, Иванов Игорь via cfe-dev wrote:

 > I am writing a tool using clang's  tooling::RefactoringTool  and want
 > to apply it to ARM targeted sources. Those rely on their own sysroot
 > to compile, which is incomplete and basically contains only lib and
 > usr directories with proprietary stuff. Its  /usr/include  headers
 > also differ from my system ones - so when parsing my sources with
 > clang tool I get a lot of parsing errors related to header ordering,
 > etc. And of course, with vendor-provided gcc4.8 everything is fine,
 > which led me to a conclusion there's something wrong with how I use
 > clang.

Hi,

I've been working on a project that involves writing clang refactoring
and analysis tools and we had similar problems.  It's possible to ignore
the compilation database and pass arguments directly to the clang front
end like this:

   your-tool <tool arguments> -- -std=c++14 -DDEBUG
-I/home/someone/include -I/some/other/directory

Everything after the -- is used by the clang front end when it's parsing
your source files; I think you may be able to use this to solve your
problem.

When you use -- I think it ignores the compilation database and just
uses whatever you supply on the command line.  We've found this to be
considerably more convenient than using compilation databases.
Unfortunately this feature doesn't seem to be very well publicised: I
think we only discovered it by looking at the Clang sources...

Best wishes,

Kenneth MacKenzie
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev