compile_commands.json for Windows/MSVC

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

compile_commands.json for Windows/MSVC

Richard-2
Some of the clang based tools, such as clang-check, require a
compile_commands.json file which is generated by CMake.  I tried
running Windows CMake to generate this file, but it issued an error
message and didn't write the file.

So I hacked one together using a Visual Studio project build log.  It
gets me by, but lots of warnings are spewed about unrecognized command
line arguments.

Does anyone else have a better method for generating these?  Maybe I
just need the right CMake incantation.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
     The Computer Graphics Museum <http://computergraphicsmuseum.org>
         The Terminals Wiki <http://terminals.classiccmp.org>
  Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Sean Silva-2
Make sure that you have the latest CMake and that you are generating projects as if clang was the compiler (i.e., not generating VS projects (although clang's cl-emulation is getting quite good, but I don't think it's quite there yet)).

-- Sean Silva

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Richard-2

In article <CAHnXoan3PPNyDF=[hidden email]>,
    Sean Silva <[hidden email]> writes:

> Make sure that you have the latest CMake

> "c:\Program Files (x86)\CMake 2.8"\bin\cmake --version
cmake version 2.8.11.2

> and that you are generating
> projects as if clang was the compiler

I followed the directions here:
<http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html>

and ran this command:

cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ../llvm

I get this message at the end:

CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_EXPORT_COMPILE_COMMANDS

...and sure enough, I don't have a compile_commands.json.

What did I do wrong?
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
     The Computer Graphics Museum <http://computergraphicsmuseum.org>
         The Terminals Wiki <http://terminals.classiccmp.org>
  Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Sean Silva-2



On Mon, Aug 12, 2013 at 8:53 PM, Richard <[hidden email]> wrote:

What did I do wrong?
 
What generator (-G) are you using?

-- Sean Silva

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Richard-2

In article <CAHnXoak2COeWg6J7e+sHH005+RfE0GC=[hidden email]>,
    Sean Silva <[hidden email]> writes:

> On Mon, Aug 12, 2013 at 8:53 PM, Richard <[hidden email]> wrote:
>
> > What did I do wrong?
> >
>
> What generator (-G) are you using?

When I originally created the build structure, I generated VS 2012
project files.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
     The Computer Graphics Museum <http://computergraphicsmuseum.org>
         The Terminals Wiki <http://terminals.classiccmp.org>
  Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Reid Kleckner-3
On Tue, Aug 13, 2013 at 12:08 AM, Richard <[hidden email]> wrote:
> What generator (-G) are you using?

When I originally created the build structure, I generated VS 2012
project files.

Searching the CMake source, I only see mentions of CMAKE_EXPORT_COMPILE_COMMANDS in the make and ninja generators.  The feature is probably unimplemented for Visual Studio project files.

ninja works fairly well with the MSVC toolchain so you could try that.

Even if you get the cl.exe command lines, clang tools won't understand them by default, and clang-cl is very new.  We haven't used with clang tools, and we haven't used it to build anything really big and complicated yet.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Richard-2

In article <CACs=ty+Bzw=[hidden email]>,
    Reid Kleckner <[hidden email]> writes:

> On Tue, Aug 13, 2013 at 12:08 AM, Richard <[hidden email]> wrote:
>
> > > What generator (-G) are you using?
> >
> > When I originally created the build structure, I generated VS 2012
> > project files.
>
>
> Searching the CMake source, I only see mentions of
> CMAKE_EXPORT_COMPILE_COMMANDS in the make and ninja generators.  The
> feature is probably unimplemented for Visual Studio project files.

OK, I'm a little confused.  The page that explains how to generate
this json file doesn't mention anything about generators or -G.

Are the directions on the page simply assuming somethign that isn't true
in this case?

> ninja works fairly well with the MSVC toolchain so you could try that.

I don't need to actually change the way I'm building the code -- I
still want VS projects for building the code.  I just need to create
this JSON file.

As I understand it, this JSON file feeds clang all the necessary
include search paths and relevant preprocessor symbol definitions in
order to properly interpret the source code in a particular
compilation context.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
     The Computer Graphics Museum <http://computergraphicsmuseum.org>
         The Terminals Wiki <http://terminals.classiccmp.org>
  Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Manuel Klimek
AFAIK nobody implemented json compile command output for Windows yet (back in the days when I implemented it, clang was so unusable on windows that nobody cared). So, currently, your approach with scrapping the compile commands from the visual studio make files seems a way to go.

The next thing we'll need is to be able to translate cl.exe command line options to clang's. One way to do this is to use the ArgumentAdjuster to replace the cl.exe flags with the corresponding clang flags - again, somebody would need to implement this.

If nobody beats us to it, at some point we'll most certainly get there, but currently, it's not too high on our priority list. Thus, patches welcome :)

Cheers,
/Manuel



On Tue, Aug 13, 2013 at 11:02 AM, Richard <[hidden email]> wrote:

In article <CACs=ty+Bzw=[hidden email]>,
    Reid Kleckner <[hidden email]> writes:

> On Tue, Aug 13, 2013 at 12:08 AM, Richard <[hidden email]> wrote:
>
> > > What generator (-G) are you using?
> >
> > When I originally created the build structure, I generated VS 2012
> > project files.
>
>
> Searching the CMake source, I only see mentions of
> CMAKE_EXPORT_COMPILE_COMMANDS in the make and ninja generators.  The
> feature is probably unimplemented for Visual Studio project files.

OK, I'm a little confused.  The page that explains how to generate
this json file doesn't mention anything about generators or -G.

Are the directions on the page simply assuming somethign that isn't true
in this case?

> ninja works fairly well with the MSVC toolchain so you could try that.

I don't need to actually change the way I'm building the code -- I
still want VS projects for building the code.  I just need to create
this JSON file.

As I understand it, this JSON file feeds clang all the necessary
include search paths and relevant preprocessor symbol definitions in
order to properly interpret the source code in a particular
compilation context.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
     The Computer Graphics Museum <http://computergraphicsmuseum.org>
         The Terminals Wiki <http://terminals.classiccmp.org>
  Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Chandler Carruth
On Tue, Aug 13, 2013 at 11:44 PM, Manuel Klimek <[hidden email]> wrote:
AFAIK nobody implemented json compile command output for Windows yet (back in the days when I implemented it, clang was so unusable on windows that nobody cared). So, currently, your approach with scrapping the compile commands from the visual studio make files seems a way to go.

As an alternative, you could add direct support to the tooling library for querying Visual Studio for the commandline if there is a reasonable IPC mechanism for doing that.

As another alternative, you could add a plugin to Visual Studio that caused it to write out the json database whenever the project settings changed... If there is a reasonable plugin layer to do that.

I would personally view both of these as strictly superior to teaching CMake to do more things as they'll also work for non-CMake projects on Windows. And if these aren't feasible, maybe some other solution that is more generic.

I also think that whatever such system you could build, that integrates cleanly on Windows, would be *very* welcome in the Clang repository, so patches most definitely welcome there. (As in, I don't know how to do this, but really really want it, so I'm crossing my fingers.)


The next thing we'll need is to be able to translate cl.exe command line options to clang's. One way to do this is to use the ArgumentAdjuster to replace the cl.exe flags with the corresponding clang flags - again, somebody would need to implement this.

I don't think this is the right approach. We should just finish the Clang driver's support that Reid and Hans have been working on. Honestly, it's already pretty close.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Kim Gräsman
On Wed, Aug 14, 2013 at 9:23 AM, Chandler Carruth <[hidden email]> wrote:

> On Tue, Aug 13, 2013 at 11:44 PM, Manuel Klimek <[hidden email]> wrote:
>>
>> AFAIK nobody implemented json compile command output for Windows yet (back
>> in the days when I implemented it, clang was so unusable on windows that
>> nobody cared). So, currently, your approach with scrapping the compile
>> commands from the visual studio make files seems a way to go.
>
> As an alternative, you could add direct support to the tooling library for
> querying Visual Studio for the commandline if there is a reasonable IPC
> mechanism for doing that.
>
> As another alternative, you could add a plugin to Visual Studio that caused
> it to write out the json database whenever the project settings changed...
> If there is a reasonable plugin layer to do that.

I'd prefer to parse the VS project files and spit out a compilation
database. They are plain XML and as of VS2010 (I think) supposed to be
backward compatible, so it's a stable format.

I could probably help with VS IPC if that's something you want to
pursue, but it's usually pretty painful and less stable than just
interpreting the project file format.

I agree that CMake should be kept out of this loop; there are LOTS of
Windows-only projects maintaining their build systems as Visual Studio
solutions.

- Kim
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Chandler Carruth

On Wed, Aug 14, 2013 at 1:57 AM, Kim Gräsman <[hidden email]> wrote:
> As an alternative, you could add direct support to the tooling library for
> querying Visual Studio for the commandline if there is a reasonable IPC
> mechanism for doing that.
>
> As another alternative, you could add a plugin to Visual Studio that caused
> it to write out the json database whenever the project settings changed...
> If there is a reasonable plugin layer to do that.

I'd prefer to parse the VS project files and spit out a compilation
database. They are plain XML and as of VS2010 (I think) supposed to be
backward compatible, so it's a stable format.

My problem with this is that it adds an extra step every time the project file changes. If you forget it, your tool begins to misbehave in potentially surprising ways.

Maybe you could make the logic involve parsing and spitting out, but embedding it into a plugin that gets notified when VS sees the project file change? (I clearly have no idea how this stuff works...)

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Kim Gräsman
On Wed, Aug 14, 2013 at 11:00 AM, Chandler Carruth <[hidden email]> wrote:

>
> On Wed, Aug 14, 2013 at 1:57 AM, Kim Gräsman <[hidden email]> wrote:
>>
>> I'd prefer to parse the VS project files and spit out a compilation
>> database. They are plain XML and as of VS2010 (I think) supposed to be
>> backward compatible, so it's a stable format.
>
> My problem with this is that it adds an extra step every time the project
> file changes. If you forget it, your tool begins to misbehave in potentially
> surprising ways.
>
> Maybe you could make the logic involve parsing and spitting out, but
> embedding it into a plugin that gets notified when VS sees the project file
> change? (I clearly have no idea how this stuff works...)

I see your concern. Yeah, it could probably be driven from a plug-in
of some kind, I don't know the details either. It thought it was messy
when I looked at it last a few years ago :-)

Isn't it just a dependency problem, though? If the modified time of
any vcxproj > the modified time of compile_commands.json it needs to
be regenerated? That way it would pick up changes that happen outside
of Visual Studio, e.g. a version control update, manual edit,
generated projects.

Actually, I started digging into Tooling yesterday, and maybe
compile_commands.json doesn't even need to be saved out. What if there
was code in Tooling to transform a Visual Studio solution into a
CompilationDatabase implementation? It would take longer to parse,
probably, but it should be a pretty direct translation.

- Kim

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Manuel Klimek
On Wed, Aug 14, 2013 at 2:10 AM, Kim Gräsman <[hidden email]> wrote:
On Wed, Aug 14, 2013 at 11:00 AM, Chandler Carruth <[hidden email]> wrote:
>
> On Wed, Aug 14, 2013 at 1:57 AM, Kim Gräsman <[hidden email]> wrote:
>>
>> I'd prefer to parse the VS project files and spit out a compilation
>> database. They are plain XML and as of VS2010 (I think) supposed to be
>> backward compatible, so it's a stable format.
>
> My problem with this is that it adds an extra step every time the project
> file changes. If you forget it, your tool begins to misbehave in potentially
> surprising ways.
>
> Maybe you could make the logic involve parsing and spitting out, but
> embedding it into a plugin that gets notified when VS sees the project file
> change? (I clearly have no idea how this stuff works...)

I see your concern. Yeah, it could probably be driven from a plug-in
of some kind, I don't know the details either. It thought it was messy
when I looked at it last a few years ago :-)

Isn't it just a dependency problem, though? If the modified time of
any vcxproj > the modified time of compile_commands.json it needs to
be regenerated? That way it would pick up changes that happen outside
of Visual Studio, e.g. a version control update, manual edit,
generated projects.

Actually, I started digging into Tooling yesterday, and maybe
compile_commands.json doesn't even need to be saved out. What if there
was code in Tooling to transform a Visual Studio solution into a
CompilationDatabase implementation? It would take longer to parse,
probably, but it should be a pretty direct translation.

I think that'd be basically the same as some IPC solution (and probably better, as msbuild seems pretty well defined / stable these days). Patches for a CompilationDatabase implementation that just reads from msbuild would be super-welcome :D

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Kim Gräsman
On Wed, Aug 14, 2013 at 11:22 AM, Manuel Klimek <[hidden email]> wrote:
>
> I think that'd be basically the same as some IPC solution (and probably
> better, as msbuild seems pretty well defined / stable these days). Patches
> for a CompilationDatabase implementation that just reads from msbuild would
> be super-welcome :D

For future reference, is there a basic XML parser available in the
LLVM/Clang tree?

- Kim
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Chandler Carruth
On Wed, Aug 14, 2013 at 11:02 AM, Kim Gräsman <[hidden email]> wrote:
On Wed, Aug 14, 2013 at 11:22 AM, Manuel Klimek <[hidden email]> wrote:
>
> I think that'd be basically the same as some IPC solution (and probably
> better, as msbuild seems pretty well defined / stable these days). Patches
> for a CompilationDatabase implementation that just reads from msbuild would
> be super-welcome :D

For future reference, is there a basic XML parser available in the
LLVM/Clang tree?

No, but Clang already has an opt-in dependency on libxml....
 

- Kim


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Sean Silva-2
In reply to this post by Richard-2



On Tue, Aug 13, 2013 at 11:02 AM, Richard <[hidden email]> wrote:
Are the directions on the page simply assuming somethign that isn't true
in this case?


They assume that you are using clang as the compiler for the project, which currently isn't going to be true for VS projects.

-- Sean Silva

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Richard-2
In reply to this post by Chandler Carruth

In article <[hidden email]>,
    Chandler Carruth <[hidden email]> writes:

> As an alternative, you could add direct support to the tooling library for
> querying Visual Studio for the commandline if there is a reasonable IPC
> mechanism for doing that.
>
> As another alternative, you could add a plugin to Visual Studio that caused
> it to write out the json database whenever the project settings changed...
> If there is a reasonable plugin layer to do that.

Both of these approaches are feasible with the first one probably
being easier.

> I would personally view both of these as strictly superior to teaching
> CMake to do more things as they'll also work for non-CMake projects on
> Windows. And if these aren't feasible, maybe some other solution that is
> more generic.

Yes, I would rather get a compile_commands.json for non-CMake projects
as well from Visual Studio.

> I don't think this is the right approach. We should just finish the Clang
> driver's support that Reid and Hans have been working on. Honestly, it's
> already pretty close.

Yeah, with my hack I get usable results from libTooling as is, just a
bunch of warnings about arguments not understood.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
     The Computer Graphics Museum <http://computergraphicsmuseum.org>
         The Terminals Wiki <http://terminals.classiccmp.org>
  Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

adrien.courdavault
Hello everyone.

I would like to know what is the status of this feature.
In fact I have been following clang on windows for some time.

i have installed the 3.4 version on Windows when it was released (thanks to you all clang developer for this BTW)
( FYI I could not find the manual of clang modernize for 3.4 )
ASAIK the driver for visual is done.

And I have Cmake 2.8.12.1 (latest version) I haven't seen anything in the changelogs for 2.8.12.2 neither 3.0 that is about this matter.

So I would like to know what I should do to get it working (I mean use the compile_commands.json with visual generator of cmake).

Thank you for your help
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Manuel Klimek
+Hans

I think this depends on how much of VC's command line arguments clang can already "eat"...


On Sun, Jan 12, 2014 at 2:15 PM, adrien.courdavault <[hidden email]> wrote:
Hello everyone.

I would like to know what is the status of this feature.
In fact I have been following clang on windows for some time.

i have installed the 3.4 version on Windows when it was released (thanks to
you all clang developer for this BTW)
( FYI I could not find the manual of clang modernize for 3.4 )
ASAIK the driver for visual is done.

And I have Cmake 2.8.12.1 (latest version) I haven't seen anything in the
changelogs for 2.8.12.2 neither 3.0 that is about this matter.

So I would like to know what I should do to get it working (I mean use the
compile_commands.json with visual generator of cmake).

Thank you for your help



--
View this message in context: http://clang-developers.42468.n3.nabble.com/compile-commands-json-for-Windows-MSVC-tp4033848p4037087.html
Sent from the Clang Developers mailing list archive at Nabble.com.
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: compile_commands.json for Windows/MSVC

Hans Wennborg
clang-cl should be able to "eat" all of them, but it will ignore a
bunch that are unsupported.

However, I don't think the clang tooling stuff is hooked up to
understand "cl-mode" arguments, since AFAIK, it doesn't invoke the
driver in the same way (but I could be wrong).

It might not be hard to make the tooling understand that the commands
from a compilation database should be interpreted in cl-mode, but I
haven't looked into it.

 - Hans

On Sun, Jan 12, 2014 at 9:17 AM, Manuel Klimek <[hidden email]> wrote:

> +Hans
>
> I think this depends on how much of VC's command line arguments clang can
> already "eat"...
>
>
> On Sun, Jan 12, 2014 at 2:15 PM, adrien.courdavault
> <[hidden email]> wrote:
>>
>> Hello everyone.
>>
>> I would like to know what is the status of this feature.
>> In fact I have been following clang on windows for some time.
>>
>> i have installed the 3.4 version on Windows when it was released (thanks
>> to
>> you all clang developer for this BTW)
>> ( FYI I could not find the manual of clang modernize for 3.4 )
>> ASAIK the driver for visual is done.
>>
>> And I have Cmake 2.8.12.1 (latest version) I haven't seen anything in the
>> changelogs for 2.8.12.2 neither 3.0 that is about this matter.
>>
>> So I would like to know what I should do to get it working (I mean use the
>> compile_commands.json with visual generator of cmake).
>>
>> Thank you for your help
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
12
Loading...