Should we stop supporting building with Visual Studio?

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
This has been on my mind for quite some time, but recently it's been popping up more and more seeing some of the issues people have run into.

Before people get the wrong idea, let me make one thing clear.  **I am not proposing we stop supporting the CMake Visual Studio generator.  I am only proposing we stop supporting actually compiling with the generated project**.  Yes the distinction is important, and I'll elaborate more on why later.  First though, here are some of the issues with the VS generator:

1) Using MSBuild is slower than Ninja.
2) Unless you remember to pass -Thost=x64 on the command line, you won't be able to successfully build.  We can (and have) updated the documentation to indicate this, but it's not intuitive and still bites people because for some reason this is not the default.
3) Even if you do pass -Thost=x64 to CMake, it will apparently still fail sometimes.  See this thread for details: http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It seems the parallel build scheduler does not do a good job and can bring a machine down.  This is not the first time though, every couple of months there's a thread about how building or running tests from within VS doesn't work.
4) Supporting it is a continuous source of errors and mistakes when writing tests.  The VS generator outputs a project which can build Debug / Release with a single project.  This means that `CMAKE_BUILD_TYPE=Debug` is a no-op on this generator.  The reason this matters for the test suite is because `${CMAKE_CURRENT_BINARY_DIR}` isn't sufficient to identify the location of the binaries.  You need `${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}` instead.

There is a continuous source of problems in our CMake [1, 2, 3, 4, 5].  It also affects tests, and every time someone adds a new lit site configuration, they have to remember to add this magic block of code:

# Support substitution of the tools_dir with user parameters. This is
# used when we can't determine the tool dir at configuration time.
try:
    config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
    config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
except KeyError:
    e = sys.exc_info()[1]
    key, = e.args
    lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))

to the file (even though only about 2 people actually understand what this does), which has caused problems several times.

5) VSCode and Visual Studio both support opening CMake projects directly now, which bypasses MSBuild.  I don't know how well Visual Studio supports LLVM's CMake, but the last time I tried it with VSCode on Linux it worked fine.

----

I mentioned earlier that the distinction between not *building* with a VS-generated project and not supporting the VS generator is important.

I don't want to speak for everyone, but I believe that *most* people use the VS generator because they want IDE support for their projects.  They want to be able to browse code, hit F5 to debug, F9 to set breakpoints, etc.  They don't necessarily care that Ctrl+Shift+B is how the code is generated versus some other incantation.  I'm asserting that it's possible to still have all the things people actually want from the VS generator without actually building from inside of VS.  In fact, I've been doing this for several years.  The workflow is:

1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.

2) Open the VS one.  You have full IDE support.

3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.  If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.

4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.

5) Hit F5.

In short, with only 2 simple additional steps (run CMake an extra time, and type a path into a window), people can have the exact workflow they are used to, plus faster builds, minus all of the problems and complexities associated with building from within VS.

And we can simplify our CMake logic and lit configuration files as well.

----


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
Can we just create a CMAKE variable (eg. LLVM_USE_NINJA_BUILD) that's set to ON by default, but allow users to turn it OFF at their discretion?

I do know that VS2017 supports CMAKE build integration through Ninja.

On Sun, Oct 7, 2018 at 4:51 PM Zachary Turner via cfe-dev <[hidden email]> wrote:
This has been on my mind for quite some time, but recently it's been popping up more and more seeing some of the issues people have run into.

Before people get the wrong idea, let me make one thing clear.  **I am not proposing we stop supporting the CMake Visual Studio generator.  I am only proposing we stop supporting actually compiling with the generated project**.  Yes the distinction is important, and I'll elaborate more on why later.  First though, here are some of the issues with the VS generator:

1) Using MSBuild is slower than Ninja.
2) Unless you remember to pass -Thost=x64 on the command line, you won't be able to successfully build.  We can (and have) updated the documentation to indicate this, but it's not intuitive and still bites people because for some reason this is not the default.
3) Even if you do pass -Thost=x64 to CMake, it will apparently still fail sometimes.  See this thread for details: http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It seems the parallel build scheduler does not do a good job and can bring a machine down.  This is not the first time though, every couple of months there's a thread about how building or running tests from within VS doesn't work.
4) Supporting it is a continuous source of errors and mistakes when writing tests.  The VS generator outputs a project which can build Debug / Release with a single project.  This means that `CMAKE_BUILD_TYPE=Debug` is a no-op on this generator.  The reason this matters for the test suite is because `${CMAKE_CURRENT_BINARY_DIR}` isn't sufficient to identify the location of the binaries.  You need `${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}` instead.

There is a continuous source of problems in our CMake [1, 2, 3, 4, 5].  It also affects tests, and every time someone adds a new lit site configuration, they have to remember to add this magic block of code:

# Support substitution of the tools_dir with user parameters. This is
# used when we can't determine the tool dir at configuration time.
try:
    config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
    config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
except KeyError:
    e = sys.exc_info()[1]
    key, = e.args
    lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))

to the file (even though only about 2 people actually understand what this does), which has caused problems several times.

5) VSCode and Visual Studio both support opening CMake projects directly now, which bypasses MSBuild.  I don't know how well Visual Studio supports LLVM's CMake, but the last time I tried it with VSCode on Linux it worked fine.

----

I mentioned earlier that the distinction between not *building* with a VS-generated project and not supporting the VS generator is important.

I don't want to speak for everyone, but I believe that *most* people use the VS generator because they want IDE support for their projects.  They want to be able to browse code, hit F5 to debug, F9 to set breakpoints, etc.  They don't necessarily care that Ctrl+Shift+B is how the code is generated versus some other incantation.  I'm asserting that it's possible to still have all the things people actually want from the VS generator without actually building from inside of VS.  In fact, I've been doing this for several years.  The workflow is:

1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.

2) Open the VS one.  You have full IDE support.

3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.  If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.

4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.

5) Hit F5.

In short, with only 2 simple additional steps (run CMake an extra time, and type a path into a window), people can have the exact workflow they are used to, plus faster builds, minus all of the problems and complexities associated with building from within VS.

And we can simplify our CMake logic and lit configuration files as well.

----

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
What would the variable do?  Ninja and VS are generators, the only way to specify them is with the -G option to cmake.  If you use the VS generator, there's no way I'm aware of to make it use ninja instead of MSBuild when you hit Ctrl+Shift+B.

That said, type ninja in a command prompt is not a terrible burden, but even if it is, people can always just create a custom Tool command that runs ninja in the specified working directory, and bind it to some keyboard combination so the workflow is almost exactly the same as what they are using today.

On Sun, Oct 7, 2018 at 8:32 PM Hussien Hussien <[hidden email]> wrote:
Can we just create a CMAKE variable (eg. LLVM_USE_NINJA_BUILD) that's set to ON by default, but allow users to turn it OFF at their discretion?

I do know that VS2017 supports CMAKE build integration through Ninja.

On Sun, Oct 7, 2018 at 4:51 PM Zachary Turner via cfe-dev <[hidden email]> wrote:
This has been on my mind for quite some time, but recently it's been popping up more and more seeing some of the issues people have run into.

Before people get the wrong idea, let me make one thing clear.  **I am not proposing we stop supporting the CMake Visual Studio generator.  I am only proposing we stop supporting actually compiling with the generated project**.  Yes the distinction is important, and I'll elaborate more on why later.  First though, here are some of the issues with the VS generator:

1) Using MSBuild is slower than Ninja.
2) Unless you remember to pass -Thost=x64 on the command line, you won't be able to successfully build.  We can (and have) updated the documentation to indicate this, but it's not intuitive and still bites people because for some reason this is not the default.
3) Even if you do pass -Thost=x64 to CMake, it will apparently still fail sometimes.  See this thread for details: http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It seems the parallel build scheduler does not do a good job and can bring a machine down.  This is not the first time though, every couple of months there's a thread about how building or running tests from within VS doesn't work.
4) Supporting it is a continuous source of errors and mistakes when writing tests.  The VS generator outputs a project which can build Debug / Release with a single project.  This means that `CMAKE_BUILD_TYPE=Debug` is a no-op on this generator.  The reason this matters for the test suite is because `${CMAKE_CURRENT_BINARY_DIR}` isn't sufficient to identify the location of the binaries.  You need `${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}` instead.

There is a continuous source of problems in our CMake [1, 2, 3, 4, 5].  It also affects tests, and every time someone adds a new lit site configuration, they have to remember to add this magic block of code:

# Support substitution of the tools_dir with user parameters. This is
# used when we can't determine the tool dir at configuration time.
try:
    config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
    config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
except KeyError:
    e = sys.exc_info()[1]
    key, = e.args
    lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))

to the file (even though only about 2 people actually understand what this does), which has caused problems several times.

5) VSCode and Visual Studio both support opening CMake projects directly now, which bypasses MSBuild.  I don't know how well Visual Studio supports LLVM's CMake, but the last time I tried it with VSCode on Linux it worked fine.

----

I mentioned earlier that the distinction between not *building* with a VS-generated project and not supporting the VS generator is important.

I don't want to speak for everyone, but I believe that *most* people use the VS generator because they want IDE support for their projects.  They want to be able to browse code, hit F5 to debug, F9 to set breakpoints, etc.  They don't necessarily care that Ctrl+Shift+B is how the code is generated versus some other incantation.  I'm asserting that it's possible to still have all the things people actually want from the VS generator without actually building from inside of VS.  In fact, I've been doing this for several years.  The workflow is:

1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.

2) Open the VS one.  You have full IDE support.

3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.  If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.

4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.

5) Hit F5.

In short, with only 2 simple additional steps (run CMake an extra time, and type a path into a window), people can have the exact workflow they are used to, plus faster builds, minus all of the problems and complexities associated with building from within VS.

And we can simplify our CMake logic and lit configuration files as well.

----

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
On Sun, Oct 7, 2018 at 4:51 PM Zachary Turner via cfe-dev
<[hidden email]> wrote:

>
> This has been on my mind for quite some time, but recently it's been popping up more and more seeing some of the issues people have run into.
>
> Before people get the wrong idea, let me make one thing clear.  **I am not proposing we stop supporting the CMake Visual Studio generator.  I am only proposing we stop supporting actually compiling with the generated project**.  Yes the distinction is important, and I'll elaborate more on why later.  First though, here are some of the issues with the VS generator:
>
> 1) Using MSBuild is slower than Ninja.
> 2) Unless you remember to pass -Thost=x64 on the command line, you won't be able to successfully build.  We can (and have) updated the documentation to indicate this, but it's not intuitive and still bites people because for some reason this is not the default.
> 3) Even if you do pass -Thost=x64 to CMake, it will apparently still fail sometimes.  See this thread for details: http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It seems the parallel build scheduler does not do a good job and can bring a machine down.  This is not the first time though, every couple of months there's a thread about how building or running tests from within VS doesn't work.
> 4) Supporting it is a continuous source of errors and mistakes when writing tests.  The VS generator outputs a project which can build Debug / Release with a single project.  This means that `CMAKE_BUILD_TYPE=Debug` is a no-op on this generator.  The reason this matters for the test suite is because `${CMAKE_CURRENT_BINARY_DIR}` isn't sufficient to identify the location of the binaries.  You need `${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}` instead.
>
> There is a continuous source of problems in our CMake [1, 2, 3, 4, 5].  It also affects tests, and every time someone adds a new lit site configuration, they have to remember to add this magic block of code:
>
> # Support substitution of the tools_dir with user parameters. This is
> # used when we can't determine the tool dir at configuration time.
> try:
>     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
>     config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
> except KeyError:
>     e = sys.exc_info()[1]
>     key, = e.args
>     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
>
> to the file (even though only about 2 people actually understand what this does), which has caused problems several times.
>
> 5) VSCode and Visual Studio both support opening CMake projects directly now, which bypasses MSBuild.  I don't know how well Visual Studio supports LLVM's CMake, but the last time I tried it with VSCode on Linux it worked fine.
>
> ----
>
> I mentioned earlier that the distinction between not *building* with a VS-generated project and not supporting the VS generator is important.
>
> I don't want to speak for everyone, but I believe that *most* people use the VS generator because they want IDE support for their projects.  They want to be able to browse code, hit F5 to debug, F9 to set breakpoints, etc.  They don't necessarily care that Ctrl+Shift+B is how the code is generated versus some other incantation.  I'm asserting that it's possible to still have all the things people actually want from the VS generator without actually building from inside of VS.  In fact, I've been doing this for several years.  The workflow is:
>
> 1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.
>
> 2) Open the VS one.  You have full IDE support.
>
> 3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.  If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.
>
> 4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.
>
> 5) Hit F5.
>
> In short, with only 2 simple additional steps (run CMake an extra time, and type a path into a window), people can have the exact workflow they are used to, plus faster builds, minus all of the problems and complexities associated with building from within VS.
>
> And we can simplify our CMake logic and lit configuration files as well.

As someone who almost exclusively uses MSVC and not Ninja on Windows,
I'll have to try this workflow out to see how it works for me in
practice. Unfortunately, I won't really be able to test it for a few
weeks.

However, I'm concerned by the idea that we want to drop support for
building with the standard workflow in a toolchain that's as popular
as MSVC is. "Here's a simple five-step process to make this
plausible", while technically feasible, does not sound like a better
user experience for people trying to get into LLVM development on
Windows. It's also hard for me to foresee what pain points the
research teams at my company will have from such a transition (they
have out-of-tree workflows that have worked fine for them for years
and this transition will incur expenses to update scripts, etc).

Based on your statements about simplifying cmake, I'm reading "stop
supporting" as "prevent entirely; do not accept patches retaining that
support". Is that accurate? If so, what timeline do you envision and
what kind of transition story is there, especially for out-of-tree
folks?

~Aaron
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
Thanks for raising this.

This is a topic I've been interested in for a while too, as I've had to do a few of those lite.site.cfg fix-ups that you mention (in fact I have one sitting unreviewed at https://reviews.llvm.org/D40522 although I've not pinged it in a long time so I'll need to double check that it's still an issue).  There are also other issues.  For example LLVM_ENABLE_ABI_BREAKING_CHECKS is implemented in such a way that by default the value is defined at CMake time based on the value of LLVM_ENABLE_ASSERTIONS which gets confusing with the Visual Studio generator where the value of LLVM_ENABLE_ASSERTIONS does not necessarily correspond to whether assertions are enabled or not.

As I understand it, what you're proposing is to not support building for any configs that return true for GENERATOR_IS_MULTI_CONFIG.  This includes all of the Visual Studio generators, but also the Xcode generator.  I'm not an Xcode user. Does anyone make use of that generator or is it entirely replaced in practice by single-config generators, i.e. Ninja?

We're still using the Visual Studio generators in production at Sony at the moment.  This is largely because until recently they were actually faster than Ninja for us due to the availability of distributed builds on our network.  We've recently patched in support for our system into our private branch of Ninja now so in theory it should be faster/on-par again but we've not yet pulled the trigger on making them the default.  If there's consensus that this is the way forward, then we'll definitely need some time to make the change internally.  I'm only speaking personally in this reply as I'll need to discuss with the rest of the team before we can reach a position, but basically I wouldn't want the conclusion of this thread to be "No dissenting voices, so here's an immediate patch to remove support!"

I've not tried the workflow you describe.  I'll try it out in the coming days to see how it works for me.  My main concerns are:

* How far will it raise the barrier of entry to new developers?  My impression is that a lot of students coming to LLVM for the first time, first build out of the box with Visual Studio before later discovering this magical thing called Ninja that will speed things up.  Potentially this could be mitigated with good enough documentation in the getting started guide I expect.

* LLVM's CMake is super-slow on Windows, and we'd need to run it twice whenever there are project changes.  This could be a significant drawback in the proposed workflow but I'll need to try it before I can say that for sure.

* My muscle memory causing repeated Ctrl+Shift+B :-).  I wonder if we could add a PRE_BUILD custom target conditional on GENERATOR_IS_MULTI_CONFIG to automatically fail any builds with a useful help message.


If the decision is that we continue supporting these generators, then at the very least we should look into adding a buildbot configured to use one of the Visual Studio Generators rather than ninja so that issues get spotted on commit.

-Greg





On Sun, 7 Oct 2018 at 21:51, Zachary Turner via cfe-dev <[hidden email]> wrote:
This has been on my mind for quite some time, but recently it's been popping up more and more seeing some of the issues people have run into.

Before people get the wrong idea, let me make one thing clear.  **I am not proposing we stop supporting the CMake Visual Studio generator.  I am only proposing we stop supporting actually compiling with the generated project**.  Yes the distinction is important, and I'll elaborate more on why later.  First though, here are some of the issues with the VS generator:

1) Using MSBuild is slower than Ninja.
2) Unless you remember to pass -Thost=x64 on the command line, you won't be able to successfully build.  We can (and have) updated the documentation to indicate this, but it's not intuitive and still bites people because for some reason this is not the default.
3) Even if you do pass -Thost=x64 to CMake, it will apparently still fail sometimes.  See this thread for details: http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It seems the parallel build scheduler does not do a good job and can bring a machine down.  This is not the first time though, every couple of months there's a thread about how building or running tests from within VS doesn't work.
4) Supporting it is a continuous source of errors and mistakes when writing tests.  The VS generator outputs a project which can build Debug / Release with a single project.  This means that `CMAKE_BUILD_TYPE=Debug` is a no-op on this generator.  The reason this matters for the test suite is because `${CMAKE_CURRENT_BINARY_DIR}` isn't sufficient to identify the location of the binaries.  You need `${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}` instead.

There is a continuous source of problems in our CMake [1, 2, 3, 4, 5].  It also affects tests, and every time someone adds a new lit site configuration, they have to remember to add this magic block of code:

# Support substitution of the tools_dir with user parameters. This is
# used when we can't determine the tool dir at configuration time.
try:
    config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
    config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
except KeyError:
    e = sys.exc_info()[1]
    key, = e.args
    lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))

to the file (even though only about 2 people actually understand what this does), which has caused problems several times.

5) VSCode and Visual Studio both support opening CMake projects directly now, which bypasses MSBuild.  I don't know how well Visual Studio supports LLVM's CMake, but the last time I tried it with VSCode on Linux it worked fine.

----

I mentioned earlier that the distinction between not *building* with a VS-generated project and not supporting the VS generator is important.

I don't want to speak for everyone, but I believe that *most* people use the VS generator because they want IDE support for their projects.  They want to be able to browse code, hit F5 to debug, F9 to set breakpoints, etc.  They don't necessarily care that Ctrl+Shift+B is how the code is generated versus some other incantation.  I'm asserting that it's possible to still have all the things people actually want from the VS generator without actually building from inside of VS.  In fact, I've been doing this for several years.  The workflow is:

1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.

2) Open the VS one.  You have full IDE support.

3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.  If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.

4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.

5) Hit F5.

In short, with only 2 simple additional steps (run CMake an extra time, and type a path into a window), people can have the exact workflow they are used to, plus faster builds, minus all of the problems and complexities associated with building from within VS.

And we can simplify our CMake logic and lit configuration files as well.

----

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
Yes i listed 5 steps, but 2 of them (#2 and #5) are exactly what you already do.

#1 I only actually do every couple of weeks, which is an improvement over building inside VS. when you build inside vs you have to close and reopen the solution every time you sync, which is really slow. You don’t actually have to regenerate the ide solution unless you need to edit a file that was added, which is rare. I’ve gone several months without regenerating the vs solution.

#3 is not *that* much different than what we already do. 6 of one, half dozen of another.

#4 is the only real diff, if you build from the ide this just works, with this workflow there’s 1 extra step. But you only really have to do it the first time.
On Mon, Oct 8, 2018 at 7:35 AM Aaron Ballman <[hidden email]> wrote:
On Sun, Oct 7, 2018 at 4:51 PM Zachary Turner via cfe-dev
<[hidden email]> wrote:
>
> This has been on my mind for quite some time, but recently it's been popping up more and more seeing some of the issues people have run into.
>
> Before people get the wrong idea, let me make one thing clear.  **I am not proposing we stop supporting the CMake Visual Studio generator.  I am only proposing we stop supporting actually compiling with the generated project**.  Yes the distinction is important, and I'll elaborate more on why later.  First though, here are some of the issues with the VS generator:
>
> 1) Using MSBuild is slower than Ninja.
> 2) Unless you remember to pass -Thost=x64 on the command line, you won't be able to successfully build.  We can (and have) updated the documentation to indicate this, but it's not intuitive and still bites people because for some reason this is not the default.
> 3) Even if you do pass -Thost=x64 to CMake, it will apparently still fail sometimes.  See this thread for details: http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It seems the parallel build scheduler does not do a good job and can bring a machine down.  This is not the first time though, every couple of months there's a thread about how building or running tests from within VS doesn't work.
> 4) Supporting it is a continuous source of errors and mistakes when writing tests.  The VS generator outputs a project which can build Debug / Release with a single project.  This means that `CMAKE_BUILD_TYPE=Debug` is a no-op on this generator.  The reason this matters for the test suite is because `${CMAKE_CURRENT_BINARY_DIR}` isn't sufficient to identify the location of the binaries.  You need `${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}` instead.
>
> There is a continuous source of problems in our CMake [1, 2, 3, 4, 5].  It also affects tests, and every time someone adds a new lit site configuration, they have to remember to add this magic block of code:
>
> # Support substitution of the tools_dir with user parameters. This is
> # used when we can't determine the tool dir at configuration time.
> try:
>     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
>     config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
> except KeyError:
>     e = sys.exc_info()[1]
>     key, = e.args
>     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
>
> to the file (even though only about 2 people actually understand what this does), which has caused problems several times.
>
> 5) VSCode and Visual Studio both support opening CMake projects directly now, which bypasses MSBuild.  I don't know how well Visual Studio supports LLVM's CMake, but the last time I tried it with VSCode on Linux it worked fine.
>
> ----
>
> I mentioned earlier that the distinction between not *building* with a VS-generated project and not supporting the VS generator is important.
>
> I don't want to speak for everyone, but I believe that *most* people use the VS generator because they want IDE support for their projects.  They want to be able to browse code, hit F5 to debug, F9 to set breakpoints, etc.  They don't necessarily care that Ctrl+Shift+B is how the code is generated versus some other incantation.  I'm asserting that it's possible to still have all the things people actually want from the VS generator without actually building from inside of VS.  In fact, I've been doing this for several years.  The workflow is:
>
> 1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.
>
> 2) Open the VS one.  You have full IDE support.
>
> 3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.  If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.
>
> 4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.
>
> 5) Hit F5.
>
> In short, with only 2 simple additional steps (run CMake an extra time, and type a path into a window), people can have the exact workflow they are used to, plus faster builds, minus all of the problems and complexities associated with building from within VS.
>
> And we can simplify our CMake logic and lit configuration files as well.

As someone who almost exclusively uses MSVC and not Ninja on Windows,
I'll have to try this workflow out to see how it works for me in
practice. Unfortunately, I won't really be able to test it for a few
weeks.

However, I'm concerned by the idea that we want to drop support for
building with the standard workflow in a toolchain that's as popular
as MSVC is. "Here's a simple five-step process to make this
plausible", while technically feasible, does not sound like a better
user experience for people trying to get into LLVM development on
Windows. It's also hard for me to foresee what pain points the
research teams at my company will have from such a transition (they
have out-of-tree workflows that have worked fine for them for years
and this transition will incur expenses to update scripts, etc).

Based on your statements about simplifying cmake, I'm reading "stop
supporting" as "prevent entirely; do not accept patches retaining that
support". Is that accurate? If so, what timeline do you envision and
what kind of transition story is there, especially for out-of-tree
folks?

~Aaron

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev

 (in fact I have one sitting unreviewed at https://reviews.llvm.org/D40522 although I've not pinged it in a long time so I'll need to double check that it's still an issue).  

Actually, ignore that.  It's a slightly different issue that I had mismembered as being related to https://reviews.llvm.org/D36263 and https://reviews.llvm.org/D38471 which I also worked on just prior to taking paternity leave!




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev


On Mon, Oct 8, 2018 at 7:42 AM Greg Bedwell <[hidden email]> wrote:
Thanks for raising this.

This is a topic I've been interested in for a while too, as I've had to do a few of those lite.site.cfg fix-ups that you mention (in fact I have one sitting unreviewed at https://reviews.llvm.org/D40522 although I've not pinged it in a long time so I'll need to double check that it's still an issue).  There are also other issues.  For example LLVM_ENABLE_ABI_BREAKING_CHECKS is implemented in such a way that by default the value is defined at CMake time based on the value of LLVM_ENABLE_ASSERTIONS which gets confusing with the Visual Studio generator where the value of LLVM_ENABLE_ASSERTIONS does not necessarily correspond to whether assertions are enabled or not.

As I understand it, what you're proposing is to not support building for any configs that return true for GENERATOR_IS_MULTI_CONFIG.  This includes all of the Visual Studio generators, but also the Xcode generator.  I'm not an Xcode user. Does anyone make use of that generator or is it entirely replaced in practice by single-config generators, i.e. Ninja?
I haven't heard of anyone using the Xcode generated project.  In fact, LLDB maintains its own hand-created Xcode project precisely because the CMake one is considered "unusable".  That said, I don't personally use Xcode or a Mac, so I can't speak for if anyone else might be using the Xcode generator.
 

We're still using the Visual Studio generators in production at Sony at the moment.  This is largely because until recently they were actually faster than Ninja for us due to the availability of distributed builds on our network.  We've recently patched in support for our system into our private branch of Ninja now so in theory it should be faster/on-par again but we've not yet pulled the trigger on making them the default.  If there's consensus that this is the way forward, then we'll definitely need some time to make the change internally.  I'm only speaking personally in this reply as I'll need to discuss with the rest of the team before we can reach a position, but basically I wouldn't want the conclusion of this thread to be "No dissenting voices, so here's an immediate patch to remove support!"
There's a patch up right now to add support for /MP.  https://reviews.llvm.org/D52193.  In theory this should also help unless you have your own distributed build system.  I'm curious what was actually faster though.  I've found hitting Ctrl+Shift+B from within Visual Studio to be much slower, but it seems like a lot of that time is going to MSBuild resolving dependencies and stuff.  Like it sometimes takes over 30 seconds before it even starts doing *anything*.
 

I've not tried the workflow you describe.  I'll try it out in the coming days to see how it works for me.  My main concerns are:

* How far will it raise the barrier of entry to new developers?  My impression is that a lot of students coming to LLVM for the first time, first build out of the box with Visual Studio before later discovering this magical thing called Ninja that will speed things up.  Potentially this could be mitigated with good enough documentation in the getting started guide I expect.
There's a couple of ways we can mitigate this.  We can print a warning when using the VS generator, and we can update the getting started guide.  But I'm not sure it will raise the barrier of entry much, if at all.  Right now new developers are struggling with building and running even with VS.  Every couple of weeks there's posts about how the test suite wouldn't run, or something is running out of heap space, or they forgot to use -Thost=x64.  
 

* LLVM's CMake is super-slow on Windows, and we'd need to run it twice whenever there are project changes.  This could be a significant drawback in the proposed workflow but I'll need to try it before I can say that for sure.
I mentioned this in my response to Aaron, but just to re-iterate here, you only ever need to run CMake on the VS project if you actually want to edit a file that has been added, which is pretty rare.  I have gone several months without re-generating and it works fine.  This is actually a big improvement over the VS-generator-only workflow.  FWIW, my experience is that the Ninja generator is at least twice as fast as the CMake generator.
 

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev

We are also currently using VS for building on Windows. Having spent the last few months working with Cmake on both Windows and Linux, I think no one on our team would be considerably inconvenienced to switch to using ninja on Windows as we already do that on Linux. Having said that, if our only workflow was on Windows with VS, it would not be trivial to make the switch, so if the decision is to stop support VS for building, part of that work will have to be to have clear documentation on how to move from a VS build to a ninja build, so that anyone who is not familiar with ninja and the difference between the generators would have a place to start.

 

That aside, I have a couple of other concerns with the proposed change.

  1. If we support VS as a generator at all, people will expect it to also build and that will probably cause just as many if not more inconveniencies than the current problems with VS as a build tool. I think that if the decision is to drop support for building through VS, then it would be cleaner to stop supporting VS as a generator entirely. This would probably imply to any other generators that support multiple configurations such as Xcode.
  2. We’d have to drop support for VS as a generator for all projects – llvm, clang, lld, lldb, compiler-rt, etc. I assume that’s what you’re proposing, but I want to make sure. All the projects will probably need their Cmake files not only cleaned up of any xcode/vs related changes, but will need a warning added that these are not supported in case someone tries to build just a subset.
  3. Anyone who is currently using the VS project to build multiple configurations will end up having to run cmake with ninja multiple times to get any configurations they build (e.g. release and debug). So it doesn’t matter if the cmake generator with ninja is faster than VS on a single run – we have to assume people will have to run it (at least) twice to re-create their current configuration setup.

 

That’s not to say that we shouldn’t drop support for VS, but we’ll have to make sure that dropping the support actually improves the experience instead of making it more complicated for the majority of people who use VS to build today.

 

Thanks,

-Stella

 

From: lldb-dev <[hidden email]> On Behalf Of Zachary Turner via lldb-dev
Sent: Monday, October 8, 2018 7:45 AM
To: Aaron Ballman <[hidden email]>
Cc: llvm-dev <[hidden email]>; cfe-dev <[hidden email]>; LLDB Dev <[hidden email]>
Subject: Re: [lldb-dev] [cfe-dev] Should we stop supporting building with Visual Studio?

 

Yes i listed 5 steps, but 2 of them (#2 and #5) are exactly what you already do.

#1 I only actually do every couple of weeks, which is an improvement over building inside VS. when you build inside vs you have to close and reopen the solution every time you sync, which is really slow. You don’t actually have to regenerate the ide solution unless you need to edit a file that was added, which is rare. I’ve gone several months without regenerating the vs solution.

#3 is not *that* much different than what we already do. 6 of one, half dozen of another.

#4 is the only real diff, if you build from the ide this just works, with this workflow there’s 1 extra step. But you only really have to do it the first time.

On Mon, Oct 8, 2018 at 7:35 AM Aaron Ballman <[hidden email]> wrote:

On Sun, Oct 7, 2018 at 4:51 PM Zachary Turner via cfe-dev
<[hidden email]> wrote:
>
> This has been on my mind for quite some time, but recently it's been popping up more and more seeing some of the issues people have run into.
>
> Before people get the wrong idea, let me make one thing clear.  **I am not proposing we stop supporting the CMake Visual Studio generator.  I am only proposing we stop supporting actually compiling with the generated project**.  Yes the distinction is important, and I'll elaborate more on why later.  First though, here are some of the issues with the VS generator:
>
> 1) Using MSBuild is slower than Ninja.
> 2) Unless you remember to pass -Thost=x64 on the command line, you won't be able to successfully build.  We can (and have) updated the documentation to indicate this, but it's not intuitive and still bites people because for some reason this is not the default.
> 3) Even if you do pass -Thost=x64 to CMake, it will apparently still fail sometimes.  See this thread for details: http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It seems the parallel build scheduler does not do a good job and can bring a machine down.  This is not the first time though, every couple of months there's a thread about how building or running tests from within VS doesn't work.
> 4) Supporting it is a continuous source of errors and mistakes when writing tests.  The VS generator outputs a project which can build Debug / Release with a single project.  This means that `CMAKE_BUILD_TYPE=Debug` is a no-op on this generator.  The reason this matters for the test suite is because `${CMAKE_CURRENT_BINARY_DIR}` isn't sufficient to identify the location of the binaries.  You need `${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}` instead.
>
> There is a continuous source of problems in our CMake [1, 2, 3, 4, 5].  It also affects tests, and every time someone adds a new lit site configuration, they have to remember to add this magic block of code:
>
> # Support substitution of the tools_dir with user parameters. This is
> # used when we can't determine the tool dir at configuration time.
> try:
>     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
>     config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
> except KeyError:
>     e = sys.exc_info()[1]
>     key, = e.args
>     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
>
> to the file (even though only about 2 people actually understand what this does), which has caused problems several times.
>
> 5) VSCode and Visual Studio both support opening CMake projects directly now, which bypasses MSBuild.  I don't know how well Visual Studio supports LLVM's CMake, but the last time I tried it with VSCode on Linux it worked fine.
>
> ----
>
> I mentioned earlier that the distinction between not *building* with a VS-generated project and not supporting the VS generator is important.
>
> I don't want to speak for everyone, but I believe that *most* people use the VS generator because they want IDE support for their projects.  They want to be able to browse code, hit F5 to debug, F9 to set breakpoints, etc.  They don't necessarily care that Ctrl+Shift+B is how the code is generated versus some other incantation.  I'm asserting that it's possible to still have all the things people actually want from the VS generator without actually building from inside of VS.  In fact, I've been doing this for several years.  The workflow is:
>
> 1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.
>
> 2) Open the VS one.  You have full IDE support.
>
> 3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.  If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.
>
> 4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.
>
> 5) Hit F5.
>
> In short, with only 2 simple additional steps (run CMake an extra time, and type a path into a window), people can have the exact workflow they are used to, plus faster builds, minus all of the problems and complexities associated with building from within VS.
>
> And we can simplify our CMake logic and lit configuration files as well.

As someone who almost exclusively uses MSVC and not Ninja on Windows,
I'll have to try this workflow out to see how it works for me in
practice. Unfortunately, I won't really be able to test it for a few
weeks.

However, I'm concerned by the idea that we want to drop support for
building with the standard workflow in a toolchain that's as popular
as MSVC is. "Here's a simple five-step process to make this
plausible", while technically feasible, does not sound like a better
user experience for people trying to get into LLVM development on
Windows. It's also hard for me to foresee what pain points the
research teams at my company will have from such a transition (they
have out-of-tree workflows that have worked fine for them for years
and this transition will incur expenses to update scripts, etc).

Based on your statements about simplifying cmake, I'm reading "stop
supporting" as "prevent entirely; do not accept patches retaining that
support". Is that accurate? If so, what timeline do you envision and
what kind of transition story is there, especially for out-of-tree
folks?

~Aaron


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
I don't think we should drop support for VS as a generator entirely, because having the IDE to browse code, Intellisense, and launch the debugger is a huge value add that I don't think anyone (including myself) would be willing to give up.

For #3 this is true, although if it makes a difference just changing the active configuration from Debug to Release inside of VS is something that takes several minutes (or at least it did the last time I tried, which admittedly was a long time ago).  Anyway, your point still stands, which is that generating the ninja won't necessarily be faster than generating VS if you have to generate multiple times.  

Since I'm the only one I can think of currently using this workflow that I've suggested, I'd be interested in seeing if some of the other VS users could try it out and offer their thoughts.  I personally think that once people get used to it they would never want to go back, but acknowledge that it can be a hurdle to untrain the memory muscle.

On Mon, Oct 8, 2018 at 10:13 AM Stella Stamenova <[hidden email]> wrote:

We are also currently using VS for building on Windows. Having spent the last few months working with Cmake on both Windows and Linux, I think no one on our team would be considerably inconvenienced to switch to using ninja on Windows as we already do that on Linux. Having said that, if our only workflow was on Windows with VS, it would not be trivial to make the switch, so if the decision is to stop support VS for building, part of that work will have to be to have clear documentation on how to move from a VS build to a ninja build, so that anyone who is not familiar with ninja and the difference between the generators would have a place to start.

 

That aside, I have a couple of other concerns with the proposed change.

  1. If we support VS as a generator at all, people will expect it to also build and that will probably cause just as many if not more inconveniencies than the current problems with VS as a build tool. I think that if the decision is to drop support for building through VS, then it would be cleaner to stop supporting VS as a generator entirely. This would probably imply to any other generators that support multiple configurations such as Xcode.
  2. We’d have to drop support for VS as a generator for all projects – llvm, clang, lld, lldb, compiler-rt, etc. I assume that’s what you’re proposing, but I want to make sure. All the projects will probably need their Cmake files not only cleaned up of any xcode/vs related changes, but will need a warning added that these are not supported in case someone tries to build just a subset.
  3. Anyone who is currently using the VS project to build multiple configurations will end up having to run cmake with ninja multiple times to get any configurations they build (e.g. release and debug). So it doesn’t matter if the cmake generator with ninja is faster than VS on a single run – we have to assume people will have to run it (at least) twice to re-create their current configuration setup.

 

That’s not to say that we shouldn’t drop support for VS, but we’ll have to make sure that dropping the support actually improves the experience instead of making it more complicated for the majority of people who use VS to build today.

 

Thanks,

-Stella

 

From: lldb-dev <[hidden email]> On Behalf Of Zachary Turner via lldb-dev
Sent: Monday, October 8, 2018 7:45 AM
To: Aaron Ballman <[hidden email]>
Cc: llvm-dev <[hidden email]>; cfe-dev <[hidden email]>; LLDB Dev <[hidden email]>
Subject: Re: [lldb-dev] [cfe-dev] Should we stop supporting building with Visual Studio?

 

Yes i listed 5 steps, but 2 of them (#2 and #5) are exactly what you already do.

#1 I only actually do every couple of weeks, which is an improvement over building inside VS. when you build inside vs you have to close and reopen the solution every time you sync, which is really slow. You don’t actually have to regenerate the ide solution unless you need to edit a file that was added, which is rare. I’ve gone several months without regenerating the vs solution.

#3 is not *that* much different than what we already do. 6 of one, half dozen of another.

#4 is the only real diff, if you build from the ide this just works, with this workflow there’s 1 extra step. But you only really have to do it the first time.

On Mon, Oct 8, 2018 at 7:35 AM Aaron Ballman <[hidden email]> wrote:

On Sun, Oct 7, 2018 at 4:51 PM Zachary Turner via cfe-dev
<[hidden email]> wrote:
>
> This has been on my mind for quite some time, but recently it's been popping up more and more seeing some of the issues people have run into.
>
> Before people get the wrong idea, let me make one thing clear.  **I am not proposing we stop supporting the CMake Visual Studio generator.  I am only proposing we stop supporting actually compiling with the generated project**.  Yes the distinction is important, and I'll elaborate more on why later.  First though, here are some of the issues with the VS generator:
>
> 1) Using MSBuild is slower than Ninja.
> 2) Unless you remember to pass -Thost=x64 on the command line, you won't be able to successfully build.  We can (and have) updated the documentation to indicate this, but it's not intuitive and still bites people because for some reason this is not the default.
> 3) Even if you do pass -Thost=x64 to CMake, it will apparently still fail sometimes.  See this thread for details: http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It seems the parallel build scheduler does not do a good job and can bring a machine down.  This is not the first time though, every couple of months there's a thread about how building or running tests from within VS doesn't work.
> 4) Supporting it is a continuous source of errors and mistakes when writing tests.  The VS generator outputs a project which can build Debug / Release with a single project.  This means that `CMAKE_BUILD_TYPE=Debug` is a no-op on this generator.  The reason this matters for the test suite is because `${CMAKE_CURRENT_BINARY_DIR}` isn't sufficient to identify the location of the binaries.  You need `${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}` instead.
>
> There is a continuous source of problems in our CMake [1, 2, 3, 4, 5].  It also affects tests, and every time someone adds a new lit site configuration, they have to remember to add this magic block of code:
>
> # Support substitution of the tools_dir with user parameters. This is
> # used when we can't determine the tool dir at configuration time.
> try:
>     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
>     config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
> except KeyError:
>     e = sys.exc_info()[1]
>     key, = e.args
>     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
>
> to the file (even though only about 2 people actually understand what this does), which has caused problems several times.
>
> 5) VSCode and Visual Studio both support opening CMake projects directly now, which bypasses MSBuild.  I don't know how well Visual Studio supports LLVM's CMake, but the last time I tried it with VSCode on Linux it worked fine.
>
> ----
>
> I mentioned earlier that the distinction between not *building* with a VS-generated project and not supporting the VS generator is important.
>
> I don't want to speak for everyone, but I believe that *most* people use the VS generator because they want IDE support for their projects.  They want to be able to browse code, hit F5 to debug, F9 to set breakpoints, etc.  They don't necessarily care that Ctrl+Shift+B is how the code is generated versus some other incantation.  I'm asserting that it's possible to still have all the things people actually want from the VS generator without actually building from inside of VS.  In fact, I've been doing this for several years.  The workflow is:
>
> 1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.
>
> 2) Open the VS one.  You have full IDE support.
>
> 3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.  If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.
>
> 4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.
>
> 5) Hit F5.
>
> In short, with only 2 simple additional steps (run CMake an extra time, and type a path into a window), people can have the exact workflow they are used to, plus faster builds, minus all of the problems and complexities associated with building from within VS.
>
> And we can simplify our CMake logic and lit configuration files as well.

As someone who almost exclusively uses MSVC and not Ninja on Windows,
I'll have to try this workflow out to see how it works for me in
practice. Unfortunately, I won't really be able to test it for a few
weeks.

However, I'm concerned by the idea that we want to drop support for
building with the standard workflow in a toolchain that's as popular
as MSVC is. "Here's a simple five-step process to make this
plausible", while technically feasible, does not sound like a better
user experience for people trying to get into LLVM development on
Windows. It's also hard for me to foresee what pain points the
research teams at my company will have from such a transition (they
have out-of-tree workflows that have worked fine for them for years
and this transition will incur expenses to update scripts, etc).

Based on your statements about simplifying cmake, I'm reading "stop
supporting" as "prevent entirely; do not accept patches retaining that
support". Is that accurate? If so, what timeline do you envision and
what kind of transition story is there, especially for out-of-tree
folks?

~Aaron


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
On 07/10/18 21:49, Zachary Turner via cfe-dev wrote:
> 1) Using MSBuild is slower than Ninja.

I use Incredibuild in work. I'm not sure if it supports Ninja, or if
that requires a different license etc. It's certainly faster with
MSBuild than using Ninja locally only.

> 2) Unless you remember to pass -Thost=x64 on the command line, you won't
> be able to successfully build.  We can (and have) updated the
> documentation to indicate this, but it's not intuitive and still bites
> people because for some reason this is not the default.

This has only just been added to the documentation. Should we give that
a chance?

This is a detail that users may have to get right, yes. What details
have to be done right (and not forgotten) in your proposal? Does it just
substitute one "detail to remember" for another?

> 3) Even if you do pass -Thost=x64 to CMake, it will apparently still
> fail sometimes.  See this thread for details:
> http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It
> seems the parallel build scheduler does not do a good job and can bring
> a machine down.  This is not the first time though, every couple of
> months there's a thread about how building or running tests from within
> VS doesn't work.

I don't know any more about this. It would be good to know more than
that it can "apparently fail sometimes".


Thanks!

Stephen.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev


On Mon, Oct 8, 2018 at 11:54 AM Stephen Kelly via cfe-dev <[hidden email]> wrote:

> 3) Even if you do pass -Thost=x64 to CMake, it will apparently still
> fail sometimes.  See this thread for details:
> http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It
> seems the parallel build scheduler does not do a good job and can bring
> a machine down.  This is not the first time though, every couple of
> months there's a thread about how building or running tests from within
> VS doesn't work.

I don't know any more about this. It would be good to know more than
that it can "apparently fail sometimes".


Sadly that's part of the problem.  Very few people actually use the Visual Studio generator for building, so a lot of times when we get people with issues, nobody knows how to help (or the person that does know doesn't see the thread).  So they get a response like "hmm, not many people actually use that workflow, can you try this instead?"

I feel bad when I can't help, and that's part of why I made this proposal in the first place, because fewer supported options in the configuration matrix means people are more likely to find someone who understands the problem when something goes wrong.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev


> On Oct 8, 2018, at 8:09 AM, Zachary Turner via lldb-dev <[hidden email]> wrote:
>
> I haven't heard of anyone using the Xcode generated project.  In fact, LLDB maintains its own hand-created Xcode project precisely because the CMake one is considered "unusable".  That said, I don't personally use Xcode or a Mac, so I can't speak for if anyone else might be using the Xcode generator.
>

All of Xcode's smart code comprehension relies on the clang index built from the project.  Xcode uses the project's build rules to construct the index.  Apparently the cmake generated Xcode project doesn't communicate the info on how to build the project and thus the index to Xcode in a way it can use.  I haven't tried it recently for lldb (I don't think anybody on the team here has), but people here were hacking on the generator to use it for developing clang & swift not too long ago. The last report I heard was that it still needs more work to be usable, and that that work was not entirely trivial.  

Without the index, Xcode is a pretty weak IDE, which is why we maintain a hand-built Xcode project.

Jim
 
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev

I build with the VS project. I find it more convenient to do that than have VS and a cmd window open to run ninja. Especially when I’ve got more than 1 copy of VS open looking at different release trains. I wouldn’t mind using ninja to build, but only if it worked when I right click on lldb and select “Build”.

 

Our buildbots use msbuild on the project, like this:

call "c:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/vcvarsall.bat" x64 & msbuild ALL_BUILD.vcxproj /m /property:Configuration=Release

 

--

Ted Woodward

Qualcomm Innovation Center, Inc.

Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

 

From: lldb-dev <[hidden email]> On Behalf Of Zachary Turner via lldb-dev
Sent: Monday, October 8, 2018 1:58 PM
To: Stephen Kelly <[hidden email]>
Cc: [hidden email]; [hidden email]; [hidden email]
Subject: Re: [lldb-dev] [cfe-dev] Should we stop supporting building with Visual Studio?

 

 

On Mon, Oct 8, 2018 at 11:54 AM Stephen Kelly via cfe-dev <[hidden email]> wrote:


> 3) Even if you do pass -Thost=x64 to CMake, it will apparently still
> fail sometimes.  See this thread for details:
> http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It
> seems the parallel build scheduler does not do a good job and can bring
> a machine down.  This is not the first time though, every couple of
> months there's a thread about how building or running tests from within
> VS doesn't work.

I don't know any more about this. It would be good to know more than
that it can "apparently fail sometimes".

 

Sadly that's part of the problem.  Very few people actually use the Visual Studio generator for building, so a lot of times when we get people with issues, nobody knows how to help (or the person that does know doesn't see the thread).  So they get a response like "hmm, not many people actually use that workflow, can you try this instead?"

 

I feel bad when I can't help, and that's part of why I made this proposal in the first place, because fewer supported options in the configuration matrix means people are more likely to find someone who understands the problem when something goes wrong.


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [lldb-dev] Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
On Mon, Oct 8, 2018 at 12:29 PM <[hidden email]> wrote:

I build with the VS project. I find it more convenient to do that than have VS and a cmd window open to run ninja. Especially when I’ve got more than 1 copy of VS open looking at different release trains. I wouldn’t mind using ninja to build, but only if it worked when I right click on lldb and select “Build”.


This seems like a bit of an extreme position to take.   We shouldn't be making decisions about supported configurations based on what keyboard / mouse incantation is used to invoke a command.  The important thing is whether or not there's a reasonable substitute for peoples' existing workflows.  Pushing (for example) Ctrl+Alt+B instead of Ctrl+Shift+B I consider reasonable (or typing ninja from a command prompt I also consider reasonable).

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
> I use Incredibuild in work. I'm not sure if it supports Ninja, or if that requires a different license etc.

It does work, doesn't require a different license, and I've had good luck with it. The tl;dr is run "BuildConsole /COMMAND="ninja [target]" /PROFILE=ninja.xml", but here's the docs link:
https://incredibuild.kayako.com/Knowledgebase/Article/View/139/0/how-to-accelerate-ninja-builds-with-incredibuild

On 10/8/18, 11:55 AM, "cfe-dev on behalf of Stephen Kelly via cfe-dev" <[hidden email] on behalf of [hidden email]> wrote:

    On 07/10/18 21:49, Zachary Turner via cfe-dev wrote:
    > 1) Using MSBuild is slower than Ninja.
   
    I use Incredibuild in work. I'm not sure if it supports Ninja, or if
    that requires a different license etc. It's certainly faster with
    MSBuild than using Ninja locally only.
   
    > 2) Unless you remember to pass -Thost=x64 on the command line, you won't
    > be able to successfully build.  We can (and have) updated the
    > documentation to indicate this, but it's not intuitive and still bites
    > people because for some reason this is not the default.
   
    This has only just been added to the documentation. Should we give that
    a chance?
   
    This is a detail that users may have to get right, yes. What details
    have to be done right (and not forgotten) in your proposal? Does it just
    substitute one "detail to remember" for another?
   
    > 3) Even if you do pass -Thost=x64 to CMake, it will apparently still
    > fail sometimes.  See this thread for details:
    > http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It
    > seems the parallel build scheduler does not do a good job and can bring
    > a machine down.  This is not the first time though, every couple of
    > months there's a thread about how building or running tests from within
    > VS doesn't work.
   
    I don't know any more about this. It would be good to know more than
    that it can "apparently fail sometimes".
   
   
    Thanks!
   
    Stephen.
   
    _______________________________________________
    cfe-dev mailing list
    [hidden email]
    http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
   

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
On Sun, Oct 7, 2018 at 10:51 PM Zachary Turner via cfe-dev
<[hidden email]> wrote:

> 1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.
>
> 2) Open the VS one.  You have full IDE support.
>
> 3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.

If there were errors, eyeball-grep the console output and manually
navigate to the affected file/line. No thanks.

> If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.
>
> 4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.
>

Make some changes in the source, hit build, and wonder why the changes
don't appear in the debuggee. (because they got compiled into the VS
dir, not the ninja dir).

Csaba
--
You can get very substantial performance improvements
by not doing the right thing. - Scott Meyers, An Effective C++11/14 Sampler
So if you're looking for a completely portable, 100% standards-conformat way
to get the wrong information: this is what you want. - Scott Meyers (C++TDaWYK)
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev


On Tue, Oct 9, 2018 at 12:49 AM Csaba Raduly <[hidden email]> wrote:
On Sun, Oct 7, 2018 at 10:51 PM Zachary Turner via cfe-dev
<[hidden email]> wrote:

> 1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.
>
> 2) Open the VS one.  You have full IDE support.
>
> 3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.

If there were errors, eyeball-grep the console output and manually
navigate to the affected file/line. No thanks.
I don’t find this to be a problem in practice.  You have to eyeball grep the output anyway to figure out which  line to double click in the build output window.  Usually it’s a file you have open in which case you don’t have to manually navigate to it.  If it’s not then yes you have to manually open the file, but you don’t have to manually navigate to the line.  You can hit Ctrl+F7 to compile just that file in VS and then double click.  Since the ninja build is faster anyway though, the whole process doesn’t actually end up taking that much more time.  Note that if you were to add a custom tool command to do your build for you and output to the VS console window, you could still double click lines there and it would be exactly the same as if you built from inside VS



> If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.
>
> 4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.
>

Make some changes in the source, hit build, and wonder why the changes
don't appear in the debuggee. (because they got compiled into the VS
dir, not the ninja dir).

This will never happen.  An incremental build takes about 10-20 seconds, compared to 5-10 minutes for a full build.  If you hit Build in VS you will notice because you probably don’t want to sit around for 5-10 minutes.  Moreover, this is just muscle memory which would cause you to accidentally hit build in VS.  I’ve probably only done this 3-4 times in as many years.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev

Sadly that's part of the problem.  Very few people actually use the Visual Studio generator for building, so a lot of times when we get people with issues, nobody knows how to help (or the person that does know doesn't see the thread).  So they get a response like "hmm, not many people actually use that workflow, can you try this instead?"
 

I do wonder how many of these issues could be alleviated if we had a bot doing builds with the VS generator.  It would, at least, shift the onus onto person who broke the workflow to figure it out rather than the next person to come along and try it.



If there were errors, eyeball-grep the console output and manually
navigate to the affected file/line. No thanks.
I don’t find this to be a problem in practice.  You have to eyeball grep the output anyway to figure out which  line to double click in the build output window.  Usually it’s a file you have open in which case you don’t have to manually navigate to it.  If it’s not then yes you have to manually open the file, but you don’t have to manually navigate to the line.  You can hit Ctrl+F7 to compile just that file in VS and then double click.  Since the ninja build is faster anyway though, the whole process doesn’t actually end up taking that much more time.  Note that if you were to add a custom tool command to do your build for you and output to the VS console window, you could still double click lines there and it would be exactly the same as if you built from inside VS

As long as errors are in MSVC format, the Error List window (which I imagine a lot of people use extensively) saves having to actually directly look at build output most of the time.  I've yet to try to the proposed workflow to see how much of an issue it really is for me in practice personally.

-Greg


 

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] Should we stop supporting building with Visual Studio?

David Blaikie via cfe-dev
In reply to this post by David Blaikie via cfe-dev
Hi Zachary,

We use LLVM JIT in SwiftShader, which is used by Google Chrome and Android (Emulator). Most development takes place in Visual Studio, where it builds as part of the rest of the SwiftShader solution. So we care about LLVM source files compiling successfully within Visual Studio.

Would it be reasonable to at least ensure that major releases (7.0, 8.0, etc.) build with Visual Studio? We don't care much about breakages in between releases, and the other issues you listed don't affect us much either due to using custom solution/project files.

Thanks for your consideration,
Nicolas Capens

On Sun, Oct 7, 2018 at 4:51 PM Zachary Turner via llvm-dev <[hidden email]> wrote:
This has been on my mind for quite some time, but recently it's been popping up more and more seeing some of the issues people have run into.

Before people get the wrong idea, let me make one thing clear.  **I am not proposing we stop supporting the CMake Visual Studio generator.  I am only proposing we stop supporting actually compiling with the generated project**.  Yes the distinction is important, and I'll elaborate more on why later.  First though, here are some of the issues with the VS generator:

1) Using MSBuild is slower than Ninja.
2) Unless you remember to pass -Thost=x64 on the command line, you won't be able to successfully build.  We can (and have) updated the documentation to indicate this, but it's not intuitive and still bites people because for some reason this is not the default.
3) Even if you do pass -Thost=x64 to CMake, it will apparently still fail sometimes.  See this thread for details: http://lists.llvm.org/pipermail/cfe-dev/2018-October/059609.html.  It seems the parallel build scheduler does not do a good job and can bring a machine down.  This is not the first time though, every couple of months there's a thread about how building or running tests from within VS doesn't work.
4) Supporting it is a continuous source of errors and mistakes when writing tests.  The VS generator outputs a project which can build Debug / Release with a single project.  This means that `CMAKE_BUILD_TYPE=Debug` is a no-op on this generator.  The reason this matters for the test suite is because `${CMAKE_CURRENT_BINARY_DIR}` isn't sufficient to identify the location of the binaries.  You need `${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}` instead.

There is a continuous source of problems in our CMake [1, 2, 3, 4, 5].  It also affects tests, and every time someone adds a new lit site configuration, they have to remember to add this magic block of code:

# Support substitution of the tools_dir with user parameters. This is
# used when we can't determine the tool dir at configuration time.
try:
    config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
    config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
except KeyError:
    e = sys.exc_info()[1]
    key, = e.args
    lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))

to the file (even though only about 2 people actually understand what this does), which has caused problems several times.

5) VSCode and Visual Studio both support opening CMake projects directly now, which bypasses MSBuild.  I don't know how well Visual Studio supports LLVM's CMake, but the last time I tried it with VSCode on Linux it worked fine.

----

I mentioned earlier that the distinction between not *building* with a VS-generated project and not supporting the VS generator is important.

I don't want to speak for everyone, but I believe that *most* people use the VS generator because they want IDE support for their projects.  They want to be able to browse code, hit F5 to debug, F9 to set breakpoints, etc.  They don't necessarily care that Ctrl+Shift+B is how the code is generated versus some other incantation.  I'm asserting that it's possible to still have all the things people actually want from the VS generator without actually building from inside of VS.  In fact, I've been doing this for several years.  The workflow is:

1) Run CMake twice, generating to separate output directories.  Once using -G "Visual Studio 15 2017" and once using -G Ninja, each to different directories.

2) Open the VS one.  You have full IDE support.

3) Instead of hitting Ctrl+Shift+B to build, have a command prompt window open and type ninja.  Wait for it to complete.  If you want to you can make a custom tool command in Visual Studio so that you can access this from a keyboard shortcut.

4) When you want to debug, set your startup project (as you normally would), right click and hit properties, go to Debugging, change Command from $(TargetPath) to <type the full path to bin/foo.exe of the program you want to debug>.

5) Hit F5.

In short, with only 2 simple additional steps (run CMake an extra time, and type a path into a window), people can have the exact workflow they are used to, plus faster builds, minus all of the problems and complexities associated with building from within VS.

And we can simplify our CMake logic and lit configuration files as well.

----

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
12