GSoC Proposal: Improve Windows Support

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

GSoC Proposal: Improve Windows Support

endlessroad1991@gmail.com
Hi all,
First, congratulations on full C++11 support :)

I want to propose "improving windows support" as a GSoC project. By "windows support", I mean using msvc headers & linker, not the mingw version.
Last time I checked(clang 3.2 release), clang can't produce executable files on Windows because of some linker error. So what's the current status and what's the major problem here?
- Win32/64 ABI, including name mangling, calling convention, etc? http://llvm.org/bugs/show_bug.cgi?id=12477
- PE/COFF object file generation?
- Frontend compatibility with MSVC C++ headers? http://llvm.org/bugs/show_bug.cgi?id=13707
I see quite some issues about windows, but it seems that their priority is not very high. So I think may jump in and try to sweep them.

I understand that Windows isn't a first-class target for clang, but I believe making clang usable on Windows will benefit lots of developers.

Some things about myself: I'm Tong Shen, a master candidate in China.
I have submitted a patch for supporting __declspec(property) grammar and the patch is accepted. It's all frontend work.
I ported Android Java VM(dalvik VM) to a experiment OS(a simple OS used in our OS class project), which involves ELF file parsing & execution, dynamic library loading, implementing missing syscall, etc.

So what do you think?
Thanks.

--
Best Regards, Tong Shen (沈彤)

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Devid
Hi,

This would be really great if Clang could at least process all the windows headers without errors.

Especially if using together with this tools.
http://www.ishani.org/web/2012/clangvsx-again/

There are also some clones with some changes to improve this situation but unfortunately it still do not really work.
https://github.com/ishani/clang-msvc
https://github.com/ishani/llvm-msvc

Of course the ultimate goal would be to be able to compile working executable.
Unfortunately I have no time to goo deep into llvm and clang to make some serious contributions for this.

best regards


Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

João Matos
In reply to this post by endlessroad1991@gmail.com
On Sat, Apr 20, 2013 at 5:16 AM, [hidden email] <[hidden email]> wrote:
I want to propose "improving windows support" as a GSoC project. By "windows support", I mean using msvc headers & linker, not the mingw version.
Last time I checked(clang 3.2 release), clang can't produce executable files on Windows because of some linker error. So what's the current status and what's the major problem here?

Awesome.

When was the last time you tested that? Clang should be able to produce executable files, it has support for MS's linker (link.exe) in the driver.
 
- Win32/64 ABI, including name mangling, calling convention, etc? http://llvm.org/bugs/show_bug.cgi?id=12477

A lot of work has been done on this area lately and it's much better than it was on 3.2. 
64-bit name mangling (http://llvm.org/bugs/show_bug.cgi?id=13792) still needs some work. There are also some bugs to fix related to lambdas, but they seem relatively easy to fix.
 
- PE/COFF object file generation?

What kind of improvements do you have in mind? AFAIK 64-bit EH is big gap in support here right now.

- Frontend compatibility with MSVC C++ headers? http://llvm.org/bugs/show_bug.cgi?id=13707
I see quite some issues about windows, but it seems that their priority is not very high. So I think may jump in and try to sweep them.

Yeah, intrinsics are the main blocker at the moment to get most of the broken headers going through. Also, 2012 library needs some relies on some quirks in preprocessor behavior that needs to be better supported.
 
So what do you think?

I think you've already proven you can work on Clang and MS frontend stuff with your patches so I'd really love to see you work on improving support even more.

[CC'ing people working on MS/Windows stuff, they might have more ideas]

--
João Matos
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Timur Iskhodzhanov
2013/4/21 João Matos <[hidden email]>:

> On Sat, Apr 20, 2013 at 5:16 AM, [hidden email]
> <[hidden email]> wrote:
>>
>> I want to propose "improving windows support" as a GSoC project. By
>> "windows support", I mean using msvc headers & linker, not the mingw
>> version.
>> Last time I checked(clang 3.2 release), clang can't produce executable
>> files on Windows because of some linker error. So what's the current status
>> and what's the major problem here?
>
>
> Awesome.
>
> When was the last time you tested that? Clang should be able to produce
> executable files, it has support for MS's linker (link.exe) in the driver.
>
>>
>> - Win32/64 ABI, including name mangling, calling convention, etc?
>> http://llvm.org/bugs/show_bug.cgi?id=12477
>
>
> A lot of work has been done on this area lately and it's much better than it
> was on 3.2.
> 64-bit name mangling (http://llvm.org/bugs/show_bug.cgi?id=13792) still
> needs some work. There are also some bugs to fix related to lambdas, but
> they seem relatively easy to fix.
>
>>
>> - PE/COFF object file generation?
>
>
> What kind of improvements do you have in mind? AFAIK 64-bit EH is big gap in
> support here right now.
>
>> - Frontend compatibility with MSVC C++ headers?
>> http://llvm.org/bugs/show_bug.cgi?id=13707
>> I see quite some issues about windows, but it seems that their priority is
>> not very high. So I think may jump in and try to sweep them.
>
>
> Yeah, intrinsics are the main blocker at the moment to get most of the
> broken headers going through. Also, 2012 library needs some relies on some
> quirks in preprocessor behavior that needs to be better supported.
>
>>
>> So what do you think?
>
>
> I think you've already proven you can work on Clang and MS frontend stuff
> with your patches so I'd really love to see you work on improving support
> even more.
>
> [CC'ing people working on MS/Windows stuff, they might have more ideas]
I'm curious who could be a mentor for such a project?

I think the reviewer for most of the patches for CodeGen so far was
John McCall exclusively since he's pretty much the only person
familiar with the codebase in question.
I'm afraid the stream of patches from me and Reid is already close to
John's available capacity.
I'm very not sure if John can also allocate time for a GSoC student.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Tobias Grosser-3
On Mon, 2013-04-22 at 17:54 +0400, Timur Iskhodzhanov wrote:
> 2013/4/21 João Matos <[hidden email]>:
> > On Sat, Apr 20, 2013 at 5:16 AM, [hidden email]
> > <[hidden email]> wrote:
[..]
> > [CC'ing people working on MS/Windows stuff, they might have more ideas]
> I'm curious who could be a mentor for such a project?
>
> I think the reviewer for most of the patches for CodeGen so far was
> John McCall exclusively since he's pretty much the only person
> familiar with the codebase in question.
> I'm afraid the stream of patches from me and Reid is already close to
> John's available capacity.
> I'm very not sure if John can also allocate time for a GSoC student.

What about yourself?

I believe dedicating time to discuss patches, work with the student,
oversee his time management and help him to work in our community are
the most important points. Obviously you also need some technical
knowledge, but important technical discussions should anyway take place
on the mailing list where John can jump in if needed.

Tobias

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Timur Iskhodzhanov
2013/4/22 Tobias Grosser <[hidden email]>:

> On Mon, 2013-04-22 at 17:54 +0400, Timur Iskhodzhanov wrote:
>> 2013/4/21 João Matos <[hidden email]>:
>> > On Sat, Apr 20, 2013 at 5:16 AM, [hidden email]
>> > <[hidden email]> wrote:
> [..]
>> > [CC'ing people working on MS/Windows stuff, they might have more ideas]
>> I'm curious who could be a mentor for such a project?
>>
>> I think the reviewer for most of the patches for CodeGen so far was
>> John McCall exclusively since he's pretty much the only person
>> familiar with the codebase in question.
>> I'm afraid the stream of patches from me and Reid is already close to
>> John's available capacity.
>> I'm very not sure if John can also allocate time for a GSoC student.
>
> What about yourself?
I'm afraid I can't allocate enough time for mentoring this summer due
to personal reasons.

> I believe dedicating time to discuss patches, work with the student,
> oversee his time management and help him to work in our community are
> the most important points. Obviously you also need some technical
> knowledge, but important technical discussions should anyway take place
> on the mailing list where John can jump in if needed.
>
> Tobias
>

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

endlessroad1991@gmail.com
What's the current status of clang on Windows?

I tried llvm & clang r180250 with MSVC 2012, with "clang++ -Xclang -cxx-abi -Xclang microsoft -c 1.cpp"
1. Can't parse MSVC headers. <iostream>, in this case
2. Even if I specified -cxxabi microsoft, "dumpbin /disasm 1.o" gives wrong mangled name: _Z1fi for "void f(int)",instead of cl compiled object(?f@@YAXH@Z)


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Timur Iskhodzhanov
2013/4/25 [hidden email] <[hidden email]>:
> What's the current status of clang on Windows?
We're working on it :)
With my local patches I can build reasonably complex source code.
We're in progress upstreaming them but it'll take some time to polish
them before review/commit process.

> I tried llvm & clang r180250 with MSVC 2012, with "clang++ -Xclang -cxx-abi
> -Xclang microsoft -c 1.cpp"
> 1. Can't parse MSVC headers. <iostream>, in this case
iostream is a tough guy - I think clang can't even compile Linux
headers in the Microsoft ABI mode.
One has to support virtual inheritance to build iostream<> as both
basic_istream<> and basic_ostream<> virtually inherit from
basic_ios<>.
And we're not working with virtual inheritance correctly in this ABI yet.

> 2. Even if I specified -cxxabi microsoft, "dumpbin /disasm 1.o" gives wrong
> mangled name: _Z1fi for "void f(int)",instead of cl compiled
> object(?f@@YAXH@Z)
That looks wrong, symbols are usually mangled correctly to me.
Can you run with -v and make sure there are no "unknown command line
argument" warnings?
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

João Matos
In reply to this post by endlessroad1991@gmail.com
On Thu, Apr 25, 2013 at 6:43 AM, [hidden email] <[hidden email]> wrote:
What's the current status of clang on Windows?

I tried llvm & clang r180250 with MSVC 2012, with "clang++ -Xclang -cxx-abi -Xclang microsoft -c 1.cpp"
1. Can't parse MSVC headers. <iostream>, in this case

You need to patch the preprocessor to get most of MSVC 2012 headers working.
 
2. Even if I specified -cxxabi microsoft, "dumpbin /disasm 1.o" gives wrong mangled name: _Z1fi for "void f(int)",instead of cl compiled object(?f@@YAXH@Z)

That should work. Anyway that reminds me it's about time we make Microsoft ABI the default when targeting the MSVC headers. What do you think? I had done a patch that did this a few months, I can send it for review.

--
João Matos
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

endlessroad1991@gmail.com
In reply to this post by Timur Iskhodzhanov
Sorry, my bad. The name mangling suddenly works now. Didn't figure out what went wrong.

About your local patches, can you publish them somewhere?

The virtual inheritance part, I've traced dynamic_cast, and found it quite complicated(single inheritance, multiple inheritance, virtual inheritance).
As it's needed for a basic header <iostream>, I think I can start working on it.
What do you think?


On Thu, Apr 25, 2013 at 3:21 PM, Timur Iskhodzhanov <[hidden email]> wrote:
2013/4/25 [hidden email] <[hidden email]>:
> What's the current status of clang on Windows?
We're working on it :)
With my local patches I can build reasonably complex source code.
We're in progress upstreaming them but it'll take some time to polish
them before review/commit process.

> I tried llvm & clang r180250 with MSVC 2012, with "clang++ -Xclang -cxx-abi
> -Xclang microsoft -c 1.cpp"
> 1. Can't parse MSVC headers. <iostream>, in this case
iostream is a tough guy - I think clang can't even compile Linux
headers in the Microsoft ABI mode.
One has to support virtual inheritance to build iostream<> as both
basic_istream<> and basic_ostream<> virtually inherit from
basic_ios<>.
And we're not working with virtual inheritance correctly in this ABI yet.

> 2. Even if I specified -cxxabi microsoft, "dumpbin /disasm 1.o" gives wrong
> mangled name: _Z1fi for "void f(int)",instead of cl compiled
> object(?f@@YAXH@Z)
That looks wrong, symbols are usually mangled correctly to me.
Can you run with -v and make sure there are no "unknown command line
argument" warnings?



--
Best Regards, Tong Shen (沈彤)

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

endlessroad1991@gmail.com
In reply to this post by João Matos
Sorry, my bad. The name mangling suddenly works now. Didn't figure out what went wrong.

About making Microsoft ABI default, since Microsoft ABI support has not matured yet, maybe we can hold back for a while...

These headers aside, what's the major problem?


On Thu, Apr 25, 2013 at 6:41 PM, João Matos <[hidden email]> wrote:
On Thu, Apr 25, 2013 at 6:43 AM, [hidden email] <[hidden email]> wrote:
What's the current status of clang on Windows?

I tried llvm & clang r180250 with MSVC 2012, with "clang++ -Xclang -cxx-abi -Xclang microsoft -c 1.cpp"
1. Can't parse MSVC headers. <iostream>, in this case

You need to patch the preprocessor to get most of MSVC 2012 headers working.
 
2. Even if I specified -cxxabi microsoft, "dumpbin /disasm 1.o" gives wrong mangled name: _Z1fi for "void f(int)",instead of cl compiled object(?f@@YAXH@Z)

That should work. Anyway that reminds me it's about time we make Microsoft ABI the default when targeting the MSVC headers. What do you think? I had done a patch that did this a few months, I can send it for review.

--
João Matos



--
Best Regards, Tong Shen (沈彤)

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

João Matos
On Thu, Apr 25, 2013 at 12:45 PM, [hidden email] <[hidden email]> wrote:
Sorry, my bad. The name mangling suddenly works now. Didn't figure out what went wrong.

About making Microsoft ABI default, since Microsoft ABI support has not matured yet, maybe we can hold back for a while...

When targeting the MSVC runtime, even if the MS ABI is not mature, it's getting better and some stuff will actually work, unlike the current defaults (Itanium manglings) which will never work.
 
I attached the MSVC 2012 preprocessor patch. Look into r163022 (it was reverted later) email thread to get some test cases.

--
João Matos
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

0001-Emulate-MSVC-s-preprocessor-macro-argument-separator.patch (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Reid Kleckner-3
On Thu, Apr 25, 2013 at 7:59 AM, João Matos <[hidden email]> wrote:

> On Thu, Apr 25, 2013 at 12:45 PM, [hidden email]
> <[hidden email]> wrote:
>>
>> Sorry, my bad. The name mangling suddenly works now. Didn't figure out
>> what went wrong.
>>
>> About making Microsoft ABI default, since Microsoft ABI support has not
>> matured yet, maybe we can hold back for a while...
>
>
> When targeting the MSVC runtime, even if the MS ABI is not mature, it's
> getting better and some stuff will actually work, unlike the current
> defaults (Itanium manglings) which will never work.

This is from the triple, right?  That defaults to *-pc-win32, which to
me means we should use the Microsoft C++ ABI.  But this could break
someone out there who forgot to target mingw.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Nico Rieck
On 25.04.2013 16:34, Reid Kleckner wrote:
> On Thu, Apr 25, 2013 at 7:59 AM, João Matos <[hidden email]> wrote:
>> On Thu, Apr 25, 2013 at 12:45 PM, [hidden email]
>> When targeting the MSVC runtime, even if the MS ABI is not mature, it's
>> getting better and some stuff will actually work, unlike the current
>> defaults (Itanium manglings) which will never work.
>
> This is from the triple, right?  That defaults to *-pc-win32, which to
> me means we should use the Microsoft C++ ABI.  But this could break
> someone out there who forgot to target mingw.

What if you use something else, like libc++?

-Nico

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Reid Kleckner-3
On Thu, Apr 25, 2013 at 10:59 AM, Nico Rieck <[hidden email]> wrote:

> On 25.04.2013 16:34, Reid Kleckner wrote:
>>
>> On Thu, Apr 25, 2013 at 7:59 AM, João Matos <[hidden email]>
>> wrote:
>>>
>>> On Thu, Apr 25, 2013 at 12:45 PM, [hidden email]
>>> When targeting the MSVC runtime, even if the MS ABI is not mature, it's
>>> getting better and some stuff will actually work, unlike the current
>>> defaults (Itanium manglings) which will never work.
>>
>>
>> This is from the triple, right?  That defaults to *-pc-win32, which to
>> me means we should use the Microsoft C++ ABI.  But this could break
>> someone out there who forgot to target mingw.
>
>
> What if you use something else, like libc++?

Then make sure the triple matches whatever was used to build libc++.

The current default ABI is a mix of the MSVC C ABI and the Itanium C++
ABI.  I don't think we want to perpetuate this situation going
forward.  Better for force users to choose between 100% mingw and 100%
native.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

João Matos
In reply to this post by Reid Kleckner-3
On Thu, Apr 25, 2013 at 3:34 PM, Reid Kleckner <[hidden email]> wrote:
This is from the triple, right?  That defaults to *-pc-win32, which to
me means we should use the Microsoft C++ ABI.  But this could break
someone out there who forgot to target mingw.

Well, I was only suggesting we default to MS ABI if we check in the driver that we are in an MSVC environment, so it would not affect those MinGW cases.

--
João Matos
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Timur Iskhodzhanov
In reply to this post by endlessroad1991@gmail.com
2013/4/25 [hidden email] <[hidden email]>:
> These headers aside, what's the major problem?
I think the biggest problem is that the Clang's design is based
heavily on the assumptions of Itanium ABI which are often different
from those in Microsoft ABI. Changing the design here and there is
something that creates a lot of code review workload and pre-review
contemplation (c).
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Proposal: Improve Windows Support

Anton Korobeynikov
In reply to this post by endlessroad1991@gmail.com
Hello

> So what do you think?
Windows support definitely deserves some love. However, it looks like
your proposal misses the important point: almost all the remaining
problems are hard. Given that you're mostly asking "what can I do?" it
seems that you're underestimating the complexity of this project.

So, maybe you will spend some time studying the issues, select the
ones fixing of which might be doable during GSoC period and return
with proper schedule or tasks?

--
With best regards, Anton Korobeynikov
Faculty of Mathematics and Mechanics, Saint Petersburg State University
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev