Distributing libclang-cpp.dll for Windows in the pre-built downloads

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
Hi all,

I'm working to transition rust-bindgen from libclang to using the
clang libtooling C++ interface directly. This is unfortunately
necessary because there is a great deal of details about AST nodes
that we need but isn't exposed by libclang. I'm comfortable using and
maintaining a library interface on top of the C++ APIs, that's not a
problem.

However, I am hitting a wall on distribution for Windows. Would it at
all a possibility to include the new libclang-cpp library and headers
on Windows in the pre-built binaries? The only real alternative I see
is making users build LLVM+Clang from source, which seems prohibitive
in many cases, or distributing my own binary build, which I prefer not
to resort to for a single open-source Rust tool.

Adding C++ libraries on Windows potentially introduces issues by
exposing the C++ ABI directly,  so I think we would have to specify
which C++ library and toolchain each release was built with.

Thanks,
Stephen
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
Including pre-build libtooling DLL in Windows binaries would be really helpful.

Right now we are building libtooling static libraries from the source code, it takes
several hours (debug/release/x86/x64 targets).

Best regards
Jan



On 18.03.2020 1:17, Stephen Crane via cfe-dev wrote:

> Hi all,
>
> I'm working to transition rust-bindgen from libclang to using the
> clang libtooling C++ interface directly. This is unfortunately
> necessary because there is a great deal of details about AST nodes
> that we need but isn't exposed by libclang. I'm comfortable using and
> maintaining a library interface on top of the C++ APIs, that's not a
> problem.
>
> However, I am hitting a wall on distribution for Windows. Would it at
> all a possibility to include the new libclang-cpp library and headers
> on Windows in the pre-built binaries? The only real alternative I see
> is making users build LLVM+Clang from source, which seems prohibitive
> in many cases, or distributing my own binary build, which I prefer not
> to resort to for a single open-source Rust tool.
>
> Adding C++ libraries on Windows potentially introduces issues by
> exposing the C++ ABI directly,  so I think we would have to specify
> which C++ library and toolchain each release was built with.
>
> Thanks,
> Stephen
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
In reply to this post by suyash singh via cfe-dev
As far as I know, these libraries are not supported on Windows.

Windows DLLs have a limit of 2^16 exported symbols, that LLVM
unfortunately exceeds.

Michael

Am Di., 17. März 2020 um 19:17 Uhr schrieb Stephen Crane via cfe-dev
<[hidden email]>:

>
> Hi all,
>
> I'm working to transition rust-bindgen from libclang to using the
> clang libtooling C++ interface directly. This is unfortunately
> necessary because there is a great deal of details about AST nodes
> that we need but isn't exposed by libclang. I'm comfortable using and
> maintaining a library interface on top of the C++ APIs, that's not a
> problem.
>
> However, I am hitting a wall on distribution for Windows. Would it at
> all a possibility to include the new libclang-cpp library and headers
> on Windows in the pre-built binaries? The only real alternative I see
> is making users build LLVM+Clang from source, which seems prohibitive
> in many cases, or distributing my own binary build, which I prefer not
> to resort to for a single open-source Rust tool.
>
> Adding C++ libraries on Windows potentially introduces issues by
> exposing the C++ ABI directly,  so I think we would have to specify
> which C++ library and toolchain each release was built with.
>
> Thanks,
> Stephen
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
On Wed, Mar 18, 2020 at 9:23 AM Michael Kruse <[hidden email]> wrote:
> Windows DLLs have a limit of 2^16 exported symbols, that LLVM
> unfortunately exceeds.

Would the individual libclang* libraries (e.g. libclangAST,
libclangParse, etc.) exceed this limit? I'll do some experimental
builds and see what I can determine.

- stephen
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
In reply to this post by suyash singh via cfe-dev
What about DLL with simple COM-like interface (abstract class with pure virtual methods)?

Best regards
Jan

On 18.03.2020 17:22, Michael Kruse via cfe-dev wrote:

> As far as I know, these libraries are not supported on Windows.
>
> Windows DLLs have a limit of 2^16 exported symbols, that LLVM
> unfortunately exceeds.
>
> Michael
>
> Am Di., 17. März 2020 um 19:17 Uhr schrieb Stephen Crane via cfe-dev
> <[hidden email]>:
>>
>> Hi all,
>>
>> I'm working to transition rust-bindgen from libclang to using the
>> clang libtooling C++ interface directly. This is unfortunately
>> necessary because there is a great deal of details about AST nodes
>> that we need but isn't exposed by libclang. I'm comfortable using and
>> maintaining a library interface on top of the C++ APIs, that's not a
>> problem.
>>
>> However, I am hitting a wall on distribution for Windows. Would it at
>> all a possibility to include the new libclang-cpp library and headers
>> on Windows in the pre-built binaries? The only real alternative I see
>> is making users build LLVM+Clang from source, which seems prohibitive
>> in many cases, or distributing my own binary build, which I prefer not
>> to resort to for a single open-source Rust tool.
>>
>> Adding C++ libraries on Windows potentially introduces issues by
>> exposing the C++ ABI directly,  so I think we would have to specify
>> which C++ library and toolchain each release was built with.
>>
>> Thanks,
>> Stephen
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
In reply to this post by suyash singh via cfe-dev
Am Mi., 18. März 2020 um 12:51 Uhr schrieb Stephen Crane <[hidden email]>:
> On Wed, Mar 18, 2020 at 9:23 AM Michael Kruse <[hidden email]> wrote:
> > Windows DLLs have a limit of 2^16 exported symbols, that LLVM
> > unfortunately exceeds.
>
> Would the individual libclang* libraries (e.g. libclangAST,
> libclangParse, etc.) exceed this limit? I'll do some experimental
> builds and see what I can determine.

BUILD_SHARED_LIBS is also not supported on Windows, but I don't know
whether for the same reason.

Michael
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
In reply to this post by suyash singh via cfe-dev
This would be a new interface that someone had to out work into.

Note that in llvm/utils/extract_symbols.py there is a script that
tries to export only symbols that are actually needed (for limited
plugin support under Windows). In my experience, it filters out too
many symbols that are needed for non-trivial/example use cases (in my
case: Polly).

Michael

Am Mi., 18. März 2020 um 12:51 Uhr schrieb Jan Ryšavý via cfe-dev
<[hidden email]>:

>
> What about DLL with simple COM-like interface (abstract class with pure virtual methods)?
>
> Best regards
> Jan
>
> On 18.03.2020 17:22, Michael Kruse via cfe-dev wrote:
> > As far as I know, these libraries are not supported on Windows.
> >
> > Windows DLLs have a limit of 2^16 exported symbols, that LLVM
> > unfortunately exceeds.
> >
> > Michael
> >
> > Am Di., 17. März 2020 um 19:17 Uhr schrieb Stephen Crane via cfe-dev
> > <[hidden email]>:
> >>
> >> Hi all,
> >>
> >> I'm working to transition rust-bindgen from libclang to using the
> >> clang libtooling C++ interface directly. This is unfortunately
> >> necessary because there is a great deal of details about AST nodes
> >> that we need but isn't exposed by libclang. I'm comfortable using and
> >> maintaining a library interface on top of the C++ APIs, that's not a
> >> problem.
> >>
> >> However, I am hitting a wall on distribution for Windows. Would it at
> >> all a possibility to include the new libclang-cpp library and headers
> >> on Windows in the pre-built binaries? The only real alternative I see
> >> is making users build LLVM+Clang from source, which seems prohibitive
> >> in many cases, or distributing my own binary build, which I prefer not
> >> to resort to for a single open-source Rust tool.
> >>
> >> Adding C++ libraries on Windows potentially introduces issues by
> >> exposing the C++ ABI directly,  so I think we would have to specify
> >> which C++ library and toolchain each release was built with.
> >>
> >> Thanks,
> >> Stephen
> >> _______________________________________________
> >> cfe-dev mailing list
> >> [hidden email]
> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> > _______________________________________________
> > cfe-dev mailing list
> > [hidden email]
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> >
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
In reply to this post by suyash singh via cfe-dev
On Wed, Mar 18, 2020 at 2:27 PM Michael Kruse <[hidden email]> wrote:
> BUILD_SHARED_LIBS is also not supported on Windows, but I don't know
> whether for the same reason.

I just noticed this. I found an explanation from 2017:
http://lists.llvm.org/pipermail/llvm-dev/2017-June/113925.html Anyone
able to elaborate more on this?

I'm not super familiar with development on Windows, so I hadn't even
thought about exports. I'm testing out the cmake
WINDOWS_EXPORT_ALL_SYMBOLS property to see if it's at all possible to
build shared libs on Windows. I'm assuming distributing static
libraries is too fragile to consider?

- stephen
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
In reply to this post by suyash singh via cfe-dev
On Wed, 18 Mar 2020, Michael Kruse via cfe-dev wrote:

> Am Mi., 18. März 2020 um 12:51 Uhr schrieb Stephen Crane <[hidden email]>:
>> On Wed, Mar 18, 2020 at 9:23 AM Michael Kruse <[hidden email]> wrote:
>> > Windows DLLs have a limit of 2^16 exported symbols, that LLVM
>> > unfortunately exceeds.
>>
>> Would the individual libclang* libraries (e.g. libclangAST,
>> libclangParse, etc.) exceed this limit? I'll do some experimental
>> builds and see what I can determine.
>
> BUILD_SHARED_LIBS is also not supported on Windows, but I don't know
> whether for the same reason.
It's not supported when building with MSVC (or with Clang in MSVC mode),
as it requires dllexport attributes on all symbols to be visible in the
external interface of the DLL (or a def file listing all the exported
symbols), and the interfaces of all the internal libraries aren't
decorated with such attributes.

If building with a MinGW toolchain (either GCC/binutils based, or
Clang/lld based), it can export all symbols if none are marked
specifically for export with dllexport attributes - and BUILD_SHARED_LIBS
should work in this configuration (at least builds with it enabled
succeeded last I tested maybe 7 weeks ago).

// Martin

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
Cmake also supporte the option WINDOWS_EXPORT_ALL_SYMBOLS for msvc. It
just creates a .def (list of symbols to export when linking a dll)
file including all symbols . LLVM's extract_symbols.py does something
similar.

Michael


Am Mi., 18. März 2020 um 16:35 Uhr schrieb Martin Storsjö <[hidden email]>:

>
> On Wed, 18 Mar 2020, Michael Kruse via cfe-dev wrote:
>
> > Am Mi., 18. März 2020 um 12:51 Uhr schrieb Stephen Crane <[hidden email]>:
> >> On Wed, Mar 18, 2020 at 9:23 AM Michael Kruse <[hidden email]> wrote:
> >> > Windows DLLs have a limit of 2^16 exported symbols, that LLVM
> >> > unfortunately exceeds.
> >>
> >> Would the individual libclang* libraries (e.g. libclangAST,
> >> libclangParse, etc.) exceed this limit? I'll do some experimental
> >> builds and see what I can determine.
> >
> > BUILD_SHARED_LIBS is also not supported on Windows, but I don't know
> > whether for the same reason.
>
> It's not supported when building with MSVC (or with Clang in MSVC mode),
> as it requires dllexport attributes on all symbols to be visible in the
> external interface of the DLL (or a def file listing all the exported
> symbols), and the interfaces of all the internal libraries aren't
> decorated with such attributes.
>
> If building with a MinGW toolchain (either GCC/binutils based, or
> Clang/lld based), it can export all symbols if none are marked
> specifically for export with dllexport attributes - and BUILD_SHARED_LIBS
> should work in this configuration (at least builds with it enabled
> succeeded last I tested maybe 7 weeks ago).
>
> // Martin
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
I was able to successfully build and use both x86 and x64 DLLs on
Windows 10 (MSVC 2019) with the following cmake invocation:
cmake ../llvm -GNinja -DLLVM_ENABLE_PROJECTS=llvm;clang
-DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PDB=ON
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD=X86
-DCMAKE_C_COMPILER=cl -DCMAKE_CXX_COMPILER=cl -DLLVM_ENABLE_RTTI=Off
-DLLVM_BUILD_LLVM_DYLIB=On -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=TRUE

I had to make a few changes to the build system (mostly commenting out
checks preventing building DLLs on Windows). I'd be happy to prep
patches to enable this mode properly in the LLVM cmake scripts, so we
can discuss the merits of that option separately.

- stephen

On Wed, Mar 18, 2020 at 2:39 PM Michael Kruse <[hidden email]> wrote:

>
> Cmake also supporte the option WINDOWS_EXPORT_ALL_SYMBOLS for msvc. It
> just creates a .def (list of symbols to export when linking a dll)
> file including all symbols . LLVM's extract_symbols.py does something
> similar.
>
> Michael
>
>
> Am Mi., 18. März 2020 um 16:35 Uhr schrieb Martin Storsjö <[hidden email]>:
> >
> > On Wed, 18 Mar 2020, Michael Kruse via cfe-dev wrote:
> >
> > > Am Mi., 18. März 2020 um 12:51 Uhr schrieb Stephen Crane <[hidden email]>:
> > >> On Wed, Mar 18, 2020 at 9:23 AM Michael Kruse <[hidden email]> wrote:
> > >> > Windows DLLs have a limit of 2^16 exported symbols, that LLVM
> > >> > unfortunately exceeds.
> > >>
> > >> Would the individual libclang* libraries (e.g. libclangAST,
> > >> libclangParse, etc.) exceed this limit? I'll do some experimental
> > >> builds and see what I can determine.
> > >
> > > BUILD_SHARED_LIBS is also not supported on Windows, but I don't know
> > > whether for the same reason.
> >
> > It's not supported when building with MSVC (or with Clang in MSVC mode),
> > as it requires dllexport attributes on all symbols to be visible in the
> > external interface of the DLL (or a def file listing all the exported
> > symbols), and the interfaces of all the internal libraries aren't
> > decorated with such attributes.
> >
> > If building with a MinGW toolchain (either GCC/binutils based, or
> > Clang/lld based), it can export all symbols if none are marked
> > specifically for export with dllexport attributes - and BUILD_SHARED_LIBS
> > should work in this configuration (at least builds with it enabled
> > succeeded last I tested maybe 7 weeks ago).
> >
> > // Martin
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing libclang-cpp.dll for Windows in the pre-built downloads

suyash singh via cfe-dev
Great! We are looking forward to test your patches and DLLs.

Best regards
Jan

Dne 19. 3. 2020 19:32 napsal uživatel Stephen Crane via cfe-dev <[hidden email]>:

I was able to successfully build and use both x86 and x64 DLLs on
Windows 10 (MSVC 2019) with the following cmake invocation:
cmake ../llvm -GNinja -DLLVM_ENABLE_PROJECTS=llvm;clang
-DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PDB=ON
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD=X86
-DCMAKE_C_COMPILER=cl -DCMAKE_CXX_COMPILER=cl -DLLVM_ENABLE_RTTI=Off
-DLLVM_BUILD_LLVM_DYLIB=On -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=TRUE

I had to make a few changes to the build system (mostly commenting out
checks preventing building DLLs on Windows). I'd be happy to prep
patches to enable this mode properly in the LLVM cmake scripts, so we
can discuss the merits of that option separately.

- stephen

On Wed, Mar 18, 2020 at 2:39 PM Michael Kruse <[hidden email]> wrote:
>
> Cmake also supporte the option WINDOWS_EXPORT_ALL_SYMBOLS for msvc. It
> just creates a .def (list of symbols to export when linking a dll)
> file including all symbols . LLVM's extract_symbols.py does something
> similar.
>
> Michael
>
>
> Am Mi., 18. März 2020 um 16:35 Uhr schrieb Martin Storsjö <[hidden email]>:
> >
> > On Wed, 18 Mar 2020, Michael Kruse via cfe-dev wrote:
> >
> > > Am Mi., 18. März 2020 um 12:51 Uhr schrieb Stephen Crane <[hidden email]>:
> > >> On Wed, Mar 18, 2020 at 9:23 AM Michael Kruse <[hidden email]> wrote:
> > >> > Windows DLLs have a limit of 2^16 exported symbols, that LLVM
> > >> > unfortunately exceeds.
> > >>
> > >> Would the individual libclang* libraries (e.g. libclangAST,
> > >> libclangParse, etc.) exceed this limit? I'll do some experimental
> > >> builds and see what I can determine.
> > >
> > > BUILD_SHARED_LIBS is also not supported on Windows, but I don't know
> > > whether for the same reason.
> >
> > It's not supported when building with MSVC (or with Clang in MSVC mode),
> > as it requires dllexport attributes on all symbols to be visible in the
> > external interface of the DLL (or a def file listing all the exported
> > symbols), and the interfaces of all the internal libraries aren't
> > decorated with such attributes.
> >
> > If building with a MinGW toolchain (either GCC/binutils based, or
> > Clang/lld based), it can export all symbols if none are marked
> > specifically for export with dllexport attributes - and BUILD_SHARED_LIBS
> > should work in this configuration (at least builds with it enabled
> > succeeded last I tested maybe 7 weeks ago).
> >
> > // Martin
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing static C++ clang libs on Windows (was: Distributing libclang-cpp.dll for Windows in the pre-built downloads)

suyash singh via cfe-dev
I've been trying to get some sort of usable C++ API DLLs built on
Windows, but I'm giving up for now. My next question: would shipping
static C++ libraries and the corresponding headers on Windows be an
option at all? We already do this for the Linux binary distributions.
Not sure who would be best to weigh in on that. Hans, Tom, any
thoughts on this?

Over the last week I've spent quite a bit of time attempting to build
reasonably useful LLVM and clang-cpp DLLs on Windows. I got LLVM 9 to
build with just the frontend, no targets, but the number of symbols
has grown since then and exceeds 64K. It looks like clang-cpp is still
under the threshold, but not by much. We could probably pick out a
minimal set of LLVM frontend libraries necessary to link the clang
frontend DLL, but that doesn't seem like a good solution to me. So
unfortunately, unless someone spends time improving the
extract_symbols.py or another similar approach, DLLs don't look viable
on Windows.

Thanks,
Stephen



On Thu, Mar 19, 2020 at 1:22 PM <[hidden email]> wrote:

>
> Great! We are looking forward to test your patches and DLLs.
>
> Best regards
> Jan
>
> Dne 19. 3. 2020 19:32 napsal uživatel Stephen Crane via cfe-dev <[hidden email]>:
>
> I was able to successfully build and use both x86 and x64 DLLs on
> Windows 10 (MSVC 2019) with the following cmake invocation:
> cmake ../llvm -GNinja -DLLVM_ENABLE_PROJECTS=llvm;clang
> -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PDB=ON
> -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD=X86
> -DCMAKE_C_COMPILER=cl -DCMAKE_CXX_COMPILER=cl -DLLVM_ENABLE_RTTI=Off
> -DLLVM_BUILD_LLVM_DYLIB=On -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=TRUE
>
> I had to make a few changes to the build system (mostly commenting out
> checks preventing building DLLs on Windows). I'd be happy to prep
> patches to enable this mode properly in the LLVM cmake scripts, so we
> can discuss the merits of that option separately.
>
> - stephen
>
> On Wed, Mar 18, 2020 at 2:39 PM Michael Kruse <[hidden email]> wrote:
> >
> > Cmake also supporte the option WINDOWS_EXPORT_ALL_SYMBOLS for msvc. It
> > just creates a .def (list of symbols to export when linking a dll)
> > file including all symbols . LLVM's extract_symbols.py does something
> > similar.
> >
> > Michael
> >
> >
> > Am Mi., 18. März 2020 um 16:35 Uhr schrieb Martin Storsjö <[hidden email]>:
> > >
> > > On Wed, 18 Mar 2020, Michael Kruse via cfe-dev wrote:
> > >
> > > > Am Mi., 18. März 2020 um 12:51 Uhr schrieb Stephen Crane <[hidden email]>:
> > > >> On Wed, Mar 18, 2020 at 9:23 AM Michael Kruse <[hidden email]> wrote:
> > > >> > Windows DLLs have a limit of 2^16 exported symbols, that LLVM
> > > >> > unfortunately exceeds.
> > > >>
> > > >> Would the individual libclang* libraries (e.g. libclangAST,
> > > >> libclangParse, etc.) exceed this limit? I'll do some experimental
> > > >> builds and see what I can determine.
> > > >
> > > > BUILD_SHARED_LIBS is also not supported on Windows, but I don't know
> > > > whether for the same reason.
> > >
> > > It's not supported when building with MSVC (or with Clang in MSVC mode),
> > > as it requires dllexport attributes on all symbols to be visible in the
> > > external interface of the DLL (or a def file listing all the exported
> > > symbols), and the interfaces of all the internal libraries aren't
> > > decorated with such attributes.
> > >
> > > If building with a MinGW toolchain (either GCC/binutils based, or
> > > Clang/lld based), it can export all symbols if none are marked
> > > specifically for export with dllexport attributes - and BUILD_SHARED_LIBS
> > > should work in this configuration (at least builds with it enabled
> > > succeeded last I tested maybe 7 weeks ago).
> > >
> > > // Martin
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Distributing static C++ clang libs on Windows

suyash singh via cfe-dev
On 03/26/2020 02:39 PM, Stephen Crane wrote:

> I've been trying to get some sort of usable C++ API DLLs built on
> Windows, but I'm giving up for now. My next question: would shipping
> static C++ libraries and the corresponding headers on Windows be an
> option at all? We already do this for the Linux binary distributions.
> Not sure who would be best to weigh in on that. Hans, Tom, any
> thoughts on this?
>
> Over the last week I've spent quite a bit of time attempting to build
> reasonably useful LLVM and clang-cpp DLLs on Windows. I got LLVM 9 to
> build with just the frontend, no targets, but the number of symbols
> has grown since then and exceeds 64K. It looks like clang-cpp is still
> under the threshold, but not by much. We could probably pick out a
> minimal set of LLVM frontend libraries necessary to link the clang
> frontend DLL, but that doesn't seem like a good solution to me. So
> unfortunately, unless someone spends time improving the
> extract_symbols.py or another similar approach, DLLs don't look viable
> on Windows.
>

If you look at this commit[1], I went through the targets
and explicitly marked all the symbols that needed to be public.
I'm not sure if this does better or worse than extract_symbols.py,
but I think it would be reasonable to start doing this in more
places throughout the code, e.g. in llvm/lib/CodeGen.

As LLVM maintainers for Fedora/RHEL, we also have an interest in reducing
the number of exported symbols as this helps make LTO optimizations
more effective.

-Tom

[1] https://github.com/llvm/llvm-project/commit/0dbcb3639451a7c20e2d5133b459552281e64455


> Thanks,
> Stephen
>
>
>
> On Thu, Mar 19, 2020 at 1:22 PM <[hidden email]> wrote:
>>
>> Great! We are looking forward to test your patches and DLLs.
>>
>> Best regards
>> Jan
>>
>> Dne 19. 3. 2020 19:32 napsal uživatel Stephen Crane via cfe-dev <[hidden email]>:
>>
>> I was able to successfully build and use both x86 and x64 DLLs on
>> Windows 10 (MSVC 2019) with the following cmake invocation:
>> cmake ../llvm -GNinja -DLLVM_ENABLE_PROJECTS=llvm;clang
>> -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PDB=ON
>> -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD=X86
>> -DCMAKE_C_COMPILER=cl -DCMAKE_CXX_COMPILER=cl -DLLVM_ENABLE_RTTI=Off
>> -DLLVM_BUILD_LLVM_DYLIB=On -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=TRUE
>>
>> I had to make a few changes to the build system (mostly commenting out
>> checks preventing building DLLs on Windows). I'd be happy to prep
>> patches to enable this mode properly in the LLVM cmake scripts, so we
>> can discuss the merits of that option separately.
>>
>> - stephen
>>
>> On Wed, Mar 18, 2020 at 2:39 PM Michael Kruse <[hidden email]> wrote:
>>>
>>> Cmake also supporte the option WINDOWS_EXPORT_ALL_SYMBOLS for msvc. It
>>> just creates a .def (list of symbols to export when linking a dll)
>>> file including all symbols . LLVM's extract_symbols.py does something
>>> similar.
>>>
>>> Michael
>>>
>>>
>>> Am Mi., 18. März 2020 um 16:35 Uhr schrieb Martin Storsjö <[hidden email]>:
>>>>
>>>> On Wed, 18 Mar 2020, Michael Kruse via cfe-dev wrote:
>>>>
>>>>> Am Mi., 18. März 2020 um 12:51 Uhr schrieb Stephen Crane <[hidden email]>:
>>>>>> On Wed, Mar 18, 2020 at 9:23 AM Michael Kruse <[hidden email]> wrote:
>>>>>>> Windows DLLs have a limit of 2^16 exported symbols, that LLVM
>>>>>>> unfortunately exceeds.
>>>>>>
>>>>>> Would the individual libclang* libraries (e.g. libclangAST,
>>>>>> libclangParse, etc.) exceed this limit? I'll do some experimental
>>>>>> builds and see what I can determine.
>>>>>
>>>>> BUILD_SHARED_LIBS is also not supported on Windows, but I don't know
>>>>> whether for the same reason.
>>>>
>>>> It's not supported when building with MSVC (or with Clang in MSVC mode),
>>>> as it requires dllexport attributes on all symbols to be visible in the
>>>> external interface of the DLL (or a def file listing all the exported
>>>> symbols), and the interfaces of all the internal libraries aren't
>>>> decorated with such attributes.
>>>>
>>>> If building with a MinGW toolchain (either GCC/binutils based, or
>>>> Clang/lld based), it can export all symbols if none are marked
>>>> specifically for export with dllexport attributes - and BUILD_SHARED_LIBS
>>>> should work in this configuration (at least builds with it enabled
>>>> succeeded last I tested maybe 7 weeks ago).
>>>>
>>>> // Martin
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>>
>

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev