Making libc++ on Linux user-friendly

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Making libc++ on Linux user-friendly

Sebastian Redl
Hi,

I'm currently setting up a fresh Linux box (recent Mint, using GCC 4.8 as system compiler) as a dev environment, and since my stuff is using cutting-edge C++, I want Clang and libc++ for this. Unfortunately, getting this to work isn't fun at all.

Clang itself is easy.

libc++ is anything but. Reading the instructions provided[1], it looks very much like building with libsupc++, pulled out of the libstdc++.so, is the best way of doing things. Using libcxxrt (or libc++abi, but why would I do that on Linux) is listed as having a little drawback: "Unfortunately you can't simply run clang with "-stdlib=libc++" at this point, as clang is set up to link for libc++ linked to libsupc++."

Therefore I go with this CMake command line:

CC=clang CXX=clang++ cmake -G Ninja -DLIBCXX_CXX_ABI=libstdc++ -DLIBCXX_LIBSUPCXX_INCLUDE_PATHS="/usr/include/c++/4.8/;/usr/include/x86_64-linux-gnu/c++/4.8/" -DCMAKE_BUILD_TYPE=Release <libc++-source-dir>

Note that I removed the install prefix so it defaults to /usr/local. This means I need an ldconfig update after installing, but that part works. Also, my Clang sits in /usr/local, so the header search path setup in Clang does the right thing. I'm also using Ninja, but I tested and it doesn't change anything compared to Makefiles.

After building and installing, I try to compile a small example program that uses dynamic_cast and iostreams.

#include <iostream>
struct Base { virtual ~Base() {} };
struct Derived : Base {
  void x() { std::cout << "Hello, World!\n"; }
};
void foo(Base& b) {
  dynamic_cast<Derived&>(b).x();
}
int main() {
    Derived d;
    foo(d);
}


And I compile it using

$ clang++ -stdlib=libc++ -o hello-llvm hello.cpp

This fails:

/usr/bin/ld: /tmp/hello-7f7d71.o: undefined reference to symbol '__cxa_free_exception@@CXXABI_1.3'
/usr/lib/x86_64-linux-gnu/libstdc++.so.6: error adding symbols: DSO missing from command line
clang-3.5: error: linker command failed with exit code 1 (use -v to see invocation)


Here's the problem: when building libc++, the linker finds the various ABI functions in libstdc++, and is quite happy with them being there. When Clang calls the linker for the actual program, though, it doesn't pass along a link flag for libstdc++, only for libc++. Thus, the links fails.

Ironically enough, I can't even manually add -lstdc++ to the command line: Clang recognizes this as an attempt to link the standard library and replaces with with -lc++ - so the final link command has two -lc++ flags and the link fails with the same error. I have to give the full path to the library as an input file to make it work. This is not a good situation.

If we strike the goal of having only one copy of the C++ ABI stuff in the final program, even if it links to a C++ library using libstdc++, then we have more options. Using libcxxrt is on the table, and so is linking against the static libsupc++ library. The latter shouldn't use any additional command line flags, so let's try this:

CC=clang CXX=clang++ cmake -G Ninja -DLIBCXX_CXX_ABI=libsupc++ -DLIBCXX_LIBSUPCXX_INCLUDE_PATHS="/usr/include/c++/4.8/;/usr/include/x86_64-linux-gnu/c++/4.8/" -DCMAKE_BUILD_TYPE=Release <libc++-source-dir>

This doesn't work either for some reason:

$ clang++ -stdlib=libc++ -o hello-llvm hello.cpp
/tmp/hello-3689e5.o: In function `foo(Base&)':
hello.cpp:(.text+0x39): undefined reference to `__dynamic_cast'
hello.cpp:(.text+0x4e): undefined reference to `__cxa_bad_cast'
clang-3.5: error: linker command failed with exit code 1 (use -v to see invocation)


Using nm to inspect libc++.so, it appears that the linker just doesn't pull in (or at least export) these two functions. This, in turn, is probably because it doesn't use them (libc++ doesn't use dynamic_cast in any compiled code; there's one use in the inline rethrow_exception, one in the template dynamic_pointer_cast, and one in an unevaluated context in type_traits). Neither is there any linker input instructing the linker to pull in the symbol, or an appropriate use of --whole-archive.

This again can be worked around by explicitly specifying linking against the source library, and here -lsupc++ works.


The bottom line is that none of the libc++ options on Linux work out of the box without additional command line arguments beyond -stdlib=libc++, and the best option (libstdc++, which prevents duplication of the ABI stuff in a process) is actually the worst case, because -lstdc++ doesn't work as a workaround.


This makes me unhappy. So I did some research into possible solutions.

The libsupc++ problem is independent of the others. It basically comes down to using --whole-archive to pull in libsupc++.a completely into the resulting .so, not just the parts that happen to be used. A patch hacking the CMakeLists.txt to do exactly that is attached.


The other problem is this: how do we make it so that no additional command line flags are required, no matter what ABI library is used? Ideally, the libc++.so would just tell the linker to pull the ABI library into the link, or claim to export the symbols itself but really just forward to the underlying library.

The first solution is exactly what libtool does, but using the wrapper isn't really an option, not to mention that it would require writing the libtool specs manually or using libtool to build libc++. Neither of these are attractive options. The linker itself does not have any way of doing this, even though reading the documentation for --rpath-link makes it sound like it does.

The second solution is what is done on MacOS. The MachO linker supports the reexport_library option. Similarly, PE/COFF DLLs on Windows support such forwarding symbols (kernel32.dll contains lots of forwarders to ntdll.dll); an OldNewThing article[2] describes this and the comments contain links to more detailed information.

It appears, however, from my reading of the ld man page and extensive web searching, that there is no way to do the equivalent thing on Linux without actually adding stub forwarders to the library in question. I have asked on StackOverflow[3], but I don't really expect an answer.

So it seems there are these options:

1) Add stub forwarders to libc++. This is annoying, but it works with libc++ modifications alone.
2) Have Clang find out by some means what flags, beyond -lc++, are needed to link against libc++. This information would have to be provided by libc++ somehow.
3) Have the user tell Clang. For example, -stdlib=libc++ could instead be -stdlib=libc++-gnuabi or -stdlib=libc++-cxxrt, depending on the ABI lib. Or the ABI could be a separate option, i.e. -stdlib=libc++ -abilib=cxxrt. Of course, this is little better than the state we have now.

Any thoughts on this? Preferred variant? Things I've overlooked?


[1] http://libcxx.llvm.org/
[2] http://blogs.msdn.com/b/oldnewthing/archive/2006/07/19/671238.aspx
[3] http://stackoverflow.com/questions/22764734/linux-equivalent-of-windows-dll-forwarders-or-macos-reexport-library

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

libsupcxx.patch (536 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Making libc++ on Linux user-friendly

David Chisnall-4
On 31 Mar 2014, at 16:27, Sebastian Redl <[hidden email]> wrote:

> The other problem is this: how do we make it so that no additional command line flags are required, no matter what ABI library is used? Ideally, the libc++.so would just tell the linker to pull the ABI library into the link, or claim to export the symbols itself but really just forward to the underlying library.

This is what we do on FreeBSD:

$ cat /usr/lib/libc++.so                                                                                                                                                        
/* $FreeBSD: release/10.0.0/lib/libc++/libc++.ldscript 253917 2013-08-03 16:23:43Z dim $ */
GROUP ( /usr/lib/libc++.so.1 /usr/lib/libcxxrt.so )

This allows -stdlib=libc++ to just add -lc++ to the ld command line and Just Work™.  It also meant that we could add or remove dependencies without having to alter linker command lines.  We do the same thing with libc.so, and I'd recommend that this be the default way of installing libc++ on Linux.  Then clang++ (or g++) just needs to know to link to libc++ and doesn't need to know anything about the specific configuration.  

David


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Making libc++ on Linux user-friendly

Octoploid
In reply to this post by Sebastian Redl
Sebastian Redl <sebastian.redl@...> writes:

Linking in libc++abi statically works fine on my Linux machine.
With this setup stdlib=libc++ is enough.

 % ldd /usr/lib/libc++.so.1.0
        linux-vdso.so.1 (0x00007fffa51fe000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00007f11469f2000)
        libc.so.6 => /lib/libc.so.6 (0x00007f114667b000)
        libm.so.6 => /lib/libm.so.6 (0x00007f11463eb000)
        librt.so.1 => /lib/librt.so.1 (0x00007f11461e3000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f11461cb000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1146d4e000)

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Making libc++ on Linux user-friendly

Hal Finkel
In reply to this post by David Chisnall-4
----- Original Message -----

> From: "David Chisnall" <[hidden email]>
> To: "Sebastian Redl" <[hidden email]>
> Cc: [hidden email]
> Sent: Monday, March 31, 2014 10:55:09 AM
> Subject: Re: [cfe-dev] Making libc++ on Linux user-friendly
>
> On 31 Mar 2014, at 16:27, Sebastian Redl
> <[hidden email]> wrote:
>
> > The other problem is this: how do we make it so that no additional
> > command line flags are required, no matter what ABI library is
> > used? Ideally, the libc++.so would just tell the linker to pull
> > the ABI library into the link, or claim to export the symbols
> > itself but really just forward to the underlying library.
>
> This is what we do on FreeBSD:
>
> $ cat /usr/lib/libc++.so
> /* $FreeBSD: release/10.0.0/lib/libc++/libc++.ldscript 253917
> 2013-08-03 16:23:43Z dim $ */
> GROUP ( /usr/lib/libc++.so.1 /usr/lib/libcxxrt.so )

Can this work for static libraries also?

 -Hal

>
> This allows -stdlib=libc++ to just add -lc++ to the ld command line
> and Just Work™.  It also meant that we could add or remove
> dependencies without having to alter linker command lines.  We do
> the same thing with libc.so, and I'd recommend that this be the
> default way of installing libc++ on Linux.  Then clang++ (or g++)
> just needs to know to link to libc++ and doesn't need to know
> anything about the specific configuration.
>
> David
>
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>

--
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Making libc++ on Linux user-friendly

David Chisnall-4
On 31 Mar 2014, at 17:45, Hal Finkel <[hidden email]> wrote:

> ----- Original Message -----
>> From: "David Chisnall" <[hidden email]>
>> To: "Sebastian Redl" <[hidden email]>
>> Cc: [hidden email]
>> Sent: Monday, March 31, 2014 10:55:09 AM
>> Subject: Re: [cfe-dev] Making libc++ on Linux user-friendly
>>
>> On 31 Mar 2014, at 16:27, Sebastian Redl
>> <[hidden email]> wrote:
>>
>>> The other problem is this: how do we make it so that no additional
>>> command line flags are required, no matter what ABI library is
>>> used? Ideally, the libc++.so would just tell the linker to pull
>>> the ABI library into the link, or claim to export the symbols
>>> itself but really just forward to the underlying library.
>>
>> This is what we do on FreeBSD:
>>
>> $ cat /usr/lib/libc++.so
>> /* $FreeBSD: release/10.0.0/lib/libc++/libc++.ldscript 253917
>> 2013-08-03 16:23:43Z dim $ */
>> GROUP ( /usr/lib/libc++.so.1 /usr/lib/libcxxrt.so )
>
> Can this work for static libraries also?

The problem doesn't arise for static libraries - you just link the libcxxrt / libsupc++ .o files into libc++.a when you link it.  Our libc++.a contains libcxxrt.

David


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Making libc++ on Linux user-friendly

Sebastian Redl
In reply to this post by David Chisnall-4

On 2014-03-31 17:55, David Chisnall wrote:

> On 31 Mar 2014, at 16:27, Sebastian Redl <[hidden email]> wrote:
>
>> The other problem is this: how do we make it so that no additional command line flags are required, no matter what ABI library is used? Ideally, the libc++.so would just tell the linker to pull the ABI library into the link, or claim to export the symbols itself but really just forward to the underlying library.
> This is what we do on FreeBSD:
>
> $ cat /usr/lib/libc++.so                                                                                                                                                        
> /* $FreeBSD: release/10.0.0/lib/libc++/libc++.ldscript 253917 2013-08-03 16:23:43Z dim $ */
> GROUP ( /usr/lib/libc++.so.1 /usr/lib/libcxxrt.so )
>
> This allows -stdlib=libc++ to just add -lc++ to the ld command line and Just Work™.  It also meant that we could add or remove dependencies without having to alter linker command lines.  We do the same thing with libc.so, and I'd recommend that this be the default way of installing libc++ on Linux.  Then clang++ (or g++) just needs to know to link to libc++ and doesn't need to know anything about the specific configuration.  
>
This is great!

Generating this file with CMake is easy, esp. since I don't actually
need to know where the ABI library is; the linker is fine with
-lstdlibc++ in the file.

Now I just need to figure out how to make CMake use that as libc++.so
instead of the symlink the library target creates.

Sebastian
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev