Quantcast

[RFC] Prefer libraries installed next to Clang over those from GCC

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[RFC] Prefer libraries installed next to Clang over those from GCC

Ammarguellat, Zahira via cfe-dev
Hi,

Background: I'm on CentOS 7 which has GCC 4.8 installed in /usr/bin and I'm
installing Clang to, let's say, /opt/clang. (I'm doing so with
CLANG_DEFAULT_CXX_STDLIB=libc++, just if you wonder why the following
matters to me...)
With the current behavior, -lunwind passed to Clang will pull in the system
default non-GNU /usr/lib/libunwind.so.8. IMO it should take LLVM's version
in /opt/clang/lib/libunwind.so.1

That's why I would like to change the default order for searching libraries
in Clang. Currently (and most probably due to historical reasons), the
Driver adds paths next to GCC first.
I propose to prefer paths next to the installed Clang. You can find the
patch doing the change here: https://reviews.llvm.org/D26244

However, this has a potential impact as Chandler pointed out in above patch
if Clang is installed to /opt/software together with other software.
With my proposed change, Clang will link against
/opt/software/lib/libunrelated.so when passed -lunrelated whereas before it
had preferred /usr/lib/libunrelated.so (assuming that the library exists in
both places).
Is that considered an important scenario or can we expect our users to
install Clang separately?

Any other feedback on that topic?
Jonas

--
Jonas Hahnfeld, MATSE-Auszubildender

IT Center
Group: High Performance Computing
Division: Computational Science and Engineering
RWTH Aachen University
Seffenter Weg 23
D 52074  Aachen (Germany)
[hidden email]
www.itc.rwth-aachen.de



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Prefer libraries installed next to Clang over those from GCC

Ammarguellat, Zahira via cfe-dev


On 01/30/2017 09:09 AM, Hahnfeld, Jonas via cfe-dev wrote:
Hi,

Background: I'm on CentOS 7 which has GCC 4.8 installed in /usr/bin and I'm
installing Clang to, let's say, /opt/clang. (I'm doing so with
CLANG_DEFAULT_CXX_STDLIB=libc++, just if you wonder why the following
matters to me...)
With the current behavior, -lunwind passed to Clang will pull in the system
default non-GNU /usr/lib/libunwind.so.8. IMO it should take LLVM's version
in /opt/clang/lib/libunwind.so.1

That's why I would like to change the default order for searching libraries
in Clang. Currently (and most probably due to historical reasons), the
Driver adds paths next to GCC first.
I propose to prefer paths next to the installed Clang. You can find the
patch doing the change here: https://reviews.llvm.org/D26244

However, this has a potential impact as Chandler pointed out in above patch
if Clang is installed to /opt/software together with other software.
With my proposed change, Clang will link against
/opt/software/lib/libunrelated.so when passed -lunrelated whereas before it
had preferred /usr/lib/libunrelated.so (assuming that the library exists in
both places).
Is that considered an important scenario or can we expect our users to
install Clang separately?

Any other feedback on that topic?

I think this is an important usage scenario, but because of the complication Chandler mentioned, I don't think we can enable it by default. I think it is very reasonable to have some CMake flag that enables this behavior.

 -Hal

Jonas

--
Jonas Hahnfeld, MATSE-Auszubildender

IT Center
Group: High Performance Computing
Division: Computational Science and Engineering
RWTH Aachen University
Seffenter Weg 23
D 52074  Aachen (Germany)
[hidden email]
www.itc.rwth-aachen.de




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Prefer libraries installed next to Clang over those from GCC

Ammarguellat, Zahira via cfe-dev
On 30 Jan 2017, at 16:57, Hal Finkel via cfe-dev <[hidden email]> wrote:

>
> On 01/30/2017 09:09 AM, Hahnfeld, Jonas via cfe-dev wrote:
>> Hi,
>>
>> Background: I'm on CentOS 7 which has GCC 4.8 installed in /usr/bin and I'm
>> installing Clang to, let's say, /opt/clang. (I'm doing so with
>> CLANG_DEFAULT_CXX_STDLIB=libc++, just if you wonder why the following
>> matters to me...)
>> With the current behavior, -lunwind passed to Clang will pull in the system
>> default non-GNU /usr/lib/libunwind.so.8. IMO it should take LLVM's version
>> in /opt/clang/lib/libunwind.so.1
>>
>> That's why I would like to change the default order for searching libraries
>> in Clang. Currently (and most probably due to historical reasons), the
>> Driver adds paths next to GCC first.
>> I propose to prefer paths next to the installed Clang. You can find the
>> patch doing the change here:
>> https://reviews.llvm.org/D26244
>>
>>
>> However, this has a potential impact as Chandler pointed out in above patch
>> if Clang is installed to /opt/software together with other software.
>> With my proposed change, Clang will link against
>> /opt/software/lib/libunrelated.so when passed -lunrelated whereas before it
>> had preferred /usr/lib/libunrelated.so (assuming that the library exists in
>> both places).
>> Is that considered an important scenario or can we expect our users to
>> install Clang separately?
>>
>> Any other feedback on that topic?
>>
>
> I think this is an important usage scenario, but because of the complication Chandler mentioned, I don't think we can enable it by default. I think it is very reasonable to have some CMake flag that enables this behavior.

Unless we’re also passing -rpath or similar, then this may also lead to confusion where you link against the /opt/clang/lib/libunwind.so, but end up using /usr/lib/libunwind.so.  In this particular example, you’re relying on the fact that the two libunwinds have different SO versions, but that’s by no means guaranteed.

Libunwind is a very bad motivation for this in general because:

- It’s an implementation of a standard ABI
- The LLVM and non-GNU versions (and the GNU fork of the non-GNU version) are intended to be compatible
- Linking both will cause things to go very badly wrong at run time

Unless you can ensure that everything is compiled either purely with clang or purely with gcc, and that you never compile something with one that uses libraries compiled by the other, then this is going to cause far more problems than it solves.

David

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Prefer libraries installed next to Clang over those from GCC

Ammarguellat, Zahira via cfe-dev
On 01/30/2017 11:03 AM, David Chisnall wrote:

> On 30 Jan 2017, at 16:57, Hal Finkel via cfe-dev <[hidden email]> wrote:
>> On 01/30/2017 09:09 AM, Hahnfeld, Jonas via cfe-dev wrote:
>>> Hi,
>>>
>>> Background: I'm on CentOS 7 which has GCC 4.8 installed in /usr/bin and I'm
>>> installing Clang to, let's say, /opt/clang. (I'm doing so with
>>> CLANG_DEFAULT_CXX_STDLIB=libc++, just if you wonder why the following
>>> matters to me...)
>>> With the current behavior, -lunwind passed to Clang will pull in the system
>>> default non-GNU /usr/lib/libunwind.so.8. IMO it should take LLVM's version
>>> in /opt/clang/lib/libunwind.so.1
>>>
>>> That's why I would like to change the default order for searching libraries
>>> in Clang. Currently (and most probably due to historical reasons), the
>>> Driver adds paths next to GCC first.
>>> I propose to prefer paths next to the installed Clang. You can find the
>>> patch doing the change here:
>>> https://reviews.llvm.org/D26244
>>>
>>>
>>> However, this has a potential impact as Chandler pointed out in above patch
>>> if Clang is installed to /opt/software together with other software.
>>> With my proposed change, Clang will link against
>>> /opt/software/lib/libunrelated.so when passed -lunrelated whereas before it
>>> had preferred /usr/lib/libunrelated.so (assuming that the library exists in
>>> both places).
>>> Is that considered an important scenario or can we expect our users to
>>> install Clang separately?
>>>
>>> Any other feedback on that topic?
>>>
>> I think this is an important usage scenario, but because of the complication Chandler mentioned, I don't think we can enable it by default. I think it is very reasonable to have some CMake flag that enables this behavior.
> Unless we’re also passing -rpath or similar, then this may also lead to confusion where you link against the /opt/clang/lib/libunwind.so, but end up using /usr/lib/libunwind.so.  In this particular example, you’re relying on the fact that the two libunwinds have different SO versions, but that’s by no means guaranteed.
>
> Libunwind is a very bad motivation for this in general because:
>
> - It’s an implementation of a standard ABI
> - The LLVM and non-GNU versions (and the GNU fork of the non-GNU version) are intended to be compatible
> - Linking both will cause things to go very badly wrong at run time
>
> Unless you can ensure that everything is compiled either purely with clang or purely with gcc, and that you never compile something with one that uses libraries compiled by the other, then this is going to cause far more problems than it solves.

Good point. Actually, perhaps this is just another motivating case for
https://reviews.llvm.org/D24933 (the Clang config files).

  -Hal

>
> David
>

--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Prefer libraries installed next to Clang over those from GCC

Ammarguellat, Zahira via cfe-dev
In reply to this post by Ammarguellat, Zahira via cfe-dev
[Note: I think my previous reply didn't went when mail program crashed,
sorry if you get it twice]

On Mon, 30 Jan 2017 15:09:46 +0000
"Hahnfeld, Jonas via cfe-dev" <[hidden email]> wrote:

> Background: I'm on CentOS 7 which has GCC 4.8 installed in /usr/bin and I'm
> installing Clang to, let's say, /opt/clang. (I'm doing so with
> CLANG_DEFAULT_CXX_STDLIB=libc++, just if you wonder why the following
> matters to me...)
> With the current behavior, -lunwind passed to Clang will pull in the system
> default non-GNU /usr/lib/libunwind.so.8. IMO it should take LLVM's version
> in /opt/clang/lib/libunwind.so.1
>
> That's why I would like to change the default order for searching libraries
> in Clang. Currently (and most probably due to historical reasons), the
> Driver adds paths next to GCC first.
> I propose to prefer paths next to the installed Clang. You can find the
> patch doing the change here: https://reviews.llvm.org/D26244
>
> However, this has a potential impact as Chandler pointed out in above patch
> if Clang is installed to /opt/software together with other software.
> With my proposed change, Clang will link against
> /opt/software/lib/libunrelated.so when passed -lunrelated whereas before it
> had preferred /usr/lib/libunrelated.so (assuming that the library exists in
> both places).
> Is that considered an important scenario or can we expect our users to
> install Clang separately?
>
> Any other feedback on that topic?
I think the best way forward is to create a separate runtime libraries
directory for clang (similarly to how gcc has one). For example,
the x86 targets could use:

  /usr/lib/clang/5.0.0/lib/linux/i386
  /usr/lib/clang/5.0.0/lib/linux/x86_64

i.e. the same directory as compiler-rt uses right now + arch
subdirectory matching suffixes used by compiler-rt libraries.

Those (dedicated) directories could safely be prepended to library
search path without risking accidentally pulling in unintended
libraries. You could install LLVM libunwind there without risking it
getting mixed with system libunwind.

We already use a similar solution for clang-suite. If you'd like to
take a look at our patches, they're at [1,2]. Sadly, they're ugly
and I didn't find time so far to clean them up and submit.

[1]:https://github.com/pathscale/clang-suite/blob/master/patches/psclang/0005-Driver-Always-add-L-paths-for-clang-runtime-director.patch
[2]:https://github.com/pathscale/clang-suite/blob/master/patches/psclang/0006-Driver-Add-rpaths-for-internal-libdirs.patch

--
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

attachment0 (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Prefer libraries installed next to Clang over those from GCC

Ammarguellat, Zahira via cfe-dev
On Mon, Jan 30, 2017 at 2:04 PM, Michał Górny via cfe-dev <[hidden email]> wrote:
I think the best way forward is to create a separate runtime libraries
directory for clang (similarly to how gcc has one). For example,
the x86 targets could use:

  /usr/lib/clang/5.0.0/lib/linux/i386
  /usr/lib/clang/5.0.0/lib/linux/x86_64

i.e. the same directory as compiler-rt uses right now + arch
subdirectory matching suffixes used by compiler-rt libraries.

+1, let's do this! Putting the architecture in the name or our compiler runtime libraries creates needless build system complexity. 

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Prefer libraries installed next to Clang over those from GCC

Ammarguellat, Zahira via cfe-dev
In reply to this post by Ammarguellat, Zahira via cfe-dev
> -----Original Message-----

> From: Dr D. Chisnall [mailto:[hidden email]] On Behalf Of David
> Chisnall
> Sent: Monday, January 30, 2017 6:04 PM
> To: Hal Finkel
> Cc: Hahnfeld, Jonas; [hidden email]
> Subject: Re: [cfe-dev] [RFC] Prefer libraries installed next to Clang over those
> from GCC
>
> On 30 Jan 2017, at 16:57, Hal Finkel via cfe-dev <[hidden email]>
> wrote:
> > I think this is an important usage scenario, but because of the complication
> Chandler mentioned, I don't think we can enable it by default. I think it is
> very reasonable to have some CMake flag that enables this behavior.
>
> Unless we’re also passing -rpath or similar, then this may also lead to
> confusion where you link against the /opt/clang/lib/libunwind.so, but end up
> using /usr/lib/libunwind.so.  In this particular example, you’re relying on the
> fact that the two libunwinds have different SO versions, but that’s by no
> means guaranteed.
On the systems I work with, I can ensure that LD_LIBRARY_PATH is set correctly. This will result in the correct libraries to be linked.

>
> Libunwind is a very bad motivation for this in general because:
>
> - It’s an implementation of a standard ABI
> - The LLVM and non-GNU versions (and the GNU fork of the non-GNU
> version) are intended to be compatible

Should we then teach libcxx and libcxxabi to link against an already installed libunwind? Currently, there's only LIBCXXABI_USE_LLVM_UNWINDER which requires LLVM's libunwind to be built...

> - Linking both will cause things to go very badly wrong at run time

That's probably the point where I got it wrong last time: I had quite a lot of test failures...

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Prefer libraries installed next to Clang over those from GCC

Ammarguellat, Zahira via cfe-dev
In reply to this post by Ammarguellat, Zahira via cfe-dev

Hi Hal,

 

Thanks for the idea: I will adapt the patch to introduce a new CMake flag and to not change the default behavior.

 

Thanks,

Jonas

 

From: Hal Finkel [mailto:[hidden email]]
Sent: Monday, January 30, 2017 5:58 PM
To: Hahnfeld, Jonas; [hidden email]
Subject: Re: [cfe-dev] [RFC] Prefer libraries installed next to Clang over those from GCC

 

 

On 01/30/2017 09:09 AM, Hahnfeld, Jonas via cfe-dev wrote:

Hi,
 
Background: I'm on CentOS 7 which has GCC 4.8 installed in /usr/bin and I'm
installing Clang to, let's say, /opt/clang. (I'm doing so with
CLANG_DEFAULT_CXX_STDLIB=libc++, just if you wonder why the following
matters to me...)
With the current behavior, -lunwind passed to Clang will pull in the system
default non-GNU /usr/lib/libunwind.so.8. IMO it should take LLVM's version
in /opt/clang/lib/libunwind.so.1
 
That's why I would like to change the default order for searching libraries
in Clang. Currently (and most probably due to historical reasons), the
Driver adds paths next to GCC first.
I propose to prefer paths next to the installed Clang. You can find the
patch doing the change here: https://reviews.llvm.org/D26244
 
However, this has a potential impact as Chandler pointed out in above patch
if Clang is installed to /opt/software together with other software.
With my proposed change, Clang will link against
/opt/software/lib/libunrelated.so when passed -lunrelated whereas before it
had preferred /usr/lib/libunrelated.so (assuming that the library exists in
both places).
Is that considered an important scenario or can we expect our users to
install Clang separately?
 
Any other feedback on that topic?


I think this is an important usage scenario, but because of the complication Chandler mentioned, I don't think we can enable it by default. I think it is very reasonable to have some CMake flag that enables this behavior.

 -Hal


 
Jonas
 
--
Jonas Hahnfeld, MATSE-Auszubildender
 
IT Center
Group: High Performance Computing
Division: Computational Science and Engineering
RWTH Aachen University
Seffenter Weg 23
D 52074  Aachen (Germany)
[hidden email]
www.itc.rwth-aachen.de
 
 




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Prefer libraries installed next to Clang over those from GCC

Ammarguellat, Zahira via cfe-dev
In reply to this post by Ammarguellat, Zahira via cfe-dev

> On 31 Jan 2017, at 00:49, Hahnfeld, Jonas <[hidden email]> wrote:
>
>> -----Original Message-----
>> From: Dr D. Chisnall [mailto:[hidden email]] On Behalf Of David
>> Chisnall
>> Sent: Monday, January 30, 2017 6:04 PM
>> To: Hal Finkel
>> Cc: Hahnfeld, Jonas; [hidden email]
>> Subject: Re: [cfe-dev] [RFC] Prefer libraries installed next to Clang over those
>> from GCC
>>
>> On 30 Jan 2017, at 16:57, Hal Finkel via cfe-dev <[hidden email]>
>> wrote:
>>> I think this is an important usage scenario, but because of the complication
>> Chandler mentioned, I don't think we can enable it by default. I think it is
>> very reasonable to have some CMake flag that enables this behavior.
>>
>> Unless we’re also passing -rpath or similar, then this may also lead to
>> confusion where you link against the /opt/clang/lib/libunwind.so, but end up
>> using /usr/lib/libunwind.so.  In this particular example, you’re relying on the
>> fact that the two libunwinds have different SO versions, but that’s by no
>> means guaranteed.
>
> On the systems I work with, I can ensure that LD_LIBRARY_PATH is set correctly. This will result in the correct libraries to be linked.

Which is fine, if you’re (a) building everything with clang (no gcc-built shared libraries) and (b) able to [remember to] put that in your environment whenever you run a clang-built thing.

>
>>
>> Libunwind is a very bad motivation for this in general because:
>>
>> - It’s an implementation of a standard ABI
>> - The LLVM and non-GNU versions (and the GNU fork of the non-GNU
>> version) are intended to be compatible
>
> Should we then teach libcxx and libcxxabi to link against an already installed libunwind? Currently, there's only LIBCXXABI_USE_LLVM_UNWINDER which requires LLVM's libunwind to be built…

Doesn’t it do this automatically if you don’t specify to use the LLVM unwinder?  On FreeBSD, we use the unwinder from libgcc_s / libgcc_eh (which actually is the LLVM unwinder on head, but is called libgcc_* for compatibility) and libcxxrt as the C++ runtime library and all of this seems to just work.


>> - Linking both will cause things to go very badly wrong at run time
>
> That's probably the point where I got it wrong last time: I had quite a lot of test failures...

Yes, and making it easier to get into this situation doesn’t sound like a good plan.  The unwinder and C++ ABI library are both low-level system components and just ripping them out and replacing them needs almost as much thought as replacing libc.  Making it a thing that end-users might to by mistake makes me very nervous.

David

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Loading...