Clang Modules: modularization of standard headers required?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Clang Modules: modularization of standard headers required?

Xin Wang via cfe-dev
I am trying to enable Clang Modules using latest version of clang on a tiny example, but getting the following error: 

error: 'std::vector' has different definitions in different modules


>> To get any benefit out of modules, one needs to introduce module maps for software libraries starting at the bottom of the stack.

Does it mean that modularization of system/std headers is **required** before going upper levels?
I am not using modulemaps on standard headers and would like to avoid that for now. 

2) When I try some older version of clang (say from April 2017) it doesn't show this error on the same input. Does it mean there was regression happened in between? Or the current behavior is actually expected?


Below comes more details and reproduction.
Full message:

In file included from main.cpp:1:
In file included from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/vector:64:
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_vector.h:210:11: error: 'std::vector' has different definitions in
      different modules; defined here
    class vector : protected _Vector_base<_Tp, _Alloc>
          ^
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_vector.h:210:11: note: definition in module 'experimental_Range_h' is here
    class vector : protected _Vector_base<_Tp, _Alloc>
          ^
1 error generated.


The files I am using:
// -- main.cpp:
#include <vector>
#include "Range.h"

int main () {
  return 0;
}

// -- Range.h:
#pragma once

#include <string>
#include "Bits.h"
#include <vector>

// -- Bit.h:
#pragma once
#include <iterator>

// -- module.modulemap:
module experimental_Bits_h {
   header "Bits.h"
   export *
}

module experimental_Range_h {
   header "Range.h"
   export *
}

Commandline: 
$ clang++ --std=c++14 -I .  -fmodules -fcxx-modules -fmodules-cache-path=./modules_out/_module_cache -c main.cpp -o main.o


$ clang++ --version
clang version 5.0.0 (http://llvm.org/git/clang.git 742c4c842393005d91a6f805f665b077014d061a) (http://llvm.org/git/llvm.git a98fd55665d422389d69d94efe631fac93d11173)

-- 
Thanks,
Dmitry

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clang Modules: modularization of standard headers required?

Xin Wang via cfe-dev
1) If you don't modularize STL, then you just won't get any
performance benefits when parsing the STL headers. Also some headers
are sometimes incompatible with modules unless they're in their own
[sub]module. E.g. you could have a header that is doing `class vector
{ MEM_TYPE a; }` and in one inclusion context we have `#define
MEM_TYPE long` and in another `#define MEM_TYPE unsigned`. This breaks
ODR checking but usually is prevented by modularizing the header that
contains `vector`.

2)  It's probably not a regression, but IIRC this ODR check that
recognizes the issue was introduced between those two revisions.
Either it's a code issue like in my example above or it's a bug in
this check. Try filing a bug report and you can usually workaround for
now by modularizing those headers.

By the way, you can mount the modulemap in the right spot on your
filesystem with clang's filesystem overlays. E.g. write a JSON file
like this [1] and then mount modulemaps for libc[2] and STL[3] by
writing the right paths into the file (or let the build system do that
for you) and then use "-ivfsoverlay<FILE>".

[1] https://github.com/root-project/root/blob/master/build/unix/modulemap.overlay.yaml.in
[2] https://github.com/root-project/root/blob/master/build/unix/libc.modulemap
[3] https://github.com/root-project/root/blob/master/build/unix/stl.cppmap

- Raphael


2017-07-01 7:23 GMT+02:00 Dmitry Panin via cfe-dev <[hidden email]>:

> I am trying to enable Clang Modules using latest version of clang on a tiny
> example, but getting the following error:
>
> error: 'std::vector' has different definitions in different modules
>
> 1) Documentation from
> http://clang.llvm.org/docs/Modules.html#modularizing-a-platform says
>
>>> To get any benefit out of modules, one needs to introduce module maps for
>>> software libraries starting at the bottom of the stack.
>
> Does it mean that modularization of system/std headers is **required**
> before going upper levels?
> I am not using modulemaps on standard headers and would like to avoid that
> for now.
>
> 2) When I try some older version of clang (say from April 2017) it doesn't
> show this error on the same input. Does it mean there was regression
> happened in between? Or the current behavior is actually expected?
>
>
> Below comes more details and reproduction.
> Full message:
>
> In file included from main.cpp:1:
> In file included from
> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/vector:64:
> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_vector.h:210:11:
> error: 'std::vector' has different definitions in
>       different modules; defined here
>     class vector : protected _Vector_base<_Tp, _Alloc>
>           ^
> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_vector.h:210:11:
> note: definition in module 'experimental_Range_h' is here
>     class vector : protected _Vector_base<_Tp, _Alloc>
>           ^
> 1 error generated.
>
>
> The files I am using:
> // -- main.cpp:
> #include <vector>
> #include "Range.h"
>
> int main () {
>   return 0;
> }
>
> // -- Range.h:
> #pragma once
>
> #include <string>
> #include "Bits.h"
> #include <vector>
>
> // -- Bit.h:
> #pragma once
> #include <iterator>
>
> // -- module.modulemap:
> module experimental_Bits_h {
>    header "Bits.h"
>    export *
> }
>
> module experimental_Range_h {
>    header "Range.h"
>    export *
> }
>
> Commandline:
> $ clang++ --std=c++14 -I .  -fmodules -fcxx-modules
> -fmodules-cache-path=./modules_out/_module_cache -c main.cpp -o main.o
>
>
> $ clang++ --version
> clang version 5.0.0 (http://llvm.org/git/clang.git
> 742c4c842393005d91a6f805f665b077014d061a) (http://llvm.org/git/llvm.git
> a98fd55665d422389d69d94efe631fac93d11173)
>
> --
> Thanks,
> Dmitry
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clang Modules: modularization of standard headers required?

Xin Wang via cfe-dev
1) If you don't modularize STL, then you just won't get any
performance benefits when parsing the STL headers. Also some headers
are sometimes incompatible with modules unless they're in their own
[sub]module. E.g. you could have a header that is doing `class vector
{ MEM_TYPE a; }` and in one inclusion context we have `#define
MEM_TYPE long` and in another `#define MEM_TYPE unsigned`. This breaks
ODR checking but usually is prevented by modularizing the header that
contains `vector`.

1) Yes, I understand that clang would still need to parse STL headers for each TU unless they are coming from modules. Apparently when I try to use modules on STL it fails with the similar ODR error but now on glibc and kernel headers and then exposes a lot more errors on our codebase itself.  I would like transition to modules to be more incremental. It's fine to be initially without performance benefits.
But if it's indeed incompatibility like you mentioned then it doesn't seem to be the other way.

 
2)  It's probably not a regression, but IIRC this ODR check that
recognizes the issue was introduced between those two revisions.
Either it's a code issue like in my example above or it's a bug in
this check. Try filing a bug report and you can usually workaround for
now by modularizing those headers.

Just to confirm my understanding: if vector definitions are exactly the same (even coming from different modules) then this ODR check shouldn't be triggering, right? 
I'll double check what's happening and file a bug if that's the issue with the check.

Thanks for the pointers to filesystem overlays! I wasn't aware of such mechanism.
 

-- 
Dmitry

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clang Modules: modularization of standard headers required?

Xin Wang via cfe-dev
2017-07-01 20:58 GMT+02:00 Dmitry Panin <[hidden email]>:

>> 1) If you don't modularize STL, then you just won't get any
>> performance benefits when parsing the STL headers. Also some headers
>> are sometimes incompatible with modules unless they're in their own
>> [sub]module. E.g. you could have a header that is doing `class vector
>> { MEM_TYPE a; }` and in one inclusion context we have `#define
>> MEM_TYPE long` and in another `#define MEM_TYPE unsigned`. This breaks
>> ODR checking but usually is prevented by modularizing the header that
>> contains `vector`.
>
>
> 1) Yes, I understand that clang would still need to parse STL headers for
> each TU unless they are coming from modules. Apparently when I try to use
> modules on STL it fails with the similar ODR error but now on glibc and
> kernel headers and then exposes a lot more errors on our codebase itself.  I
> would like transition to modules to be more incremental. It's fine to be
> initially without performance benefits.
> But if it's indeed incompatibility like you mentioned then it doesn't seem
> to be the other way.
>
>
>>
>> 2)  It's probably not a regression, but IIRC this ODR check that
>> recognizes the issue was introduced between those two revisions.
>> Either it's a code issue like in my example above or it's a bug in
>> this check. Try filing a bug report and you can usually workaround for
>> now by modularizing those headers.
>
>
> Just to confirm my understanding: if vector definitions are exactly the same
> (even coming from different modules) then this ODR check shouldn't be
> triggering, right?
> I'll double check what's happening and file a bug if that's the issue with
> the check.

Sounds about right. In your case it's either that that the definition
depends on the inclusion context of stl_vector.h or the ODR checking
has a bug.

As you're anyway building clang from source it seems, you could try
going to this line where we print the error
(https://github.com/llvm-mirror/clang/blob/1124518852a283a657d709877b9b474f1fc7e5fc/lib/Serialization/ASTReader.cpp#L9383
) and add
```
FirstDecl->dumpColor();
SecondDecl->dumpColor();
```
and then recompile and retry. Then you also get the AST of the vector
decl printed by clang on stderr (which is much more useful to debug).

> Thanks for the pointers to filesystem overlays! I wasn't aware of such
> mechanism.
>
>
> --
> Dmitry
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Loading...