RFC: [MSVC] Late parsing of in-class defined member functions in template classes after instantiation for better compatibility.

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

RFC: [MSVC] Late parsing of in-class defined member functions in template classes after instantiation for better compatibility.

Neil Nelson via cfe-dev

Motivation

=========

Currently clang in MSVC compatibility mode has many troubles with MS provided headers (ATL / WTL), which use template classes.

MSVC actively uses unqualified lookup in dependent bases, lookup at the instantiation point (non-dependent names may be resolved on things declared later) etc. and all this stuff is the main cause of incompatibility between clang and MSVC.

Clang tries to emulate MSVC behavior but it may fail in many cases. For example:

 

template <class T>

struct S : public T {

       S(T) {}

       S() {}

       void foo() {

             S<T> bar(baz);

       }

};

 

struct S1 {

       int baz;

};

 

S<S1> s;

 

Similar code exists in WTL header files (for example, atlctrlx.h, atlctrlw.h etc.). Clang fails with the following messages:

 

test.cpp:6:12: error: unknown type name 'baz'

  S<T> bar(baz);

           ^

test.cpp:6:11: warning: parentheses were disambiguated as a function declaration [-Wvexing-parse]

  S<T> bar(baz);

          ^~~~~

test.cpp:6:12: note: add a pair of parentheses to declare a variable

  S<T> bar(baz);

 

In addition, it may produce invalid AST from MSVC point of view:

 

typedef int baz;

template <class T>

struct S : public T {

       S() {}

       S(int) {}

       void foo() {

             S<T> bar(baz);

             ++bar.barz;

       }

};

 

struct S1 {

       static int baz;

       int barz;

};

int S1::baz = 0;

 

S<S1> s;

 

int main() {

       s.foo();

       return 0;

}

 

test.cpp:8:8: error: member reference base type 'S<S1> (baz)' (aka 'S<S1> (int)') is not a structure or union  

  ++bar.barz;

    ~~~^~~~~

test.cpp:21:5: note: in instantiation of member function 'S<S1>::foo' requested here

  s.foo();

 

Though these examples are not "quite correct" but required for MSVC compatibility and parsing system headers.

 

Proposed solution

==============

It seems the only truly universal and generic solution would be to perform parsing of in-class defined member functions during instantiation of template classes (in MSVC compatibility mode only).

To do this clang could store lexed tokens for member functions definitions within ClassTemplateDecl for later parsing during template instantiation.   

It will allow resolving many possible issues with lookup in dependent base classes and removing many already existing MSVC-specific hacks/workarounds from the clang code.

 

Please kindly provide feedback for the proposal stated above; we (Intel compiler team) plan to handle implementation to reach full clang compatibility with ATL / WTL headers and improve MSVC compatibility in general.

--
Best regards,
Alexey Bataev
=============
Software Engineer
Intel Compiler Team

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: RFC: [MSVC] Late parsing of in-class defined member functions in template classes after instantiation for better compatibility.

Neil Nelson via cfe-dev

Reid, Richard, any thoughts about this?

Best regards,
Alexey Bataev
=============
Software Engineer
Intel Compiler Team
20.05.2016 20:36, Alexey Bataev via cfe-dev пишет:

Motivation

=========

Currently clang in MSVC compatibility mode has many troubles with MS provided headers (ATL / WTL), which use template classes.

MSVC actively uses unqualified lookup in dependent bases, lookup at the instantiation point (non-dependent names may be resolved on things declared later) etc. and all this stuff is the main cause of incompatibility between clang and MSVC.

Clang tries to emulate MSVC behavior but it may fail in many cases. For example:

 

template <class T>

struct S : public T {

       S(T) {}

       S() {}

       void foo() {

             S<T> bar(baz);

       }

};

 

struct S1 {

       int baz;

};

 

S<S1> s;

 

Similar code exists in WTL header files (for example, atlctrlx.h, atlctrlw.h etc.). Clang fails with the following messages:

 

test.cpp:6:12: error: unknown type name 'baz'

  S<T> bar(baz);

           ^

test.cpp:6:11: warning: parentheses were disambiguated as a function declaration [-Wvexing-parse]

  S<T> bar(baz);

          ^~~~~

test.cpp:6:12: note: add a pair of parentheses to declare a variable

  S<T> bar(baz);

 

In addition, it may produce invalid AST from MSVC point of view:

 

typedef int baz;

template <class T>

struct S : public T {

       S() {}

       S(int) {}

       void foo() {

             S<T> bar(baz);

             ++bar.barz;

       }

};

 

struct S1 {

       static int baz;

       int barz;

};

int S1::baz = 0;

 

S<S1> s;

 

int main() {

       s.foo();

       return 0;

}

 

test.cpp:8:8: error: member reference base type 'S<S1> (baz)' (aka 'S<S1> (int)') is not a structure or union  

  ++bar.barz;

    ~~~^~~~~

test.cpp:21:5: note: in instantiation of member function 'S<S1>::foo' requested here

  s.foo();

 

Though these examples are not "quite correct" but required for MSVC compatibility and parsing system headers.

 

Proposed solution

==============

It seems the only truly universal and generic solution would be to perform parsing of in-class defined member functions during instantiation of template classes (in MSVC compatibility mode only).

To do this clang could store lexed tokens for member functions definitions within ClassTemplateDecl for later parsing during template instantiation.   

It will allow resolving many possible issues with lookup in dependent base classes and removing many already existing MSVC-specific hacks/workarounds from the clang code.

 

Please kindly provide feedback for the proposal stated above; we (Intel compiler team) plan to handle implementation to reach full clang compatibility with ATL / WTL headers and improve MSVC compatibility in general.

--
Best regards,
Alexey Bataev
=============
Software Engineer
Intel Compiler Team


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: RFC: [MSVC] Late parsing of in-class defined member functions in template classes after instantiation for better compatibility.

Neil Nelson via cfe-dev
We've thought about it before, but we've always been able to overcome
each compatibility issue as it was encountered:
https://llvm.org/bugs/show_bug.cgi?id=18714

I think we need a sample ATL and WRL-based app that uses all of the
relevant headers, and instantiates all of the popular templates.
Clearly you guys are seeing issues that we never saw while building
Chromium. It's hard for us to say if a new template instantiation mode
is necessary when we're not looking at the same problems.

We're also trying to keep the scope of our compatibility limited to
whats in "system headers". If you expand the scope to be "have perfect
MSVC compatibility", then yes, the only logical solution is to to
token-based instantiation of member functions. I think if that's what
you need, then go ahead and try implementing that mode, and if it's
lightweight and non-invasive, we could take it upstream. It would be a
good replacement for -fdelayed-template-parsing.

I think now that Microsoft has started shipping clang as an option in
Visual Studio, future version of these headers are likely to get
cleaner, and eventually I'd like to turn off
-fdelayed-template-parsing by default.

On Wed, May 25, 2016 at 4:45 AM, Alexey Bataev <[hidden email]> wrote:

> Reid, Richard, any thoughts about this?
>
> Best regards,
> Alexey Bataev
> =============
> Software Engineer
> Intel Compiler Team
>
> 20.05.2016 20:36, Alexey Bataev via cfe-dev пишет:
>
> Motivation
>
> =========
>
> Currently clang in MSVC compatibility mode has many troubles with MS
> provided headers (ATL / WTL), which use template classes.
>
> MSVC actively uses unqualified lookup in dependent bases, lookup at the
> instantiation point (non-dependent names may be resolved on things declared
> later) etc. and all this stuff is the main cause of incompatibility between
> clang and MSVC.
>
> Clang tries to emulate MSVC behavior but it may fail in many cases. For
> example:
>
>
>
> template <class T>
>
> struct S : public T {
>
>        S(T) {}
>
>        S() {}
>
>        void foo() {
>
>              S<T> bar(baz);
>
>        }
>
> };
>
>
>
> struct S1 {
>
>        int baz;
>
> };
>
>
>
> S<S1> s;
>
>
>
> Similar code exists in WTL header files (for example, atlctrlx.h, atlctrlw.h
> etc.). Clang fails with the following messages:
>
>
>
> test.cpp:6:12: error: unknown type name 'baz'
>
>   S<T> bar(baz);
>
>            ^
>
> test.cpp:6:11: warning: parentheses were disambiguated as a function
> declaration [-Wvexing-parse]
>
>   S<T> bar(baz);
>
>           ^~~~~
>
> test.cpp:6:12: note: add a pair of parentheses to declare a variable
>
>   S<T> bar(baz);
>
>
>
> In addition, it may produce invalid AST from MSVC point of view:
>
>
>
> typedef int baz;
>
> template <class T>
>
> struct S : public T {
>
>        S() {}
>
>        S(int) {}
>
>        void foo() {
>
>              S<T> bar(baz);
>
>              ++bar.barz;
>
>        }
>
> };
>
>
>
> struct S1 {
>
>        static int baz;
>
>        int barz;
>
> };
>
> int S1::baz = 0;
>
>
>
> S<S1> s;
>
>
>
> int main() {
>
>        s.foo();
>
>        return 0;
>
> }
>
>
>
> test.cpp:8:8: error: member reference base type 'S<S1> (baz)' (aka 'S<S1>
> (int)') is not a structure or union
>
>   ++bar.barz;
>
>     ~~~^~~~~
>
> test.cpp:21:5: note: in instantiation of member function 'S<S1>::foo'
> requested here
>
>   s.foo();
>
>
>
> Though these examples are not "quite correct" but required for MSVC
> compatibility and parsing system headers.
>
>
>
> Proposed solution
>
> ==============
>
> It seems the only truly universal and generic solution would be to perform
> parsing of in-class defined member functions during instantiation of
> template classes (in MSVC compatibility mode only).
>
> To do this clang could store lexed tokens for member functions definitions
> within ClassTemplateDecl for later parsing during template instantiation.
>
> It will allow resolving many possible issues with lookup in dependent base
> classes and removing many already existing MSVC-specific hacks/workarounds
> from the clang code.
>
>
>
> Please kindly provide feedback for the proposal stated above; we (Intel
> compiler team) plan to handle implementation to reach full clang
> compatibility with ATL / WTL headers and improve MSVC compatibility in
> general.
>
> --
>
> Best regards,
> Alexey Bataev
> =============
> Software Engineer
> Intel Compiler Team
>
>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: RFC: [MSVC] Late parsing of in-class defined member functions in template classes after instantiation for better compatibility.

Neil Nelson via cfe-dev
Reid,
Thanks for your answer.
We have many problems with samples shipped with ATL/WTL. And the main
problem is that clang founds errors not in these samples, but in ATL/WTL
header files used by these samples. I mentioned couple of examples used
in headers, which cause clang to emit errors, but also there are some
others. And it is getting harder and harder to find a way to fix these
incompatibility issues. So I think it is a wise decision to start
implementation of token-based instantiation.
We will start work on this.

Best regards,
Alexey Bataev
=============
Software Engineer
Intel Compiler Team

25.05.2016 20:35, Reid Kleckner пишет:

> We've thought about it before, but we've always been able to overcome
> each compatibility issue as it was encountered:
> https://llvm.org/bugs/show_bug.cgi?id=18714
>
> I think we need a sample ATL and WRL-based app that uses all of the
> relevant headers, and instantiates all of the popular templates.
> Clearly you guys are seeing issues that we never saw while building
> Chromium. It's hard for us to say if a new template instantiation mode
> is necessary when we're not looking at the same problems.
>
> We're also trying to keep the scope of our compatibility limited to
> whats in "system headers". If you expand the scope to be "have perfect
> MSVC compatibility", then yes, the only logical solution is to to
> token-based instantiation of member functions. I think if that's what
> you need, then go ahead and try implementing that mode, and if it's
> lightweight and non-invasive, we could take it upstream. It would be a
> good replacement for -fdelayed-template-parsing.
>
> I think now that Microsoft has started shipping clang as an option in
> Visual Studio, future version of these headers are likely to get
> cleaner, and eventually I'd like to turn off
> -fdelayed-template-parsing by default.
>
> On Wed, May 25, 2016 at 4:45 AM, Alexey Bataev <[hidden email]> wrote:
>> Reid, Richard, any thoughts about this?
>>
>> Best regards,
>> Alexey Bataev
>> =============
>> Software Engineer
>> Intel Compiler Team
>>
>> 20.05.2016 20:36, Alexey Bataev via cfe-dev пишет:
>>
>> Motivation
>>
>> =========
>>
>> Currently clang in MSVC compatibility mode has many troubles with MS
>> provided headers (ATL / WTL), which use template classes.
>>
>> MSVC actively uses unqualified lookup in dependent bases, lookup at the
>> instantiation point (non-dependent names may be resolved on things declared
>> later) etc. and all this stuff is the main cause of incompatibility between
>> clang and MSVC.
>>
>> Clang tries to emulate MSVC behavior but it may fail in many cases. For
>> example:
>>
>>
>>
>> template <class T>
>>
>> struct S : public T {
>>
>>         S(T) {}
>>
>>         S() {}
>>
>>         void foo() {
>>
>>               S<T> bar(baz);
>>
>>         }
>>
>> };
>>
>>
>>
>> struct S1 {
>>
>>         int baz;
>>
>> };
>>
>>
>>
>> S<S1> s;
>>
>>
>>
>> Similar code exists in WTL header files (for example, atlctrlx.h, atlctrlw.h
>> etc.). Clang fails with the following messages:
>>
>>
>>
>> test.cpp:6:12: error: unknown type name 'baz'
>>
>>    S<T> bar(baz);
>>
>>             ^
>>
>> test.cpp:6:11: warning: parentheses were disambiguated as a function
>> declaration [-Wvexing-parse]
>>
>>    S<T> bar(baz);
>>
>>            ^~~~~
>>
>> test.cpp:6:12: note: add a pair of parentheses to declare a variable
>>
>>    S<T> bar(baz);
>>
>>
>>
>> In addition, it may produce invalid AST from MSVC point of view:
>>
>>
>>
>> typedef int baz;
>>
>> template <class T>
>>
>> struct S : public T {
>>
>>         S() {}
>>
>>         S(int) {}
>>
>>         void foo() {
>>
>>               S<T> bar(baz);
>>
>>               ++bar.barz;
>>
>>         }
>>
>> };
>>
>>
>>
>> struct S1 {
>>
>>         static int baz;
>>
>>         int barz;
>>
>> };
>>
>> int S1::baz = 0;
>>
>>
>>
>> S<S1> s;
>>
>>
>>
>> int main() {
>>
>>         s.foo();
>>
>>         return 0;
>>
>> }
>>
>>
>>
>> test.cpp:8:8: error: member reference base type 'S<S1> (baz)' (aka 'S<S1>
>> (int)') is not a structure or union
>>
>>    ++bar.barz;
>>
>>      ~~~^~~~~
>>
>> test.cpp:21:5: note: in instantiation of member function 'S<S1>::foo'
>> requested here
>>
>>    s.foo();
>>
>>
>>
>> Though these examples are not "quite correct" but required for MSVC
>> compatibility and parsing system headers.
>>
>>
>>
>> Proposed solution
>>
>> ==============
>>
>> It seems the only truly universal and generic solution would be to perform
>> parsing of in-class defined member functions during instantiation of
>> template classes (in MSVC compatibility mode only).
>>
>> To do this clang could store lexed tokens for member functions definitions
>> within ClassTemplateDecl for later parsing during template instantiation.
>>
>> It will allow resolving many possible issues with lookup in dependent base
>> classes and removing many already existing MSVC-specific hacks/workarounds
>> from the clang code.
>>
>>
>>
>> Please kindly provide feedback for the proposal stated above; we (Intel
>> compiler team) plan to handle implementation to reach full clang
>> compatibility with ATL / WTL headers and improve MSVC compatibility in
>> general.
>>
>> --
>>
>> Best regards,
>> Alexey Bataev
>> =============
>> Software Engineer
>> Intel Compiler Team
>>
>>

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: RFC: [MSVC] Late parsing of in-class defined member functions in template classes after instantiation for better compatibility.

Neil Nelson via cfe-dev
In reply to this post by Neil Nelson via cfe-dev
On Wed, May 25, 2016 at 8:35 PM, Reid Kleckner via cfe-dev <[hidden email]> wrote:
I think now that Microsoft has started shipping clang as an option in
Visual Studio, future version of these headers are likely to get
cleaner, and eventually I'd like to turn off
-fdelayed-template-parsing by default.
Reid, don't hold your breath...


(though maybe this changed since and MS compiler engineers can clarify current standing)

Yours,
Andrey
=====
Software Engineer
Intel Compiler Team

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev