Clang failing lots of FrontendC* tests

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Clang failing lots of FrontendC* tests

Jeffrey Yasskin
I managed to convince check-lit to run the FrontendC tests using clang
by applying the following patch to my config.status (yes, it's a hack,
and you'll have to change the paths to match your system):

=========
661,664c661,664
< s,@LLVMGCCCOMMAND@,|#_!!_#|llvm-gcc,g
< s,@LLVMGXXCOMMAND@,|#_!!_#|llvm-g++,g
< s,@LLVMGCC@,|#_!!_#|,g
< s,@LLVMGXX@,|#_!!_#|,g
---
> s,@LLVMGCCCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang,g
> s,@LLVMGXXCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang++,g
> s,@LLVMGCC@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang,g
> s,@LLVMGXX@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang++,g
675,676c675,676
< s,@LLVMGCCDIR@,|#_!!_#|,g
< s,@LLVMGCC_LANGS@,|#_!!_#|,g
---
> s,@LLVMGCCDIR@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin,g
> s,@LLVMGCC_LANGS@,|#_!!_#|c\,c++\,objc\,obj-c++,g
========

The following 110 tests failed, although the LLVMC ones don't look
like real failures, and some of the ObjC ones are just warnings. I
intend to just ignore these failures in my own testing, but I thought
you guys might like to know. The full check-lit output is attached.

********************
Unexpected Passing Tests (3):
    LLVM :: FrontendC++/2009-07-16-PrivateCopyConstructor.cpp
    LLVM :: FrontendC/2004-02-13-IllegalVararg.c
    LLVM :: FrontendC/2006-09-18-fwrite-cast-crash.c

********************
Failing Tests (110):
    LLVM :: FrontendC++/2005-02-20-BrokenReferenceTest.cpp
    LLVM :: FrontendC++/2005-07-21-VirtualBaseAccess.cpp
    LLVM :: FrontendC++/2006-09-27-Debug-Protection.cpp
    LLVM :: FrontendC++/2006-11-30-Pubnames.cpp
    LLVM :: FrontendC++/2007-01-02-UnboundedArray.cpp
    LLVM :: FrontendC++/2007-03-27-FunctionVarRename.cpp
    LLVM :: FrontendC++/2007-04-11-InlineStorageClassC++.cpp
    LLVM :: FrontendC++/2007-07-29-RestrictPtrArg.cpp
    LLVM :: FrontendC++/2007-07-29-RestrictRefArg.cpp
    LLVM :: FrontendC++/2007-08-01-RestrictMethod.cpp
    LLVM :: FrontendC++/2009-02-16-CtorNames-dbg.cpp
    LLVM :: FrontendC++/2009-05-04-PureConstNounwind.cpp
    LLVM :: FrontendC++/2009-06-30-ByrefBlock.cpp
    LLVM :: FrontendC++/2009-07-15-LineNumbers.cpp
    LLVM :: FrontendC++/2009-12-23-MissingSext.cpp
    LLVM :: FrontendC++/2010-02-17-DbgArtificialArg.cpp
    LLVM :: FrontendC++/alignstack.cpp
    LLVM :: FrontendC++/m64-ptr.cpp
    LLVM :: FrontendC++/ptr-to-method-devirt.cpp
    LLVM :: FrontendC++/varargs.cpp
    LLVM :: FrontendC/2003-06-29-MultipleFunctionDefinition.c
    LLVM :: FrontendC/2003-08-06-BuiltinSetjmpLongjmp.c
    LLVM :: FrontendC/2003-12-14-ExternInlineSupport.c
    LLVM :: FrontendC/2004-01-08-ExternInlineRedefine.c
    LLVM :: FrontendC/2004-02-13-Memset.c
    LLVM :: FrontendC/2004-03-16-AsmRegisterCrash.c
    LLVM :: FrontendC/2004-11-27-VariableSizeInStructure.c
    LLVM :: FrontendC/2005-02-27-MarkGlobalConstant.c
    LLVM :: FrontendC/2005-07-20-SqrtNoErrno.c
    LLVM :: FrontendC/2005-10-18-VariableSizedElementCrash.c
    LLVM :: FrontendC/2006-03-03-MissingInitializer.c
    LLVM :: FrontendC/2006-07-31-PR854.c
    LLVM :: FrontendC/2006-12-14-ordered_expr.c
    LLVM :: FrontendC/2007-02-04-WITH_SIZE_EXPR.c
    LLVM :: FrontendC/2007-02-05-nested.c
    LLVM :: FrontendC/2007-02-16-VariableSizeStructArg.c
    LLVM :: FrontendC/2007-02-16-WritableStrings.c
    LLVM :: FrontendC/2007-02-25-C-DotDotDot.c
    LLVM :: FrontendC/2007-03-06-VarSizeInStruct1.c
    LLVM :: FrontendC/2007-03-06-VarSizeInStruct2.c
    LLVM :: FrontendC/2007-03-27-VarLengthArray.c
    LLVM :: FrontendC/2007-04-11-InlineStorageClassC89.c
    LLVM :: FrontendC/2007-04-11-InlineStorageClassC99.c
    LLVM :: FrontendC/2007-04-24-VolatileStructCopy.c
    LLVM :: FrontendC/2007-05-07-NestedStructReturn.c
    LLVM :: FrontendC/2007-05-08-PCH.c
    LLVM :: FrontendC/2007-06-15-AnnotateAttribute.c
    LLVM :: FrontendC/2007-07-29-RestrictPtrArg.c
    LLVM :: FrontendC/2007-09-20-GcrootAttribute.c
    LLVM :: FrontendC/2007-10-01-BuildArrayRef.c
    LLVM :: FrontendC/2007-10-15-VoidPtr.c
    LLVM :: FrontendC/2007-11-07-CopyAggregateAlign.c
    LLVM :: FrontendC/2007-11-07-ZeroAggregateAlign.c
    LLVM :: FrontendC/2007-11-27-SExtZExt.c
    LLVM :: FrontendC/2007-12-VarArrayDebug.c
    LLVM :: FrontendC/2008-01-07-UnusualIntSize.c
    LLVM :: FrontendC/2008-01-11-ChainConsistency.c
    LLVM :: FrontendC/2008-02-11-AnnotateBuiltin.c
    LLVM :: FrontendC/2008-03-24-BitField-And-Alloca.c
    LLVM :: FrontendC/2008-04-08-NoExceptions.c
    LLVM :: FrontendC/2008-05-19-AlwaysInline.c
    LLVM :: FrontendC/2008-07-08-FAbsAttributes.c
    LLVM :: FrontendC/2008-07-29-EHLabel.ll
    LLVM :: FrontendC/2008-08-07-AlignPadding1.c
    LLVM :: FrontendC/2008-08-07-AlignPadding2.c
    LLVM :: FrontendC/2008-11-11-AnnotateStructFieldAttribute.c
    LLVM :: FrontendC/2009-01-05-BlockInlining.c
    LLVM :: FrontendC/2009-03-09-WeakDeclarations-1.c
    LLVM :: FrontendC/2009-04-22-UnknownSize.c
    LLVM :: FrontendC/2009-05-04-EnumInreg.c
    LLVM :: FrontendC/2009-06-18-StaticInitTailPadPack.c
    LLVM :: FrontendC/2009-07-17-VoidParameter.c
    LLVM :: FrontendC/2009-07-22-StructLayout.c
    LLVM :: FrontendC/2009-08-11-AsmBlocksComplexJumpTarget.c
    LLVM :: FrontendC/2009-09-24-SqrtErrno.c
    LLVM :: FrontendC/2009-12-07-BitFieldAlignment.c
    LLVM :: FrontendC/2010-01-13-MemBarrier.c
    LLVM :: FrontendC/2010-02-15-DbgStaticVar.c
    LLVM :: FrontendC/alignstack.c
    LLVM :: FrontendC/cstring-align.c
    LLVM :: FrontendC/fp-logical.c
    LLVM :: FrontendC/libcalls-d.c
    LLVM :: FrontendC/libcalls-ld.c
    LLVM :: FrontendC/libcalls.c
    LLVM :: FrontendC/nested-functions.c
    LLVM :: FrontendC/pr4349.c
    LLVM :: FrontendC/redef-ext-inline.c
    LLVM :: FrontendC/union-align.c
    LLVM :: FrontendC/wchar-const.c
    LLVM :: FrontendObjC/2007-09-25-EH.m
    LLVM :: FrontendObjC/2008-11-12-Metadata.m
    LLVM :: FrontendObjC/2008-11-24-ConstCFStrings.m
    LLVM :: FrontendObjC/2008-11-25-Blocks.m
    LLVM :: FrontendObjC/2009-01-26-WriteBarrier-2.m
    LLVM :: FrontendObjC/2009-04-14-AsmSection.m
    LLVM :: FrontendObjC/2009-04-27-bitfield-vs-ivar.m
    LLVM :: FrontendObjC/2009-08-05-utf16.m
    LLVM :: FrontendObjC/2009-08-17-DebugInfo.m
    LLVM :: FrontendObjC/2010-02-11-fwritable-stringsBug.m
    LLVM :: LLVMC/C++/dash-x.cpp
    LLVM :: LLVMC/C++/hello.cpp
    LLVM :: LLVMC/C++/together.cpp
    LLVM :: LLVMC/C/emit-llvm.c
    LLVM :: LLVMC/C/hello.c
    LLVM :: LLVMC/C/include.c
    LLVM :: LLVMC/C/opt-test.c
    LLVM :: LLVMC/C/sink.c
    LLVM :: LLVMC/C/wall.c
    LLVM :: LLVMC/ObjC++/hello.mm
    LLVM :: LLVMC/ObjC/hello.m


Jeffrey

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

frontend_tests.out (300K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Clang failing lots of FrontendC* tests

Douglas Gregor

On Mar 6, 2010, at 5:47 PM, Jeffrey Yasskin wrote:

I managed to convince check-lit to run the FrontendC tests using clang
by applying the following patch to my config.status (yes, it's a hack,
and you'll have to change the paths to match your system):

=========
661,664c661,664
< s,@LLVMGCCCOMMAND@,|#_!!_#|llvm-gcc,g
< s,@LLVMGXXCOMMAND@,|#_!!_#|llvm-g++,g
< s,@LLVMGCC@,|#_!!_#|,g
< s,@LLVMGXX@,|#_!!_#|,g
---
s,@LLVMGCCCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang,g
s,@LLVMGXXCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang++,g
s,@LLVMGCC@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang,g
s,@LLVMGXX@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang++,g
675,676c675,676
< s,@LLVMGCCDIR@,|#_!!_#|,g
< s,@LLVMGCC_LANGS@,|#_!!_#|,g
---
s,@LLVMGCCDIR@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin,g
s,@LLVMGCC_LANGS@,|#_!!_#|c\,c++\,objc\,obj-c++,g
========

The following 110 tests failed, although the LLVMC ones don't look
like real failures, and some of the ObjC ones are just warnings. I
intend to just ignore these failures in my own testing, but I thought
you guys might like to know. The full check-lit output is attached.

Our view on the FrontendC* tests is that they're testing very specific behaviors of llvm-gcc that we don't necessarily care about for Clang. At some point, I'd like to look at the failures to convince myself that these tests aren't uncovering real bugs, but even when I do... I'd rather move the useful tests in Clang's regression test suite.

- Doug

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Clang failing lots of FrontendC* tests

Jeffrey Yasskin
On Sat, Mar 6, 2010 at 9:09 PM, Douglas Gregor <[hidden email]> wrote:

>
> On Mar 6, 2010, at 5:47 PM, Jeffrey Yasskin wrote:
>
> I managed to convince check-lit to run the FrontendC tests using clang
> by applying the following patch to my config.status (yes, it's a hack,
> and you'll have to change the paths to match your system):
>
> =========
> 661,664c661,664
> < s,@LLVMGCCCOMMAND@,|#_!!_#|llvm-gcc,g
> < s,@LLVMGXXCOMMAND@,|#_!!_#|llvm-g++,g
> < s,@LLVMGCC@,|#_!!_#|,g
> < s,@LLVMGXX@,|#_!!_#|,g
> ---
>
> s,@LLVMGCCCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang,g
>
> s,@LLVMGXXCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang++,g
>
> s,@LLVMGCC@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang,g
>
> s,@LLVMGXX@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang++,g
>
> 675,676c675,676
> < s,@LLVMGCCDIR@,|#_!!_#|,g
> < s,@LLVMGCC_LANGS@,|#_!!_#|,g
> ---
>
> s,@LLVMGCCDIR@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin,g
>
> s,@LLVMGCC_LANGS@,|#_!!_#|c\,c++\,objc\,obj-c++,g
>
> ========
>
> The following 110 tests failed, although the LLVMC ones don't look
> like real failures, and some of the ObjC ones are just warnings. I
> intend to just ignore these failures in my own testing, but I thought
> you guys might like to know. The full check-lit output is attached.
>
> Our view on the FrontendC* tests is that they're testing very specific
> behaviors of llvm-gcc that we don't necessarily care about for Clang. At
> some point, I'd like to look at the failures to convince myself that these
> tests aren't uncovering real bugs, but even when I do... I'd rather move the
> useful tests in Clang's regression test suite.

Thanks for looking. There are some interesting tests in FrontendC for
how llvm's backend handles debug information, for example, that aren't
covered by the rest of the test suite. (I know because they found a
bug in one of my recent changes that wasn't caught by the rest of the
suite.) Possibly we should just compile all of the FrontendC* tests to
IR and add them elsewhere in the test suite?
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Clang failing lots of FrontendC* tests

Douglas Gregor


Sent from my iPhone

On Mar 6, 2010, at 10:45 PM, Jeffrey Yasskin <[hidden email]>  
wrote:

> On Sat, Mar 6, 2010 at 9:09 PM, Douglas Gregor <[hidden email]>  
> wrote:
>>
>> On Mar 6, 2010, at 5:47 PM, Jeffrey Yasskin wrote:
>>
>> I managed to convince check-lit to run the FrontendC tests using  
>> clang
>> by applying the following patch to my config.status (yes, it's a  
>> hack,
>> and you'll have to change the paths to match your system):
>>
>> =========
>> 661,664c661,664
>> < s,@LLVMGCCCOMMAND@,|#_!!_#|llvm-gcc,g
>> < s,@LLVMGXXCOMMAND@,|#_!!_#|llvm-g++,g
>> < s,@LLVMGCC@,|#_!!_#|,g
>> < s,@LLVMGXX@,|#_!!_#|,g
>> ---
>>
>> s,@LLVMGCCCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/
>> bin/clang,g
>>
>> s,@LLVMGXXCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/
>> bin/clang++,g
>>
>> s,@LLVMGCC@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/
>> clang,g
>>
>> s,@LLVMGXX@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang
>> ++,g
>>
>> 675,676c675,676
>> < s,@LLVMGCCDIR@,|#_!!_#|,g
>> < s,@LLVMGCC_LANGS@,|#_!!_#|,g
>> ---
>>
>> s,@LLVMGCCDIR@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin,g
>>
>> s,@LLVMGCC_LANGS@,|#_!!_#|c\,c++\,objc\,obj-c++,g
>>
>> ========
>>
>> The following 110 tests failed, although the LLVMC ones don't look
>> like real failures, and some of the ObjC ones are just warnings. I
>> intend to just ignore these failures in my own testing, but I thought
>> you guys might like to know. The full check-lit output is attached.
>>
>> Our view on the FrontendC* tests is that they're testing very  
>> specific
>> behaviors of llvm-gcc that we don't necessarily care about for  
>> Clang. At
>> some point, I'd like to look at the failures to convince myself  
>> that these
>> tests aren't uncovering real bugs, but even when I do... I'd rather  
>> move the
>> useful tests in Clang's regression test suite.
>
> Thanks for looking. There are some interesting tests in FrontendC for
> how llvm's backend handles debug information, for example, that aren't
> covered by the rest of the test suite. (I know because they found a
> bug in one of my recent changes that wasn't caught by the rest of the
> suite.) Possibly we should just compile all of the FrontendC* tests to
> IR and add them elsewhere in the test suite?

I think that makes sense. Perhaps Daniel has an opinion on this?
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev