Clang codegen tests and LLVM passes

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Clang codegen tests and LLVM passes

Kristof Beyls via cfe-dev
Hello.

This was brought up several times now, but looks like it needs to be
mentioned more globally. Clang has many (~1516) front-end codegen tests.
Out of those tests, 364 tests are likely "broken":
$ clang/test$ grep -rl " -O[1-3]" | wc -l
364

Clang codegen tests should not be checking anything more than
clang IR codegen itself. They should not be testing the asm output.
They should not be invoking any middle-end optimization passes.

I suspect some of those 364 tests are as fragile.
It would be great if no new tests would have this design issue.
It would also be great to reduce the number of such tests.

In particular, this time i stumbled into this with
https://reviews.llvm.org/D67318
that broke clang/test/CodeGen/arm_acle.c
Given the nature of that test (it doesn't just greedily check too much, so it
could be relaxed to still pass, it basically contains end-to-end testing
for certain ARM intrinsics) so it can't really be relaxed.
I had to disable that test as i'm not sure how it should be fixed.

Roman.
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Clang codegen tests and LLVM passes

Kristof Beyls via cfe-dev
On Mon, Sep 16, 2019 at 11:03 AM Roman Lebedev via cfe-dev <[hidden email]> wrote:
Hello.

This was brought up several times now, but looks like it needs to be
mentioned more globally. Clang has many (~1516) front-end codegen tests.
Out of those tests, 364 tests are likely "broken":
$ clang/test$ grep -rl " -O[1-3]" | wc -l
364

It's an issue, but it's not as bad as that. Many tests use `-O1 -disable-llvm-passes` to get "optimized" IR output from clang which is slightly different. This matters for lifetime markers, for example.

I tried this query and got these results:
$ git grep -l " -O[1-3]"  ../clang/test | xargs grep -L -l ' -disable' | wc -l
99

So, 99/~1516 -> ~6.5% tests use the bad pattern. Some of the tests are truly integration tests, checking that clang emits an IR pattern that LLVM can optimize, but I think most are not.

I think the real issue here is that writing CHECK lines for clang's IR output is a pain, so people lean on the optimizer to clean up the IR first so they can match something that's more logical. If we did something to improve that, the situation wouldn't be as bad.

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev