-ferror-limit=1 by default?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

-ferror-limit=1 by default?

Kristof Beyls via cfe-dev
Hi All!

Replying to John here on twitter:

https://twitter.com/echristo/status/1192506726920011776

where he was noticing the helpfulness of -ferror-limit=1 as making a
huge usability difference. I acknowledge that being able to do good
error recovery and keep going is useful in some cases, but for day to
day interactive this might be a better usability default? Especially
with the compiler fast enough for the edit/compile cycle?

Thoughts? Shall we do this?

-eric
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: -ferror-limit=1 by default?

Kristof Beyls via cfe-dev


> -----Original Message-----
> From: cfe-dev <[hidden email]> On Behalf Of Eric
> Christopher via cfe-dev
> Sent: Thursday, November 7, 2019 1:23 PM
> To: Clang Dev <[hidden email]>
> Cc: Richard Smith <[hidden email]>
> Subject: [cfe-dev] -ferror-limit=1 by default?
>
> Hi All!
>
> Replying to John here on twitter:
>
> https://twitter.com/echristo/status/1192506726920011776
>
> where he was noticing the helpfulness of -ferror-limit=1 as making a
> huge usability difference. I acknowledge that being able to do good
> error recovery and keep going is useful in some cases, but for day to
> day interactive this might be a better usability default? Especially
> with the compiler fast enough for the edit/compile cycle?
>
> Thoughts? Shall we do this?

This feels like a case where interactive use and build-system use
might want different defaults, because the overhead of a starting up
a big build is typically a lot higher.  Would this be an imposition
on all the build systems out there, that for usability they would have
to set a higher error-limit?
--paulr

>
> -eric
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: -ferror-limit=1 by default?

Kristof Beyls via cfe-dev
In reply to this post by Kristof Beyls via cfe-dev
I'd vote against it, personally - I think I often look at more than one error at a time - especially with an API change/rename/reordering of function arguments/etc - I can go through at least a handful of the call site updates in one pass.

(we /might/ be able to do some experiments at Google scale with this sort of thing (if we found a good way around the build caching issues (eg: could run a sample based on file names so the cache didn't thrash due to actually random changes in output)) & see if developers take less or more time on average to converge on a successful build)

Taking a compiler-development look at the "30 pages of crap" to see where it goes so far off the rails as to be unhelpful might be useful. I know Richard's (& maybe Richard Trieu and other folks too) done some things/is looking into ways to be more selective about overload candidates and template instantiation stacks, etc.

On Thu, Nov 7, 2019 at 10:23 AM Eric Christopher via cfe-dev <[hidden email]> wrote:
Hi All!

Replying to John here on twitter:

https://twitter.com/echristo/status/1192506726920011776

where he was noticing the helpfulness of -ferror-limit=1 as making a
huge usability difference. I acknowledge that being able to do good
error recovery and keep going is useful in some cases, but for day to
day interactive this might be a better usability default? Especially
with the compiler fast enough for the edit/compile cycle?

Thoughts? Shall we do this?

-eric
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: -ferror-limit=1 by default?

Kristof Beyls via cfe-dev
In reply to this post by Kristof Beyls via cfe-dev
Responding to you and Dave at once because you brought up similar issues:

I can definitely see that IDEs/larger build systems might want to set
a different value versus the command line compilation. I'm definitely
looking at changing the default and not removing the option :)

The Google scale experiment or the refactoring change is a good
counter argument in a lot of ways - I've done that a lot in my own use
of the compiler and knowing the next 10 or so cases is pretty useful.
That said, I'm not certain it's more or less useful than "simple"
clean errors that are easy to spot in the terminal rather than having
to scroll to more of them.

I've definitely gotten a bunch of comments that go with "what about
tooling uses" which I'll count refactoring under and a number of
people wanting just the first error in my earlier thread.

Seems to be a decent set of things. I'm uncertain how much experiment
we want to run here - but I could see it being a useful change to see
what people think.

-eric

On Thu, Nov 7, 2019 at 10:35 AM Robinson, Paul <[hidden email]> wrote:

>
>
>
> > -----Original Message-----
> > From: cfe-dev <[hidden email]> On Behalf Of Eric
> > Christopher via cfe-dev
> > Sent: Thursday, November 7, 2019 1:23 PM
> > To: Clang Dev <[hidden email]>
> > Cc: Richard Smith <[hidden email]>
> > Subject: [cfe-dev] -ferror-limit=1 by default?
> >
> > Hi All!
> >
> > Replying to John here on twitter:
> >
> > https://twitter.com/echristo/status/1192506726920011776
> >
> > where he was noticing the helpfulness of -ferror-limit=1 as making a
> > huge usability difference. I acknowledge that being able to do good
> > error recovery and keep going is useful in some cases, but for day to
> > day interactive this might be a better usability default? Especially
> > with the compiler fast enough for the edit/compile cycle?
> >
> > Thoughts? Shall we do this?
>
> This feels like a case where interactive use and build-system use
> might want different defaults, because the overhead of a starting up
> a big build is typically a lot higher.  Would this be an imposition
> on all the build systems out there, that for usability they would have
> to set a higher error-limit?
> --paulr
>
> >
> > -eric
> > _______________________________________________
> > cfe-dev mailing list
> > [hidden email]
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: -ferror-limit=1 by default?

Kristof Beyls via cfe-dev
In reply to this post by Kristof Beyls via cfe-dev
Note that there also is a -Wfatal-errors option.

Michael
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: -ferror-limit=1 by default?

Kristof Beyls via cfe-dev
There is! I think one of the things I like is that -ferror-limit also
says "if you'd like more warnings then you can use a different option"
rather than something like -Wfatal-errors.

Also thinking about usability defaults for the greater good rather
than my use case. I set my own command line options like I like :)

-eric

On Thu, Nov 7, 2019 at 1:56 PM Michael Kruse <[hidden email]> wrote:
>
> Note that there also is a -Wfatal-errors option.
>
> Michael
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: -ferror-limit=1 by default?

Kristof Beyls via cfe-dev
Without having an opinion on what should be the default, I find
-Wfatal-errors more idiomatic. Errors after the first are not robust
and I have no use case for "I need exactly 2 errors". Rather, if the
following errors might be useful, I'd remove the -Wfatal-errors and
less/scroll though the entire list.

Michael

Am Do., 7. Nov. 2019 um 16:05 Uhr schrieb Eric Christopher <[hidden email]>:

>
> There is! I think one of the things I like is that -ferror-limit also
> says "if you'd like more warnings then you can use a different option"
> rather than something like -Wfatal-errors.
>
> Also thinking about usability defaults for the greater good rather
> than my use case. I set my own command line options like I like :)
>
> -eric
>
> On Thu, Nov 7, 2019 at 1:56 PM Michael Kruse <[hidden email]> wrote:
> >
> > Note that there also is a -Wfatal-errors option.
> >
> > Michael
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: -ferror-limit=1 by default?

Kristof Beyls via cfe-dev
In reply to this post by Kristof Beyls via cfe-dev
On 07/11/2019 18:23, Eric Christopher via cfe-dev wrote:

> Hi All!
>
> Replying to John here on twitter:
>
> https://twitter.com/echristo/status/1192506726920011776
>
> where he was noticing the helpfulness of -ferror-limit=1 as making a
> huge usability difference. I acknowledge that being able to do good
> error recovery and keep going is useful in some cases, but for day to
> day interactive this might be a better usability default? Especially
> with the compiler fast enough for the edit/compile cycle?
>
> Thoughts? Shall we do this?

Anecdotally, I've set -ferror-limit=3 over the last six months  for a
few projects that I work on.  A few observations:

  - The compile time for non-trivial C++ compilation units is
sufficiently long that an extra compile cycle before getting to the
second error is a noticeable delay.

  - With a limit of 3 (picked in a completely non-scientific way), I
have a roughly 50:50 chance of being able to fix all three errors at the
same time and a higher chance of being able to fix 2.

  - Occasionally, I will have 20 instances of the same root-cause.  If
an API changes in a header, it's useful to find all callees.  I think we
could do something useful with heuristics to catch things like this (for
example, if the same invalid identifier appears in multiple places in
what appears to be the same kind of scope, report them together).  With
-ferror-limit=3 currently, I have multiple compile cycles to find them
all (typically, I just do one build with no error limit in these cases
today).

  - If I miss a closing brace, 0 of the errors that clang reports are
useful, at any error limit.  A single check that counts the number of
open and close braces and, if they are different numbers with a fast
response would be very helpful.  At the moment, it can be 10-20 seconds
for a heavily templated file before I see an error that is caused by
something that the lexer could have seen (all of the errors are then
nonsense, because they're caused by trying to parse a method definition
inside another method or similar).

TL;DR: An error limit of 3 gives me too many errors some times and not
enough at other times, I think a limit of 1 would give not enough a lot
more often.

David
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: -ferror-limit=1 by default?

Kristof Beyls via cfe-dev
In reply to this post by Kristof Beyls via cfe-dev
We work pretty hard to make errors after the first as robust as possible; if the user experience is that our recovery from errors in some particular case is bad, we should try to fix that before resorting to a default of -Wfatal-errors / -ferror-limit=1.

We also have some unusual cases where the first error reported is actually a consequence of a later error (or warning) that we've not yet diagnosed. This can happen due to our taking the unintended path through a parse ambiguity, or (occasionally) due to our attempt to work out how to diagnose an error (particularly in typo correction) encountering further errors. Stopping at the first error can in rare cases completely hide the error message that explains what's wrong.

I think it's better to view this as a class of individually-fixable problems rather than a systemic problem, identify the particular cases where our error recovery is bad, and improve those cases.

On Thu, Nov 7, 2019 at 3:10 PM Michael Kruse <[hidden email]> wrote:
Without having an opinion on what should be the default, I find
-Wfatal-errors more idiomatic. Errors after the first are not robust
and I have no use case for "I need exactly 2 errors". Rather, if the
following errors might be useful, I'd remove the -Wfatal-errors and
less/scroll though the entire list.

Michael

Am Do., 7. Nov. 2019 um 16:05 Uhr schrieb Eric Christopher <[hidden email]>:
>
> There is! I think one of the things I like is that -ferror-limit also
> says "if you'd like more warnings then you can use a different option"
> rather than something like -Wfatal-errors.
>
> Also thinking about usability defaults for the greater good rather
> than my use case. I set my own command line options like I like :)
>
> -eric
>
> On Thu, Nov 7, 2019 at 1:56 PM Michael Kruse <[hidden email]> wrote:
> >
> > Note that there also is a -Wfatal-errors option.
> >
> > Michael

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: -ferror-limit=1 by default?

Kristof Beyls via cfe-dev
For me it was less about error recovery and that they're useful, but
rather primarily that they're "too many" and that focusing on
individual ones was more helpful - also things like maybe =3 rather
than =1 or something.

At any rate, it appears the result is mixed enough that it's worth
just leaving it as is and maybe documenting behavior a bit more
clearly as a helpful option.

On Tue, Nov 12, 2019 at 4:38 PM Richard Smith <[hidden email]> wrote:

>
> We work pretty hard to make errors after the first as robust as possible; if the user experience is that our recovery from errors in some particular case is bad, we should try to fix that before resorting to a default of -Wfatal-errors / -ferror-limit=1.
>
> We also have some unusual cases where the first error reported is actually a consequence of a later error (or warning) that we've not yet diagnosed. This can happen due to our taking the unintended path through a parse ambiguity, or (occasionally) due to our attempt to work out how to diagnose an error (particularly in typo correction) encountering further errors. Stopping at the first error can in rare cases completely hide the error message that explains what's wrong.
>
> I think it's better to view this as a class of individually-fixable problems rather than a systemic problem, identify the particular cases where our error recovery is bad, and improve those cases.
>
> On Thu, Nov 7, 2019 at 3:10 PM Michael Kruse <[hidden email]> wrote:
>>
>> Without having an opinion on what should be the default, I find
>> -Wfatal-errors more idiomatic. Errors after the first are not robust
>> and I have no use case for "I need exactly 2 errors". Rather, if the
>> following errors might be useful, I'd remove the -Wfatal-errors and
>> less/scroll though the entire list.
>>
>> Michael
>>
>> Am Do., 7. Nov. 2019 um 16:05 Uhr schrieb Eric Christopher <[hidden email]>:
>> >
>> > There is! I think one of the things I like is that -ferror-limit also
>> > says "if you'd like more warnings then you can use a different option"
>> > rather than something like -Wfatal-errors.
>> >
>> > Also thinking about usability defaults for the greater good rather
>> > than my use case. I set my own command line options like I like :)
>> >
>> > -eric
>> >
>> > On Thu, Nov 7, 2019 at 1:56 PM Michael Kruse <[hidden email]> wrote:
>> > >
>> > > Note that there also is a -Wfatal-errors option.
>> > >
>> > > Michael
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev