[FileCheck] RFC: Add support for line anchors.

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[FileCheck] RFC: Add support for line anchors.

Keane, Erich via cfe-dev
Hello,

I was wondering about extending FileCheck to enable creating line
anchors. These are numeric variables that hold the value of the line
number that where they were defined.

The motivation for this comes from test cases using clang-based
diagnostics which often include notes attached to source locations in
different parts of the file. In order to test for the correct location
of the note, the line number has to be written explicitly or as an,
often large, offset to the current line. This harms both readability
and maintainability. Using this new system one could append a line of
interest with an anchor-comment and refer back to it inside FileCheck.

I have created a basic patch that implements this here
https://reviews.llvm.org/D84037 but it definitely needs a few looks
over by people who are more clued up on the internal of FileCheck.

The current syntax, based off this patch, is as follows:
  - Added a command line option called `anchor-prefix` which is a
comma-seperated list of prefixes to be used when declaring anchors.
This is defaulted to `LINE-ANCHOR`
  - To declare a anchor in the test file use
    `LINE-ANCHOR: ANCHOR_NAME`
    note: If you specify a different anchor-prefix using the command
line, use that name instead of `LINE-ANCHOR`
    ANCHOR_NAME Follows the rules all other variable names aside from
the fact it can't start with '$'.
  - When referring to an anchor in a check use the same numeric
variable syntax that FileCheck already supports:
    `CHECK: [[#ANCHOR_NAME]][[#ANCHOR_NAME+1]]`

Here is a brief (contrived) example of the usage of this:
```
#define BAD_FUNCTION() badFunction() // LINE-ANCHOR: BAD_FUNC
// Further down in the file
  BAD_FUNCTION();
CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
CHECK-NOTES :[[#BAD-FUNC]]:3: note: expanded from macro 'BAD_FUNCTION'
```

Regards,
Nathan James


_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [FileCheck] RFC: Add support for line anchors.

Keane, Erich via cfe-dev
Hi Nathan,

On Fri, Jul 17, 2020 at 12:23 PM Nathan James via cfe-dev <[hidden email]> wrote:
Hello,

I was wondering about extending FileCheck to enable creating line
anchors. These are numeric variables that hold the value of the line
number that where they were defined.

I think something like this could be useful.  However, I think it would be more useful to have a general directive for defining FileCheck variables inline without trying to capture from input.  For example:

```
#define BAD_FUNCTION() badFunction() // CHECK-DEFINE: [[#BAD_FUNC:@LINE]]
// Further down in the file
  BAD_FUNCTION();
CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
CHECK-NOTES :[[#BAD_FUNC]]:3: note: expanded from macro 'BAD_FUNCTION'
```

The exact syntax is debatable.

I think this form is more useful because it can also define strings or numerics (or maybe even patterns) to be reused in multiple FileCheck directives across multiple FileCheck calls from different RUN lines.  Currently, you either have to capture such a variable from the input or specify it with -D on every FileCheck call that needs it.

James Henderson: Weren't you working on something like this?

Thanks.

Joel
 

The motivation for this comes from test cases using clang-based
diagnostics which often include notes attached to source locations in
different parts of the file. In order to test for the correct location
of the note, the line number has to be written explicitly or as an,
often large, offset to the current line. This harms both readability
and maintainability. Using this new system one could append a line of
interest with an anchor-comment and refer back to it inside FileCheck.

I have created a basic patch that implements this here
https://reviews.llvm.org/D84037 but it definitely needs a few looks
over by people who are more clued up on the internal of FileCheck.

The current syntax, based off this patch, is as follows:
  - Added a command line option called `anchor-prefix` which is a
comma-seperated list of prefixes to be used when declaring anchors.
This is defaulted to `LINE-ANCHOR`
  - To declare a anchor in the test file use
    `LINE-ANCHOR: ANCHOR_NAME`
    note: If you specify a different anchor-prefix using the command
line, use that name instead of `LINE-ANCHOR`
    ANCHOR_NAME Follows the rules all other variable names aside from
the fact it can't start with '$'.
  - When referring to an anchor in a check use the same numeric
variable syntax that FileCheck already supports:
    `CHECK: [[#ANCHOR_NAME]][[#ANCHOR_NAME+1]]`

Here is a brief (contrived) example of the usage of this:
```
#define BAD_FUNCTION() badFunction() // LINE-ANCHOR: BAD_FUNC
// Further down in the file
  BAD_FUNCTION();
CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
CHECK-NOTES :[[#BAD-FUNC]]:3: note: expanded from macro 'BAD_FUNCTION'
```

Regards,
Nathan James


_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [FileCheck] RFC: Add support for line anchors.

Keane, Erich via cfe-dev
On Fri, Jul 17, 2020 at 2:52 PM Joel E. Denny <[hidden email]> wrote:
Hi Nathan,

On Fri, Jul 17, 2020 at 12:23 PM Nathan James via cfe-dev <[hidden email]> wrote:
Hello,

I was wondering about extending FileCheck to enable creating line
anchors. These are numeric variables that hold the value of the line
number that where they were defined.

I think something like this could be useful.  However, I think it would be more useful to have a general directive for defining FileCheck variables inline without trying to capture from input.  For example:

```
#define BAD_FUNCTION() badFunction() // CHECK-DEFINE: [[#BAD_FUNC:@LINE]]
// Further down in the file
  BAD_FUNCTION();
CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
CHECK-NOTES :[[#BAD_FUNC]]:3: note: expanded from macro 'BAD_FUNCTION'
```

The exact syntax is debatable.

I think this form is more useful because it can also define strings or numerics (or maybe even patterns) to be reused in multiple FileCheck directives across multiple FileCheck calls from different RUN lines.  Currently, you either have to capture such a variable from the input or specify it with -D on every FileCheck call that needs it.

James Henderson: Weren't you working on something like this?

Thanks.

Joel
 

The motivation for this comes from test cases using clang-based
diagnostics which often include notes attached to source locations in
different parts of the file. In order to test for the correct location
of the note, the line number has to be written explicitly or as an,
often large, offset to the current line. This harms both readability
and maintainability. Using this new system one could append a line of
interest with an anchor-comment and refer back to it inside FileCheck.

Also, just in case you weren't aware, there is a Clang facility for checking its own diagnostics:


You'd put a directive for each diagnostic on or near the line it cites.

Joel
 

I have created a basic patch that implements this here
https://reviews.llvm.org/D84037 but it definitely needs a few looks
over by people who are more clued up on the internal of FileCheck.

The current syntax, based off this patch, is as follows:
  - Added a command line option called `anchor-prefix` which is a
comma-seperated list of prefixes to be used when declaring anchors.
This is defaulted to `LINE-ANCHOR`
  - To declare a anchor in the test file use
    `LINE-ANCHOR: ANCHOR_NAME`
    note: If you specify a different anchor-prefix using the command
line, use that name instead of `LINE-ANCHOR`
    ANCHOR_NAME Follows the rules all other variable names aside from
the fact it can't start with '$'.
  - When referring to an anchor in a check use the same numeric
variable syntax that FileCheck already supports:
    `CHECK: [[#ANCHOR_NAME]][[#ANCHOR_NAME+1]]`

Here is a brief (contrived) example of the usage of this:
```
#define BAD_FUNCTION() badFunction() // LINE-ANCHOR: BAD_FUNC
// Further down in the file
  BAD_FUNCTION();
CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
CHECK-NOTES :[[#BAD-FUNC]]:3: note: expanded from macro 'BAD_FUNCTION'
```

Regards,
Nathan James


_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [FileCheck] RFC: Add support for line anchors.

Keane, Erich via cfe-dev
In reply to this post by Keane, Erich via cfe-dev
Hi Joel,

That sounds like a very nice idea and definitely a direction I could
get behind. However I feel that outside the use case I suggested, this
functionality would only be used to compress CHECK lines that contain
repeated text, not saying its a bad or good thing though. WDYT?

~Nathan

On Fri, 2020-07-17 at 14:52 -0400, Joel E. Denny via cfe-dev wrote:

> Hi Nathan,
>
> On Fri, Jul 17, 2020 at 12:23 PM Nathan James via cfe-dev <
> [hidden email]> wrote:
> > Hello,
> >
> > I was wondering about extending FileCheck to enable creating line
> > anchors. These are numeric variables that hold the value of the
> > line
> > number that where they were defined.
>
> I think something like this could be useful.  However, I think it
> would be more useful to have a general directive for defining
> FileCheck variables inline without trying to capture from input.  For
> example:
>
> ```
> #define BAD_FUNCTION() badFunction() // CHECK-DEFINE:
> [[#BAD_FUNC:@LINE]]
> // Further down in the file
>   BAD_FUNCTION();
> CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
> CHECK-NOTES :[[#BAD_FUNC]]:3: note: expanded from macro
> 'BAD_FUNCTION'
> ```
>
> The exact syntax is debatable.
>
> I think this form is more useful because it can also define strings
> or numerics (or maybe even patterns) to be reused in multiple
> FileCheck directives across multiple FileCheck calls from different
> RUN lines.  Currently, you either have to capture such a variable
> from the input or specify it with -D on every FileCheck call that
> needs it.
>
> James Henderson: Weren't you working on something like this?
>
> Thanks.
>
> Joel
>  
> > The motivation for this comes from test cases using clang-based
> > diagnostics which often include notes attached to source locations
> > in
> > different parts of the file. In order to test for the correct
> > location
> > of the note, the line number has to be written explicitly or as an,
> > often large, offset to the current line. This harms both
> > readability
> > and maintainability. Using this new system one could append a line
> > of
> > interest with an anchor-comment and refer back to it inside
> > FileCheck.
> >
> > I have created a basic patch that implements this here
> > https://reviews.llvm.org/D84037 but it definitely needs a few looks
> > over by people who are more clued up on the internal of FileCheck.
> >
> > The current syntax, based off this patch, is as follows:
> >   - Added a command line option called `anchor-prefix` which is a
> > comma-seperated list of prefixes to be used when declaring anchors.
> > This is defaulted to `LINE-ANCHOR`
> >   - To declare a anchor in the test file use
> >     `LINE-ANCHOR: ANCHOR_NAME`
> >     note: If you specify a different anchor-prefix using the
> > command
> > line, use that name instead of `LINE-ANCHOR`
> >     ANCHOR_NAME Follows the rules all other variable names aside
> > from
> > the fact it can't start with '$'.
> >   - When referring to an anchor in a check use the same numeric
> > variable syntax that FileCheck already supports:
> >     `CHECK: [[#ANCHOR_NAME]][[#ANCHOR_NAME+1]]`
> >
> > Here is a brief (contrived) example of the usage of this:
> > ```
> > #define BAD_FUNCTION() badFunction() // LINE-ANCHOR: BAD_FUNC
> > // Further down in the file
> >   BAD_FUNCTION();
> > CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
> > CHECK-NOTES :[[#BAD-FUNC]]:3: note: expanded from macro
> > 'BAD_FUNCTION'
> > ```
> >
> > Regards,
> > Nathan James
> >
> >
> > _______________________________________________
> > cfe-dev mailing list
> > [hidden email]
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [FileCheck] RFC: Add support for line anchors.

Keane, Erich via cfe-dev
In reply to this post by Keane, Erich via cfe-dev
On Fri, 2020-07-17 at 14:59 -0400, Joel E. Denny via cfe-dev wrote:

> On Fri, Jul 17, 2020 at 2:52 PM Joel E. Denny <[hidden email]>
> wrote:
> > Hi Nathan,
> >
> > On Fri, Jul 17, 2020 at 12:23 PM Nathan James via cfe-dev <
> > [hidden email]> wrote:
> > > Hello,
> > >
> > > I was wondering about extending FileCheck to enable creating line
> > > anchors. These are numeric variables that hold the value of the
> > > line
> > > number that where they were defined.
> >
> > I think something like this could be useful.  However, I think it
> > would be more useful to have a general directive for defining
> > FileCheck variables inline without trying to capture from input.
> > For example:
> >
> > ```
> > #define BAD_FUNCTION() badFunction() // CHECK-DEFINE:
> > [[#BAD_FUNC:@LINE]]
> > // Further down in the file
> >   BAD_FUNCTION();
> > CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
> > CHECK-NOTES :[[#BAD_FUNC]]:3: note: expanded from macro
> > 'BAD_FUNCTION'
> > ```
> >
> > The exact syntax is debatable.
> >
> > I think this form is more useful because it can also define strings
> > or numerics (or maybe even patterns) to be reused in multiple
> > FileCheck directives across multiple FileCheck calls from different
> > RUN lines.  Currently, you either have to capture such a variable
> > from the input or specify it with -D on every FileCheck call that
> > needs it.
> >
> > James Henderson: Weren't you working on something like this?
> >
> > Thanks.
> >
> > Joel
> >  
> > > The motivation for this comes from test cases using clang-based
> > > diagnostics which often include notes attached to source
> > > locations in
> > > different parts of the file. In order to test for the correct
> > > location
> > > of the note, the line number has to be written explicitly or as
> > > an,
> > > often large, offset to the current line. This harms both
> > > readability
> > > and maintainability. Using this new system one could append a
> > > line of
> > > interest with an anchor-comment and refer back to it inside
> > > FileCheck.
>
> Also, just in case you weren't aware, there is a Clang facility for
> checking its own diagnostics:
>
> http://clang.llvm.org/doxygen/classclang_1_1VerifyDiagnosticConsumer.html#details
>
> You'd put a directive for each diagnostic on or near the line it
> cites.
>
> Joel
>  
The use case I was mainly thinking of was clang tools, specifically
clang-tidy. There you need to be able to check diagnostics and fix-its
in one file ideally. Right now you can't use the
VerifyDiagnosticConsumer in there and it also won't be able to veryify
fix-its.

~Nathan

> > > I have created a basic patch that implements this here
> > > https://reviews.llvm.org/D84037 but it definitely needs a few
> > > looks
> > > over by people who are more clued up on the internal of
> > > FileCheck.
> > >
> > > The current syntax, based off this patch, is as follows:
> > >   - Added a command line option called `anchor-prefix` which is a
> > > comma-seperated list of prefixes to be used when declaring
> > > anchors.
> > > This is defaulted to `LINE-ANCHOR`
> > >   - To declare a anchor in the test file use
> > >     `LINE-ANCHOR: ANCHOR_NAME`
> > >     note: If you specify a different anchor-prefix using the
> > > command
> > > line, use that name instead of `LINE-ANCHOR`
> > >     ANCHOR_NAME Follows the rules all other variable names aside
> > > from
> > > the fact it can't start with '$'.
> > >   - When referring to an anchor in a check use the same numeric
> > > variable syntax that FileCheck already supports:
> > >     `CHECK: [[#ANCHOR_NAME]][[#ANCHOR_NAME+1]]`
> > >
> > > Here is a brief (contrived) example of the usage of this:
> > > ```
> > > #define BAD_FUNCTION() badFunction() // LINE-ANCHOR: BAD_FUNC
> > > // Further down in the file
> > >   BAD_FUNCTION();
> > > CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
> > > CHECK-NOTES :[[#BAD-FUNC]]:3: note: expanded from macro
> > > 'BAD_FUNCTION'
> > > ```
> > >
> > > Regards,
> > > Nathan James
> > >
> > >
> > > _______________________________________________
> > > cfe-dev mailing list
> > > [hidden email]
> > > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [FileCheck] RFC: Add support for line anchors.

Keane, Erich via cfe-dev
In reply to this post by Keane, Erich via cfe-dev
On Fri, Jul 17, 2020 at 4:23 PM Nathan James <[hidden email]> wrote:
Hi Joel,

That sounds like a very nice idea and definitely a direction I could
get behind. However I feel that outside the use case I suggested, this
functionality would only be used to compress CHECK lines that contain
repeated text, not saying its a bad or good thing though. WDYT?

I can think of a few use cases:

1. Capturing `@LINE`, as you suggest.
2. Making a single directive's complex pattern more readable by extracting and naming components.
3. Avoiding the maintenance burden of repeating the same string, numeric, or pattern expression multiple times within a single directive or across multiple directives.

`-D` can sometimes help with #2 and #3.  However, it still has to be repeated across multiple RUN lines in some cases.  Moreover, it's sometimes better to define variables near their uses, but the RUN lines can be far away.

As an example of #2 and #3, I've occasionally written subtle patterns, such as `{{$[[:space:]]}}` for newline, that I wanted to use in multiple places.  It's not immediately obvious what that pattern does, but `[[NL]]` would be much clearer and could be documented once at its definition.

It just occurred to me that the new directive might require a "CHECK-DEF-CONST" form that is processed before all other directives.  Otherwise, forward-references wouldn't be possible.  You might want forward-references for #1, for example.

Joel
 

~Nathan

On Fri, 2020-07-17 at 14:52 -0400, Joel E. Denny via cfe-dev wrote:
> Hi Nathan,
>
> On Fri, Jul 17, 2020 at 12:23 PM Nathan James via cfe-dev <
> [hidden email]> wrote:
> > Hello,
> >
> > I was wondering about extending FileCheck to enable creating line
> > anchors. These are numeric variables that hold the value of the
> > line
> > number that where they were defined.
>
> I think something like this could be useful.  However, I think it
> would be more useful to have a general directive for defining
> FileCheck variables inline without trying to capture from input.  For
> example:
>
> ```
> #define BAD_FUNCTION() badFunction() // CHECK-DEFINE:
> [[#BAD_FUNC:@LINE]]
> // Further down in the file
>   BAD_FUNCTION();
> CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
> CHECK-NOTES :[[#BAD_FUNC]]:3: note: expanded from macro
> 'BAD_FUNCTION'
> ```
>
> The exact syntax is debatable.
>
> I think this form is more useful because it can also define strings
> or numerics (or maybe even patterns) to be reused in multiple
> FileCheck directives across multiple FileCheck calls from different
> RUN lines.  Currently, you either have to capture such a variable
> from the input or specify it with -D on every FileCheck call that
> needs it.
>
> James Henderson: Weren't you working on something like this?
>
> Thanks.
>
> Joel

> > The motivation for this comes from test cases using clang-based
> > diagnostics which often include notes attached to source locations
> > in
> > different parts of the file. In order to test for the correct
> > location
> > of the note, the line number has to be written explicitly or as an,
> > often large, offset to the current line. This harms both
> > readability
> > and maintainability. Using this new system one could append a line
> > of
> > interest with an anchor-comment and refer back to it inside
> > FileCheck.
> >
> > I have created a basic patch that implements this here
> > https://reviews.llvm.org/D84037 but it definitely needs a few looks
> > over by people who are more clued up on the internal of FileCheck.
> >
> > The current syntax, based off this patch, is as follows:
> >   - Added a command line option called `anchor-prefix` which is a
> > comma-seperated list of prefixes to be used when declaring anchors.
> > This is defaulted to `LINE-ANCHOR`
> >   - To declare a anchor in the test file use
> >     `LINE-ANCHOR: ANCHOR_NAME`
> >     note: If you specify a different anchor-prefix using the
> > command
> > line, use that name instead of `LINE-ANCHOR`
> >     ANCHOR_NAME Follows the rules all other variable names aside
> > from
> > the fact it can't start with '$'.
> >   - When referring to an anchor in a check use the same numeric
> > variable syntax that FileCheck already supports:
> >     `CHECK: [[#ANCHOR_NAME]][[#ANCHOR_NAME+1]]`
> >
> > Here is a brief (contrived) example of the usage of this:
> > ```
> > #define BAD_FUNCTION() badFunction() // LINE-ANCHOR: BAD_FUNC
> > // Further down in the file
> >   BAD_FUNCTION();
> > CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
> > CHECK-NOTES :[[#BAD-FUNC]]:3: note: expanded from macro
> > 'BAD_FUNCTION'
> > ```
> >
> > Regards,
> > Nathan James
> >
> >
> > _______________________________________________
> > cfe-dev mailing list
> > [hidden email]
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: [FileCheck] RFC: Add support for line anchors.

Keane, Erich via cfe-dev
As Joel mentioned, I was at one point looking at implementing a CHECK-DEFINE. The idea was to define a variable inline without it actually matching any text. My use-case was where a string might want reusing in different runs of FileCheck, perhaps sometimes with a -NOT after it. The separate runs meant that using -D was not particularly appealing, as it just meant defining the string in two different places, whilst the -NOT suffix meant reusing the CHECK line itself wasn't viable either. Repeating the string also meant an update in one place might not get done in the other place, and since CHECK-NOT patterns are brittle, the test might start passing spuriously.

Simple example:

# RUN: tool --success | FileCheck %s --check-prefix=SUCCESS
# RUN: tool --fail | FileCheck %s --check-prefix=FAIL

# SUCCESS: success message
# FAIL-NOT: success message

Could become:

# RUN: tool --success | FileCheck %s --check-prefixes=DEF,SUCCESS
# RUN: tool --fail | FileCheck %s --check-prefixes=DEF,FAIL

# DEF-DEFINE: [[MSG:success message]]
# SUCCESS: [[MSG]]
# FAIL-NOT: [[MSG]]

Unfortunately, this wasn't something I could sink much time into, and I've now put it on the back shelf for the foreseeable future. I'd be happy to put a patch up on Phabricator for others to pick up and work with, if they want?

I got a fair distance towards a prototype solution, but I wasn't particularly happy with the internal implementation, plus there were several cases where it either didn't work, or broke something else. The biggest problem I faced was that you probably don't want DEFINE directives defining variables that appear before it, but FileCheck's two-pass design made it tricky to handle that.

Back to the original problem: I haven't thought this through completely, but I wonder if you could use [hidden email]'s recent "extract" tool to split the pattern file up used by FileCheck somehow (see https://reviews.llvm.org/D83834). By discarding the leading lines (using --no-leading-lines IIRC), you could make the @LINE directives more localised. Essentially it would look something like:

# RUN: extract input1.c --no-leading-lines %s > %t1.c
# RUN: do-a-thing %s | FileCheck %t1.c
#--- input1.c
this line causes some output # CHECK: [[@LINE]]: ...

Hope that helps!

James

On Fri, 17 Jul 2020 at 21:57, Joel E. Denny via cfe-dev <[hidden email]> wrote:
On Fri, Jul 17, 2020 at 4:23 PM Nathan James <[hidden email]> wrote:
Hi Joel,

That sounds like a very nice idea and definitely a direction I could
get behind. However I feel that outside the use case I suggested, this
functionality would only be used to compress CHECK lines that contain
repeated text, not saying its a bad or good thing though. WDYT?

I can think of a few use cases:

1. Capturing `@LINE`, as you suggest.
2. Making a single directive's complex pattern more readable by extracting and naming components.
3. Avoiding the maintenance burden of repeating the same string, numeric, or pattern expression multiple times within a single directive or across multiple directives.

`-D` can sometimes help with #2 and #3.  However, it still has to be repeated across multiple RUN lines in some cases.  Moreover, it's sometimes better to define variables near their uses, but the RUN lines can be far away.

As an example of #2 and #3, I've occasionally written subtle patterns, such as `{{$[[:space:]]}}` for newline, that I wanted to use in multiple places.  It's not immediately obvious what that pattern does, but `[[NL]]` would be much clearer and could be documented once at its definition.

It just occurred to me that the new directive might require a "CHECK-DEF-CONST" form that is processed before all other directives.  Otherwise, forward-references wouldn't be possible.  You might want forward-references for #1, for example.

Joel
 

~Nathan

On Fri, 2020-07-17 at 14:52 -0400, Joel E. Denny via cfe-dev wrote:
> Hi Nathan,
>
> On Fri, Jul 17, 2020 at 12:23 PM Nathan James via cfe-dev <
> [hidden email]> wrote:
> > Hello,
> >
> > I was wondering about extending FileCheck to enable creating line
> > anchors. These are numeric variables that hold the value of the
> > line
> > number that where they were defined.
>
> I think something like this could be useful.  However, I think it
> would be more useful to have a general directive for defining
> FileCheck variables inline without trying to capture from input.  For
> example:
>
> ```
> #define BAD_FUNCTION() badFunction() // CHECK-DEFINE:
> [[#BAD_FUNC:@LINE]]
> // Further down in the file
>   BAD_FUNCTION();
> CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
> CHECK-NOTES :[[#BAD_FUNC]]:3: note: expanded from macro
> 'BAD_FUNCTION'
> ```
>
> The exact syntax is debatable.
>
> I think this form is more useful because it can also define strings
> or numerics (or maybe even patterns) to be reused in multiple
> FileCheck directives across multiple FileCheck calls from different
> RUN lines.  Currently, you either have to capture such a variable
> from the input or specify it with -D on every FileCheck call that
> needs it.
>
> James Henderson: Weren't you working on something like this?
>
> Thanks.
>
> Joel

> > The motivation for this comes from test cases using clang-based
> > diagnostics which often include notes attached to source locations
> > in
> > different parts of the file. In order to test for the correct
> > location
> > of the note, the line number has to be written explicitly or as an,
> > often large, offset to the current line. This harms both
> > readability
> > and maintainability. Using this new system one could append a line
> > of
> > interest with an anchor-comment and refer back to it inside
> > FileCheck.
> >
> > I have created a basic patch that implements this here
> > https://reviews.llvm.org/D84037 but it definitely needs a few looks
> > over by people who are more clued up on the internal of FileCheck.
> >
> > The current syntax, based off this patch, is as follows:
> >   - Added a command line option called `anchor-prefix` which is a
> > comma-seperated list of prefixes to be used when declaring anchors.
> > This is defaulted to `LINE-ANCHOR`
> >   - To declare a anchor in the test file use
> >     `LINE-ANCHOR: ANCHOR_NAME`
> >     note: If you specify a different anchor-prefix using the
> > command
> > line, use that name instead of `LINE-ANCHOR`
> >     ANCHOR_NAME Follows the rules all other variable names aside
> > from
> > the fact it can't start with '$'.
> >   - When referring to an anchor in a check use the same numeric
> > variable syntax that FileCheck already supports:
> >     `CHECK: [[#ANCHOR_NAME]][[#ANCHOR_NAME+1]]`
> >
> > Here is a brief (contrived) example of the usage of this:
> > ```
> > #define BAD_FUNCTION() badFunction() // LINE-ANCHOR: BAD_FUNC
> > // Further down in the file
> >   BAD_FUNCTION();
> > CHECK-NOTES: [[@LINE-1]]:3: warning: called a bad function
> > CHECK-NOTES :[[#BAD-FUNC]]:3: note: expanded from macro
> > 'BAD_FUNCTION'
> > ```
> >
> > Regards,
> > Nathan James
> >
> >
> > _______________________________________________
> > cfe-dev mailing list
> > [hidden email]
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev