Does SourceLocation need more documentation?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Does SourceLocation need more documentation?

Eric Fiselier via cfe-dev
I never felt that I completely understood the code I wrote/reviewed in tools dealing with SourceLocation. And clangd has had several bugs that boil down to wrong locations in the presence of macros.

I couldn't find a good guide for how to think about them, so I spent a while reading the source code and wrote a tool to visualize them, which helped me a lot:
(black edges are expansion, yellow edges are source)

Did I miss some documentation of this concept?
Would it be worth writing up an article illustrated with some examples, to add to llvm.org? Maybe this is niche or well-understood, interested in what others think.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Does SourceLocation need more documentation?

Eric Fiselier via cfe-dev
On Mon, Apr 9, 2018 at 8:06 PM Sam McCall via cfe-dev <[hidden email]> wrote:
I never felt that I completely understood the code I wrote/reviewed in tools dealing with SourceLocation. And clangd has had several bugs that boil down to wrong locations in the presence of macros.

I couldn't find a good guide for how to think about them, so I spent a while reading the source code and wrote a tool to visualize them, which helped me a lot:
(black edges are expansion, yellow edges are source)

Cool. I once had a CL to help print SLs in a more readable way, but it bitrotted under Richard's unforgiving eyes :)
This is way cooler though.
 
Did I miss some documentation of this concept?

Nope. I touched a bit on this in my AST talk, but it's really missing.
 
Would it be worth writing up an article illustrated with some examples, to add to llvm.org? Maybe this is niche or well-understood, interested in what others think.

A talk at an LLVM conf would also be cool :D

Cheers,
/Manuel
 

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Does SourceLocation need more documentation?

Eric Fiselier via cfe-dev
In reply to this post by Eric Fiselier via cfe-dev
On Mon, Apr 9, 2018 at 2:06 PM, Sam McCall via cfe-dev <[hidden email]> wrote:
I never felt that I completely understood the code I wrote/reviewed in tools dealing with SourceLocation. And clangd has had several bugs that boil down to wrong locations in the presence of macros.

I couldn't find a good guide for how to think about them, so I spent a while reading the source code and wrote a tool to visualize them, which helped me a lot:

More documentation is always good, but I for one have no idea what's going on in that image, and I like to think that I mostly understand SourceLocation :-)
 

(black edges are expansion, yellow edges are source)

Did I miss some documentation of this concept?
Would it be worth writing up an article illustrated with some examples, to add to llvm.org? Maybe this is niche or well-understood, interested in what others think.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Does SourceLocation need more documentation?

Eric Fiselier via cfe-dev


On Mon, Apr 9, 2018, 20:15 Nico Weber <[hidden email]> wrote:
On Mon, Apr 9, 2018 at 2:06 PM, Sam McCall via cfe-dev <[hidden email]> wrote:
I never felt that I completely understood the code I wrote/reviewed in tools dealing with SourceLocation. And clangd has had several bugs that boil down to wrong locations in the presence of macros.

I couldn't find a good guide for how to think about them, so I spent a while reading the source code and wrote a tool to visualize them, which helped me a lot:

More documentation is always good, but I for one have no idea what's going on in that image, and I like to think that I mostly understand SourceLocation :-)
That's very useful feedback!
It definitely needs some work, and some text.
But maybe it's just the wrong model.

The idea is that SourceLocation is just a (FileID, offset) pair, and everyone knows how files and offsets work. So the difficult part is understanding the real and "virtual" files (SLocEntry) and how they relate to each other.

The nodes in the diagram are the SLocEntrys in an example program (the main.cpp file).
The text contents are shown, this is what a (FileID, offset) pair indexes into. (For expansions, this is the spelling).

The black edges are (immediate) expansion location. This forms the logical macro expression tree that everyone is familiar with. Partial traversal can be useful/meaningful.
Labels show what expression in the parent is being substituted.

Yellow edges show (immediate) spelling location. As far as I can tell, these are mainly useful a) to get the content of expansion SLocEntrys which don't have buffers and b) to produce diagnostics that reference actual source code. But partial traversal here doesn't seem to be useful.

 

(black edges are expansion, yellow edges are source)

Did I miss some documentation of this concept?
Would it be worth writing up an article illustrated with some examples, to add to llvm.org? Maybe this is niche or well-understood, interested in what others think.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Does SourceLocation need more documentation?

Eric Fiselier via cfe-dev
In reply to this post by Eric Fiselier via cfe-dev
Hi Sam,

On Mon, Apr 9, 2018 at 8:06 PM, Sam McCall via cfe-dev
<[hidden email]> wrote:

> I never felt that I completely understood the code I wrote/reviewed in tools
> dealing with SourceLocation. And clangd has had several bugs that boil down
> to wrong locations in the presence of macros.
>
> I couldn't find a good guide for how to think about them, so I spent a while
> reading the source code and wrote a tool to visualize them, which helped me
> a lot:
>   https://imgur.com/f3cXp7E
> (black edges are expansion, yellow edges are source)
>
> Did I miss some documentation of this concept?
> Would it be worth writing up an article illustrated with some examples, to
> add to llvm.org? Maybe this is niche or well-understood, interested in what
> others think.

Just to answer the question from your subject line: yes, that would be
very useful!

There are two challenges to locations, IMHO;

1) Details of the design: what are the concepts and mechanisms?
2) Use cases: how can we track macro expansion? Concatenation? What's
a bare-minimum handling of file vs macro locations? How can we be more
ambitious, e.g. attribute different parts of a macro to different
files?

I sometimes think it should be possible to build a queryable AST-like
structure for preprocessor symbols on top of the current location
design, but whenever I try actualize it, I get lost in the maze :)

So anything that helps clarify the model would be lovely.

- Kim
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev