Delay Clang CodeGen until AST is fully populated

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Delay Clang CodeGen until AST is fully populated

Nathan Ridge via cfe-dev
Hello,

We are having issues with modifying the Clang AST while it is being built, and we suspect they are due to it being code generated before it is fully populated, and we were wondering if it was possible to require the Sema to be fully finished before doing any CodeGen?

An example would be when we process custom pragma:

int my_global;

int my_function() {
    my_global = 1;
}

int main() {
#pragma FOO SOMETHING var=my_global
    my_function();
    return 0;
}

Here the way I process the pragma is that it attach to the AST node for my_global, an attribute node. So that once I code-generate that global variable I can "do something" based on that attribute. Unfortunately, today Clang might codegen that global as soon as it deem it necessary (typically, when code-generating my_function as it uses it). And the pragma has not yet been semantically parsed yet.

If I were to use -ast-print and dump the result into a file, then the attribute would be there, and if I parse that file again I would get the correct codegen. But I assume it is unsafe to rely on being able to parse -ast-print and/or it's AST to be the same.

Any input on that issue?

--
Alexandre Isoard

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Delay Clang CodeGen until AST is fully populated

Nathan Ridge via cfe-dev
It seems that there's the same issue with following code:

**************************************************************
template<typename T>
class complex{
public:
T x;
T y;
void add( complex b) {
x = x + b.x;
y = y + b.y;
}
};

typedef complex<float> AA;

float sub(AA b[10][5]) {
return 0;
}

float dut( AA a[10][10][5] ) {

return sub(a[3]);;
}
**************************************************************  

and clang will generate following code:

**************************************************************  
%class.complex = type { float, float }

define dso_local float @_Z3subPA5_7complexIfE([5 x i8]*) #0 {
%2 = alloca [5 x i8]*, align 8
store [5 x i8]* %0, [5 x i8]** %2, align 8
ret float 0.000000e+00
}

define dso_local float @_Z3dutPA10_A5_7complexIfE([10 x [5 x %class.complex]]*) #0 {
%2 = alloca [10 x [5 x %class.complex]]*, align 8
%3 = alloca [5 x %class.complex]*, align 8
store [10 x [5 x %class.complex]]* %0, [10 x [5 x %class.complex]]** %2, align 8
%4 = load [10 x [5 x %class.complex]]*, [10 x [5 x %class.complex]]** %2, align 8
%5 = getelementptr inbounds [10 x [5 x %class.complex]], [10 x [5 x %class.complex]]* %4, i64 3
%6 = getelementptr inbounds [10 x [5 x %class.complex]], [10 x [5 x %class.complex]]* %5, i32 0, i32 0
store [5 x %class.complex]* %6, [5 x %class.complex]** %3, align 8
%7 = load [5 x %class.complex]*, [5 x %class.complex]** %3, align 8
%8 = bitcast [5 x %class.complex]* %7 to [5 x i8]*
%9 = call float @_Z3subPA5_7complexIfE([5 x i8]* %8)
ret float %9
}
**************************************************************  
If we use '-ast-print', and compile the '-ast-print' output again, then it will generate the complete type, like:

************************************************************** 
%class.complex = type { float, float }

; Function Attrs: noinline nounwind optnone uwtable
define float @_Z3subPA5_7complexIfE([5 x %class.complex]* %b) #0 {
entry:
  %b.addr = alloca [5 x %class.complex]*, align 8
  store [5 x %class.complex]* %b, [5 x %class.complex]** %b.addr, align 8
  ret float 0.000000e+00
}

; Function Attrs: noinline nounwind optnone uwtable
define float @_Z3dutPA10_A5_7complexIfE([10 x [5 x %class.complex]]* %a) #0 {
entry:
  %a.addr = alloca [10 x [5 x %class.complex]]*, align 8
  store [10 x [5 x %class.complex]]* %a, [10 x [5 x %class.complex]]** %a.addr, align 8
  %0 = load [10 x [5 x %class.complex]]*, [10 x [5 x %class.complex]]** %a.addr, align 8
  %arrayidx = getelementptr inbounds [10 x [5 x %class.complex]], [10 x [5 x %class.complex]]* %0, i64 3
  %arraydecay = getelementptr inbounds [10 x [5 x %class.complex]], [10 x [5 x %class.complex]]* %arrayidx, i32 0, i32 0
  %call = call float @_Z3subPA5_7complexIfE([5 x %class.complex]* %arraydecay)
  ret float %call
**************************************************************  

Are we confident it does not introduce issues to do the CodeGen early?
If we had an option to do the codeGen after complete AST build, we could at least detect this kind of difference at testing?

Thanks,
Fangqing

Alexandre Isoard via cfe-dev <[hidden email]> 于2019年6月25日周二 下午7:19写道:
Hello,

We are having issues with modifying the Clang AST while it is being built, and we suspect they are due to it being code generated before it is fully populated, and we were wondering if it was possible to require the Sema to be fully finished before doing any CodeGen?

An example would be when we process custom pragma:

int my_global;

int my_function() {
    my_global = 1;
}

int main() {
#pragma FOO SOMETHING var=my_global
    my_function();
    return 0;
}

Here the way I process the pragma is that it attach to the AST node for my_global, an attribute node. So that once I code-generate that global variable I can "do something" based on that attribute. Unfortunately, today Clang might codegen that global as soon as it deem it necessary (typically, when code-generating my_function as it uses it). And the pragma has not yet been semantically parsed yet.

If I were to use -ast-print and dump the result into a file, then the attribute would be there, and if I parse that file again I would get the correct codegen. But I assume it is unsafe to rely on being able to parse -ast-print and/or it's AST to be the same.

Any input on that issue?

--
Alexandre Isoard
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Delay Clang CodeGen until AST is fully populated

Nathan Ridge via cfe-dev
We are really struggling with separating Sema and CodeGen.

Here are the few dead-end we hit:

1) We tried to use -emit-ast to get a PCH-like AST, but it seems it does not do template specialization.
That is, when we feed it again into clang, some of the Sema is still run after CodeGen.
We stopped there, because we are not sure this can help us.

2) We tried -traditional-cpp and -no-integrated-cpp which is supposed to forbid clang driver to merge the different stages (it didn't help). I do not know if this was effective, but we didn't see any difference. So maybe it is the same issue as the previous one? We also stopped there, because we use clang -cc1 anyway.

3) We started looking into ASTConsumer(s) and we are thinking if it is possible to write an ASTConsumer that "forward" the AST to the CodeGen and only implement the HandleTranslationUnit so that it ensure it receive the entire AST before forwarding it. But I guess we need to explore it then run manually each of the CodeGen method? Will it have the same issue about template instantiation?

We would really appreciate if anybody could shed some light, we don't know that part of the infrastructure.
Are we even going into the right direction? Is there already an option that does that?

--
Alexandre Isoard

_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Delay Clang CodeGen until AST is fully populated

Nathan Ridge via cfe-dev
Hi  ,

I Want to unsubscribe emails.
Please do need full.

Regards,
Srinivas

-----"cfe-dev" <[hidden email]> wrote: -----
To: Fangqing Du <[hidden email]>
From: Alexandre Isoard via cfe-dev
Sent by: "cfe-dev"
Date: 07/02/2019 04:23AM
Cc: Clang Dev <[hidden email]>
Subject: Re: [cfe-dev] Delay Clang CodeGen until AST is fully populated

"External email. Open with Caution"
We are really struggling with separating Sema and CodeGen.

Here are the few dead-end we hit:

1) We tried to use -emit-ast to get a PCH-like AST, but it seems it does not do template specialization.
That is, when we feed it again into clang, some of the Sema is still run after CodeGen.
We stopped there, because we are not sure this can help us.

2) We tried -traditional-cpp and -no-integrated-cpp which is supposed to forbid clang driver to merge the different stages (it didn't help). I do not know if this was effective, but we didn't see any difference. So maybe it is the same issue as the previous one? We also stopped there, because we use clang -cc1 anyway.

3) We started looking into ASTConsumer(s) and we are thinking if it is possible to write an ASTConsumer that "forward" the AST to the CodeGen and only implement the HandleTranslationUnit so that it ensure it receive the entire AST before forwarding it. But I guess we need to explore it then run manually each of the CodeGen method? Will it have the same issue about template instantiation?

We would really appreciate if anybody could shed some light, we don't know that part of the infrastructure.
Are we even going into the right direction? Is there already an option that does that?

--
Alexandre Isoard
_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you


_______________________________________________
cfe-dev mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev