moving the clang-omp merge along

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

moving the clang-omp merge along

Jack Howarth
     Andrey Bokhanko expressed interest in getting the clang-omp merge done in time for the 3.5 release but wants guidance on the process. I suggested starting with the creation a new clang-omp branch upstream rebased on clang trunk  for generation of merge patch. Unfortunately merging the  current changes from the clang-omp (based on clang 3.4) to a clang-omp (based on clang trunk)  looks very difficult as selective patches have been committed into clang trunk from clang-omp and don't appear to have been kept synchronized with the current changes from upstream. Does anyone know if these new files from previous commits out of clang-omp contain any local changes which haven't been back ported to clang-omp? It would seem that postponing this merge will just make the process harder as time goes on if selective merges from clang-omp into clang trunk continue in the interim. Hopefully the folks who did the original selective commits would help detangle this mess.
           Jack

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Philip Reames
I would strongly recommend that you get your current branch in sync with clang-TOT as a first step.  Once this done, you should separate individual patches and submit them for review.  Based on previous history, the community is unlikely to accept a single massive change set. 

p.s. The tone of your last sentence is less than ideal.  These are the folks actually working on getting the work you are interested merged into upstream.  You should thank them, not critique them.  (I'm not one of them, btw.)

Philip

On 05/28/2014 03:19 PM, Jack Howarth wrote:
     Andrey Bokhanko expressed interest in getting the clang-omp merge done in time for the 3.5 release but wants guidance on the process. I suggested starting with the creation a new clang-omp branch upstream rebased on clang trunk  for generation of merge patch. Unfortunately merging the  current changes from the clang-omp (based on clang 3.4) to a clang-omp (based on clang trunk)  looks very difficult as selective patches have been committed into clang trunk from clang-omp and don't appear to have been kept synchronized with the current changes from upstream. Does anyone know if these new files from previous commits out of clang-omp contain any local changes which haven't been back ported to clang-omp? It would seem that postponing this merge will just make the process harder as time goes on if selective merges from clang-omp into clang trunk continue in the interim. Hopefully the folks who did the original selective commits would help detangle this mess.
           Jack


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Chandler Carruth
In reply to this post by Jack Howarth

On Wed, May 28, 2014 at 3:19 PM, Jack Howarth <[hidden email]> wrote:
Andrey Bokhanko expressed interest in getting the clang-omp merge done in time for the 3.5 release but wants guidance on the process.

I think a bunch of the confusion comes from phrasing this as a merge. It isn't.

Andrey and others are working hard to contribute the functionality to Clang, but the github project you're referencing is not a branch of Clang, and the code there which isn't also in Clang's trunk hasn't been contributed to Clang and isn't even covered by the LLVM project... This means that it hasn't been code reviewed, etc.

The Intel folks are contributing small incremental patches to Clang and they are being reviewed in the normal process. This is IMO the best way to add new features to Clang and has been standard practice for some time. I don't think there is some other process which should be followed instead because the release date is getting close, and I also don't think there is any other process that we *could* follow given that the branch isn't part of the Clang project... If the contributions arrive, get reviewed, and are committed in time for the release, awesome. If not, there will be another release after that. =]

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Alp Toker

On 29/05/2014 03:42, Chandler Carruth wrote:

>
> On Wed, May 28, 2014 at 3:19 PM, Jack Howarth
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Andrey Bokhanko expressed interest in getting the clang-omp
>     merge done in time for the 3.5 release but wants guidance on the
>     process.
>
>
> I think a bunch of the confusion comes from phrasing this as a merge.
> It isn't.
>
> Andrey and others are working hard to contribute the functionality to
> Clang, but the github project you're referencing is not a branch of
> Clang, and the code there which isn't also in Clang's trunk hasn't
> been contributed to Clang and isn't even covered by the LLVM
> project... This means that it hasn't been code reviewed, etc.
>
> The Intel folks are contributing small incremental patches to Clang
> and they are being reviewed in the normal process. This is IMO the
> best way to add new features to Clang and has been standard practice
> for some time. I don't think there is some other process which should
> be followed instead because the release date is getting close,

I second this. If you look at the OMP reviews so far they've uncovered
significant improvements pre-commit that wouldn't have been pointed out
after the fact.

> and I also don't think there is any other process that we *could*
> follow given that the branch isn't part of the Clang project... If the
> contributions arrive, get reviewed, and are committed in time for the
> release, awesome. If not, there will be another release after that. =]

If anything it would have been nice to have discussed the overall design
of the OMP frontend earlier in development. The fact the patches are
landing broadly as written in order to keep it moving is already
something of a concession :-)

Alp.


>
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

--
http://www.nuanti.com
the browser experts

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Jack Howarth
In reply to this post by Philip Reames
Philip,
    From observing many merges in FSF gcc over the years, it is crazy to take a new branch, selectively pull in small sections and then take long breaks where the two start to rapidly fork. If a branch is to be merged, the process should at least be scheduled such that the process will take place over a known period of time so attempts can be made to keep the two in sync or at least keep track of where the two have begun to diverge. At the moment, there are quite a few files introduced from clang-omp that are no longer in sync and the svn web browser access doesn't seem to allow you to easily view the commit history on individual files to see if they have been changed since the original commits.
             Jack


On Wed, May 28, 2014 at 8:25 PM, Philip Reames <[hidden email]> wrote:
I would strongly recommend that you get your current branch in sync with clang-TOT as a first step.  Once this done, you should separate individual patches and submit them for review.  Based on previous history, the community is unlikely to accept a single massive change set. 

p.s. The tone of your last sentence is less than ideal.  These are the folks actually working on getting the work you are interested merged into upstream.  You should thank them, not critique them.  (I'm not one of them, btw.)

Philip


On 05/28/2014 03:19 PM, Jack Howarth wrote:
     Andrey Bokhanko expressed interest in getting the clang-omp merge done in time for the 3.5 release but wants guidance on the process. I suggested starting with the creation a new clang-omp branch upstream rebased on clang trunk  for generation of merge patch. Unfortunately merging the  current changes from the clang-omp (based on clang 3.4) to a clang-omp (based on clang trunk)  looks very difficult as selective patches have been committed into clang trunk from clang-omp and don't appear to have been kept synchronized with the current changes from upstream. Does anyone know if these new files from previous commits out of clang-omp contain any local changes which haven't been back ported to clang-omp? It would seem that postponing this merge will just make the process harder as time goes on if selective merges from clang-omp into clang trunk continue in the interim. Hopefully the folks who did the original selective commits would help detangle this mess.
           Jack


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Jack Howarth
Philip,
     Also, it would be quite unfortunate if the previous partial merge of clang-omp, which I assume was done to make the -fopenmp/-lgomp support in clang trunk more usable, in the end became a major impediment towards getting the clang-omp changes into clang trunk in time for 3.5's release.
             Jack


On Wed, May 28, 2014 at 9:21 PM, Jack Howarth <[hidden email]> wrote:
Philip,
    From observing many merges in FSF gcc over the years, it is crazy to take a new branch, selectively pull in small sections and then take long breaks where the two start to rapidly fork. If a branch is to be merged, the process should at least be scheduled such that the process will take place over a known period of time so attempts can be made to keep the two in sync or at least keep track of where the two have begun to diverge. At the moment, there are quite a few files introduced from clang-omp that are no longer in sync and the svn web browser access doesn't seem to allow you to easily view the commit history on individual files to see if they have been changed since the original commits.
             Jack


On Wed, May 28, 2014 at 8:25 PM, Philip Reames <[hidden email]> wrote:
I would strongly recommend that you get your current branch in sync with clang-TOT as a first step.  Once this done, you should separate individual patches and submit them for review.  Based on previous history, the community is unlikely to accept a single massive change set. 

p.s. The tone of your last sentence is less than ideal.  These are the folks actually working on getting the work you are interested merged into upstream.  You should thank them, not critique them.  (I'm not one of them, btw.)

Philip


On 05/28/2014 03:19 PM, Jack Howarth wrote:
     Andrey Bokhanko expressed interest in getting the clang-omp merge done in time for the 3.5 release but wants guidance on the process. I suggested starting with the creation a new clang-omp branch upstream rebased on clang trunk  for generation of merge patch. Unfortunately merging the  current changes from the clang-omp (based on clang 3.4) to a clang-omp (based on clang trunk)  looks very difficult as selective patches have been committed into clang trunk from clang-omp and don't appear to have been kept synchronized with the current changes from upstream. Does anyone know if these new files from previous commits out of clang-omp contain any local changes which haven't been back ported to clang-omp? It would seem that postponing this merge will just make the process harder as time goes on if selective merges from clang-omp into clang trunk continue in the interim. Hopefully the folks who did the original selective commits would help detangle this mess.
           Jack


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Alp Toker
In reply to this post by Jack Howarth

On 29/05/2014 04:21, Jack Howarth wrote:
> Philip,
>     From observing many merges in FSF gcc over the years, it is crazy
> to take a new branch, selectively pull in small sections

"Crazy"? Selectively pulling in small chunks is the only realistic way
to deal with the task.

> and then take long breaks where the two start to rapidly fork. If a
> branch is to be merged, the process should at least be scheduled such
> that the process will take place over a known period of time so
> attempts can be made to keep the two in sync or at least keep track of
> where the two have begun to diverge. At the moment, there are quite a
> few files introduced from clang-omp that are no longer in sync and the
> svn web browser access doesn't seem to allow you to easily view the
> commit history on individual files

Jack, are you seriously talking about taking on a merge of this scale
using the *SVN web interface*.. and then complaining about that to Philip?

There are tools for that: Consider using clang's own
format/tooling/refactoring together with git to track upstream changes
and automate the sync work for your out-of-tree branch.

It's no wonder you're getting stuck if you were trying to coordinate the
effort with WebSVN, after all the one thing you can be sure of is that
upstream code won't stay still in the LLVM project.

Alp.



> to see if they have been changed since the original commits.
>              Jack
>
>
> On Wed, May 28, 2014 at 8:25 PM, Philip Reames
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     I would strongly recommend that you get your current branch in
>     sync with clang-TOT as a first step.  Once this done, you should
>     separate individual patches and submit them for review. Based on
>     previous history, the community is unlikely to accept a single
>     massive change set.
>
>     p.s. The tone of your last sentence is less than ideal. These are
>     the folks actually working on getting the work you are interested
>     merged into upstream.  You should thank them, not critique them.
>     (I'm not one of them, btw.)
>
>     Philip
>
>
>     On 05/28/2014 03:19 PM, Jack Howarth wrote:
>>     Andrey Bokhanko expressed interest in getting the clang-omp
>>     merge done in time for the 3.5 release but wants guidance on the
>>     process. I suggested starting with the creation a new clang-omp
>>     branch upstream rebased on clang trunk  for generation of merge
>>     patch. Unfortunately merging the  current changes from the
>>     clang-omp (based on clang 3.4) to a clang-omp (based on clang
>>     trunk)  looks very difficult as selective patches have been
>>     committed into clang trunk from clang-omp and don't appear to
>>     have been kept synchronized with the current changes from
>>     upstream. Does anyone know if these new files from previous
>>     commits out of clang-omp contain any local changes which haven't
>>     been back ported to clang-omp? It would seem that postponing this
>>     merge will just make the process harder as time goes on if
>>     selective merges from clang-omp into clang trunk continue in the
>>     interim. Hopefully the folks who did the original selective
>>     commits would help detangle this mess.
>>                Jack
>>
>>
>>     _______________________________________________
>>     cfe-dev mailing list
>>     [hidden email]  <mailto:[hidden email]>
>>     http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
>     _______________________________________________
>     cfe-dev mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

--
http://www.nuanti.com
the browser experts

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Jack Howarth
As far as I can tell the commit history of the OPENMP support into clang trunk is represented
by the following commits…


This assumes that everyone has been labeling the comments in their commits with OpenMP.
There was at least one instance where this was misspelled.



On Wed, May 28, 2014 at 11:39 PM, Alp Toker <[hidden email]> wrote:

On 29/05/2014 04:21, Jack Howarth wrote:
Philip,
    From observing many merges in FSF gcc over the years, it is crazy to take a new branch, selectively pull in small sections

"Crazy"? Selectively pulling in small chunks is the only realistic way to deal with the task.


and then take long breaks where the two start to rapidly fork. If a branch is to be merged, the process should at least be scheduled such that the process will take place over a known period of time so attempts can be made to keep the two in sync or at least keep track of where the two have begun to diverge. At the moment, there are quite a few files introduced from clang-omp that are no longer in sync and the svn web browser access doesn't seem to allow you to easily view the commit history on individual files

Jack, are you seriously talking about taking on a merge of this scale using the *SVN web interface*.. and then complaining about that to Philip?

There are tools for that: Consider using clang's own format/tooling/refactoring together with git to track upstream changes and automate the sync work for your out-of-tree branch.

It's no wonder you're getting stuck if you were trying to coordinate the effort with WebSVN, after all the one thing you can be sure of is that upstream code won't stay still in the LLVM project.

Alp.



to see if they have been changed since the original commits.
             Jack


On Wed, May 28, 2014 at 8:25 PM, Philip Reames <[hidden email] <mailto:[hidden email]>> wrote:

    I would strongly recommend that you get your current branch in
    sync with clang-TOT as a first step.  Once this done, you should
    separate individual patches and submit them for review. Based on
    previous history, the community is unlikely to accept a single
    massive change set.

    p.s. The tone of your last sentence is less than ideal. These are
    the folks actually working on getting the work you are interested
    merged into upstream.  You should thank them, not critique them.     (I'm not one of them, btw.)

    Philip


    On 05/28/2014 03:19 PM, Jack Howarth wrote:
    Andrey Bokhanko expressed interest in getting the clang-omp
    merge done in time for the 3.5 release but wants guidance on the
    process. I suggested starting with the creation a new clang-omp
    branch upstream rebased on clang trunk  for generation of merge
    patch. Unfortunately merging the  current changes from the
    clang-omp (based on clang 3.4) to a clang-omp (based on clang
    trunk)  looks very difficult as selective patches have been
    committed into clang trunk from clang-omp and don't appear to
    have been kept synchronized with the current changes from
    upstream. Does anyone know if these new files from previous
    commits out of clang-omp contain any local changes which haven't
    been back ported to clang-omp? It would seem that postponing this
    merge will just make the process harder as time goes on if
    selective merges from clang-omp into clang trunk continue in the
    interim. Hopefully the folks who did the original selective
    commits would help detangle this mess.
               Jack


    _______________________________________________
    cfe-dev mailing list
    [hidden email]  <mailto:[hidden email]>
    http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev


    _______________________________________________
    cfe-dev mailing list
    [hidden email] <mailto:[hidden email]>

    http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

--
http://www.nuanti.com
the browser experts



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Andrey Bokhanko
In reply to this post by Jack Howarth
All,

To clarify, what I meant is getting OMP runtime library (not clang-omp branch!) as a part of 3.5 release. I specifically referred to this thread: http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025 started by Jack. I thought he means libiomp when saying "openmp support" -- apparently, I was mistaken. Hence the confusion.

After so many talks with Chandler, I know very well that upstreaming full OpenMP support is a long way to go. :-)

Also, the whole desire to put the library into 3.5 release is a responce to the criticism expressed by Chandler in this message: http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html.

While we are on the topic, let me update on some other things happening in responce to other issues highlighted by Chandler:

- This library is not being developed as an active part of the LLVM
community, even if it is checked into SVN as part of the LLVM project and
under its license. See r197914 where there is a code drop of many months
worth of development with *no* change log, attribution, information, or
other participation in any part of the community.

This is changing, and many developers joining the whole OpenMP in clang support effort. I can say that Michael Wong and many of his colleagues from IBM are involved; Eric Stotzer and his colleagues from TI are involved; Barbara Chapman and her group from UofHouston is involved; Guansong Zhang from AMD is involved. Obviously, Hal Finkel, Chris Bergstrom, Steve Noonan and many others continue to be actively involved as well.

Take a look at the recent activity in openmp-dev mailing list. More to come.


- There has been essentially *zero* discussion with the rest of the clang
or llvm community about this library. There are separate mailing lists
which have nearly no traffic since the code drop.

Take a look at this month's messages: http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html. In general, as more people get interested, more traffic became generated. It's a chicken and egg problem...

- There has been no effort to make this library even work properly with
Clang as a host compiler. See the copious notes that only Clang 3.3 is
supported, and that not full featured.

It is buildable with clang now. Moreover, regular buildbots, with both gcc and clang, are running:

http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian

- The build system is totally disjoint from LLVM's, in fact it is an
entirely custom Perl build system that is unlike anything in use by the
LLVM project.

We started to work on improving build system. Stay tuned.

- There are *zero tests* in the open source repository!!!! This is even
called out in the original submission and on the primary website. We simply
*cannot* ship and link against a runtime library which has no tests!

University of Houston contributed OpenUH test suite: http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html. Sunita Chandrasekaran from the University works on integrating this suite into LLVM test system.

BTW, any advice with how to approach this would be *much* appreciated!

- No part of this library has gone through an LLVM release process either,
not even as a "new" or "beta" project.

Aha! So, you support inclusion of openmp (library, not compiler) in 3.5 as well? :-)

Andrey



On Thu, May 29, 2014 at 2:19 AM, Jack Howarth <[hidden email]> wrote:
     Andrey Bokhanko expressed interest in getting the clang-omp merge done in time for the 3.5 release but wants guidance on the process. I suggested starting with the creation a new clang-omp branch upstream rebased on clang trunk  for generation of merge patch. Unfortunately merging the  current changes from the clang-omp (based on clang 3.4) to a clang-omp (based on clang trunk)  looks very difficult as selective patches have been committed into clang trunk from clang-omp and don't appear to have been kept synchronized with the current changes from upstream. Does anyone know if these new files from previous commits out of clang-omp contain any local changes which haven't been back ported to clang-omp? It would seem that postponing this merge will just make the process harder as time goes on if selective merges from clang-omp into clang trunk continue in the interim. Hopefully the folks who did the original selective commits would help detangle this mess.
           Jack

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Chandler Carruth

On Thu, May 29, 2014 at 6:46 AM, Andrey Bokhanko <[hidden email]> wrote:
While we are on the topic, let me update on some other things happening in responce to other issues highlighted by Chandler:

I just wanted to say, I am very happy to see progress starting here. =] All of my comments below are meant to help the effort move faster and deal better with the LLVM project.
 
- This library is not being developed as an active part of the LLVM
community, even if it is checked into SVN as part of the LLVM project and
under its license. See r197914 where there is a code drop of many months
worth of development with *no* change log, attribution, information, or
other participation in any part of the community.

This is changing, and many developers joining the whole OpenMP in clang support effort. I can say that Michael Wong and many of his colleagues from IBM are involved; Eric Stotzer and his colleagues from TI are involved; Barbara Chapman and her group from UofHouston is involved; Guansong Zhang from AMD is involved. Obviously, Hal Finkel, Chris Bergstrom, Steve Noonan and many others continue to be actively involved as well.

Take a look at the recent activity in openmp-dev mailing list. More to come.

Pretty happy about this, and looking forward to more. Some things that would be helpful (IMO, others may disagree) to start looking into how/when to adopt or integrate:

- Start more closely following the LLVM developer policy around code review and small, incremental patches and development.
  - Maybe use code review tools like Phabricator? Dunno what would be needed to make this work well, probably some stuff...
- Check that the codebase compiles with the same host toolchain baseline as the rest of the LLVM project[1]
- Maybe work out a plan toward generally conforming with the coding standards[2] where applicable?
 

It is buildable with clang now. Moreover, regular buildbots, with both gcc and clang, are running:

http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian

Just want to say, this in particular is fantastic.


- The build system is totally disjoint from LLVM's, in fact it is an
entirely custom Perl build system that is unlike anything in use by the
LLVM project.

We started to work on improving build system. Stay tuned.

Very cool. I would look closely at the compiler-rt and libc++ CMake builds. Hopefully useful.


- There are *zero tests* in the open source repository!!!! This is even
called out in the original submission and on the primary website. We simply
*cannot* ship and link against a runtime library which has no tests!

University of Houston contributed OpenUH test suite: http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html. Sunita Chandrasekaran from the University works on integrating this suite into LLVM test system.

BTW, any advice with how to approach this would be *much* appreciated!

I think the right way to go is something along the lines of the ASan (in compiler-rt) lit tests, but maybe others have a better idea. I agree that testing here is hard.


- No part of this library has gone through an LLVM release process either,
not even as a "new" or "beta" project.

Aha! So, you support inclusion of openmp (library, not compiler) in 3.5 as well? :-)

Hehe, I see what you did there.

I do genuinely support it going through the release process, but I think that there are two things that need to be pretty solid first:

1) the build system work probably needs to be pretty solid. i'd be happy with support on par with compiler-rt or libc++ in CMake...
2) the test suite needs to be at least reasonably well integrated so release testers actually hit it and exercise the code
3) we need a good switch in Clang to use iomp (I know you or someone on the patch thread worked on this, did it land? if so, done!)

Maybe others have other thoughts, but I think those three are my key ones.

Anyways, thanks for the clarification and all the hard work on this stuff.

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Alp Toker
In reply to this post by Andrey Bokhanko
Thanks for the summary Andrey.

The underlying problem For the OpenMP *runtime* is really lack of
visibility and sidelining on the openmp-dev list.

It's absolutely critical to close down the low-volume openmp-dev list
and fold the subscribers into one of the more active mailing lists,
either llvm-dev or cfe-dev.

Until that's done the patches won't get review from the mainstream LLVM
developer community, build system experts won't join in etc.

Alp.



On 29/05/2014 16:46, Andrey Bokhanko wrote:

> All,
>
> To clarify, what I meant is getting OMP runtime library (not clang-omp
> branch!) as a part of 3.5 release. I specifically referred to this
> thread:
> http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025 
> started by Jack. I thought he means libiomp when saying "openmp
> support" -- apparently, I was mistaken. Hence the confusion.
>
> After so many talks with Chandler, I know very well that upstreaming
> full OpenMP support is a long way to go. :-)
>
> Also, the whole desire to put the library into 3.5 release is a
> responce to the criticism expressed by Chandler in this message:
> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html.
>
> While we are on the topic, let me update on some other things
> happening in responce to other issues highlighted by Chandler:
>
> - This library is not being developed as an active part of the LLVM
> community, even if it is checked into SVN as part of the LLVM project and
> under its license. See r197914 where there is a code drop of many months
> worth of development with *no* change log, attribution, information, or
> other participation in any part of the community.
>
> This is changing, and many developers joining the whole OpenMP in
> clang support effort. I can say that Michael Wong and many of his
> colleagues from IBM are involved; Eric Stotzer and his colleagues from
> TI are involved; Barbara Chapman and her group from UofHouston is
> involved; Guansong Zhang from AMD is involved. Obviously, Hal Finkel,
> Chris Bergstrom, Steve Noonan and many others continue to be actively
> involved as well.
>
> Take a look at the recent activity in openmp-dev mailing list. More to
> come.
>
>
> - There has been essentially *zero* discussion with the rest of the clang
> or llvm community about this library. There are separate mailing lists
> which have nearly no traffic since the code drop.
>
> Take a look at this month's messages:
> http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html. In
> general, as more people get interested, more traffic became generated.
> It's a chicken and egg problem...
>
> - There has been no effort to make this library even work properly with
> Clang as a host compiler. See the copious notes that only Clang 3.3 is
> supported, and that not full featured.
>
> It is buildable with clang now. Moreover, regular buildbots, with both
> gcc and clang, are running:
>
> http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
> http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian
>
> - The build system is totally disjoint from LLVM's, in fact it is an
> entirely custom Perl build system that is unlike anything in use by the
> LLVM project.
>
> We started to work on improving build system. Stay tuned.
>
> - There are *zero tests* in the open source repository!!!! This is even
> called out in the original submission and on the primary website. We
> simply
> *cannot* ship and link against a runtime library which has no tests!
>
> University of Houston contributed OpenUH test suite:
> http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html. Sunita
> Chandrasekaran from the University works on integrating this suite
> into LLVM test system.
>
> BTW, any advice with how to approach this would be *much* appreciated!
>
> - No part of this library has gone through an LLVM release process either,
> not even as a "new" or "beta" project.
>
> Aha! So, you support inclusion of openmp (library, not compiler) in
> 3.5 as well? :-)
>
> Andrey
>
>
>
> On Thu, May 29, 2014 at 2:19 AM, Jack Howarth
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Andrey Bokhanko expressed interest in getting the clang-omp
>     merge done in time for the 3.5 release but wants guidance on the
>     process. I suggested starting with the creation a new clang-omp
>     branch upstream rebased on clang trunk  for generation of merge
>     patch. Unfortunately merging the  current changes from the
>     clang-omp (based on clang 3.4) to a clang-omp (based on clang
>     trunk)  looks very difficult as selective patches have been
>     committed into clang trunk from clang-omp and don't appear to have
>     been kept synchronized with the current changes from upstream.
>     Does anyone know if these new files from previous commits out of
>     clang-omp contain any local changes which haven't been back ported
>     to clang-omp? It would seem that postponing this merge will just
>     make the process harder as time goes on if selective merges from
>     clang-omp into clang trunk continue in the interim. Hopefully
>     the folks who did the original selective commits would help
>     detangle this mess.
>                Jack
>
>     _______________________________________________
>     cfe-dev mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

--
http://www.nuanti.com
the browser experts

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Jack Howarth
Alp,
    My take on this is that we have a chicken and the egg situation here. As long as the
clang-omp changes haven't been merged into clang trunk, we will be unable to switch
the default of -fopenmp from -liomp5. So once the clang-omp merge is well underway, 
interest in integrating the openmp component into the existing build machinery will
rapidly pick up.
              Jack


On Thu, May 29, 2014 at 10:39 AM, Alp Toker <[hidden email]> wrote:
Thanks for the summary Andrey.

The underlying problem For the OpenMP *runtime* is really lack of visibility and sidelining on the openmp-dev list.

It's absolutely critical to close down the low-volume openmp-dev list and fold the subscribers into one of the more active mailing lists, either llvm-dev or cfe-dev.

Until that's done the patches won't get review from the mainstream LLVM developer community, build system experts won't join in etc.

Alp.




On 29/05/2014 16:46, Andrey Bokhanko wrote:
All,

To clarify, what I meant is getting OMP runtime library (not clang-omp branch!) as a part of 3.5 release. I specifically referred to this thread: http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025 started by Jack. I thought he means libiomp when saying "openmp support" -- apparently, I was mistaken. Hence the confusion.

After so many talks with Chandler, I know very well that upstreaming full OpenMP support is a long way to go. :-)

Also, the whole desire to put the library into 3.5 release is a responce to the criticism expressed by Chandler in this message: http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html.

While we are on the topic, let me update on some other things happening in responce to other issues highlighted by Chandler:

- This library is not being developed as an active part of the LLVM
community, even if it is checked into SVN as part of the LLVM project and
under its license. See r197914 where there is a code drop of many months
worth of development with *no* change log, attribution, information, or
other participation in any part of the community.

This is changing, and many developers joining the whole OpenMP in clang support effort. I can say that Michael Wong and many of his colleagues from IBM are involved; Eric Stotzer and his colleagues from TI are involved; Barbara Chapman and her group from UofHouston is involved; Guansong Zhang from AMD is involved. Obviously, Hal Finkel, Chris Bergstrom, Steve Noonan and many others continue to be actively involved as well.

Take a look at the recent activity in openmp-dev mailing list. More to come.


- There has been essentially *zero* discussion with the rest of the clang
or llvm community about this library. There are separate mailing lists
which have nearly no traffic since the code drop.

Take a look at this month's messages: http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html. In general, as more people get interested, more traffic became generated. It's a chicken and egg problem...

- There has been no effort to make this library even work properly with
Clang as a host compiler. See the copious notes that only Clang 3.3 is
supported, and that not full featured.

It is buildable with clang now. Moreover, regular buildbots, with both gcc and clang, are running:

http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian

- The build system is totally disjoint from LLVM's, in fact it is an
entirely custom Perl build system that is unlike anything in use by the
LLVM project.

We started to work on improving build system. Stay tuned.

- There are *zero tests* in the open source repository!!!! This is even
called out in the original submission and on the primary website. We simply
*cannot* ship and link against a runtime library which has no tests!

University of Houston contributed OpenUH test suite: http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html. Sunita Chandrasekaran from the University works on integrating this suite into LLVM test system.

BTW, any advice with how to approach this would be *much* appreciated!

- No part of this library has gone through an LLVM release process either,
not even as a "new" or "beta" project.

Aha! So, you support inclusion of openmp (library, not compiler) in 3.5 as well? :-)

Andrey



On Thu, May 29, 2014 at 2:19 AM, Jack Howarth <[hidden email] <mailto:[hidden email]>> wrote:

    Andrey Bokhanko expressed interest in getting the clang-omp
    merge done in time for the 3.5 release but wants guidance on the
    process. I suggested starting with the creation a new clang-omp
    branch upstream rebased on clang trunk  for generation of merge
    patch. Unfortunately merging the  current changes from the
    clang-omp (based on clang 3.4) to a clang-omp (based on clang
    trunk)  looks very difficult as selective patches have been
    committed into clang trunk from clang-omp and don't appear to have
    been kept synchronized with the current changes from upstream.
    Does anyone know if these new files from previous commits out of
    clang-omp contain any local changes which haven't been back ported
    to clang-omp? It would seem that postponing this merge will just
    make the process harder as time goes on if selective merges from
    clang-omp into clang trunk continue in the interim. Hopefully
    the folks who did the original selective commits would help
    detangle this mess.
               Jack

    _______________________________________________
    cfe-dev mailing list
    [hidden email] <mailto:[hidden email]>
    http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev





_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

--
http://www.nuanti.com
the browser experts



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Jack Howarth
In reply to this post by Andrey Bokhanko
Andrey,
     Thanks for the clarifications. Everyone I asked had been telling me that llvm.org openmp svn was *the*
development repository for openmp but I thought that was unlikely. Where is that active tree at so we can
monitor for changes that need to be migrated to the llvm.org copy of openmp? In particular, I am interested
in finding the fix failure of…

Testing for "omp_taskyield":
Generating sources .............. success
Compiling soures ................ success
Running test with 8 threads ..... success ...sh: line 1: 87638 Segmentation fault: 11  ./bin/c/ctest_omp_taskyield > bin/c/ctest_omp_taskyield.out
... and verified with 139% certainty

in the OpenMP3.1_Validation test suite on darwin. My understanding is that this is to be fixed in
a new version of openmp but is unclear where to find that copy. Since the upstream developers of
OpenMP3.1_Validation asked for it to be added to the openmp trunk in llvm.org, it would be nice
to fix that for the copy currently checked into llvm.org's openmp trunk.
      I assume you intend to eventually move the clang-omp development completely into clang trunk, no?
That would be a worthwhile goal for clang 3.5 so that you don't have to constantly rebase the clang-omp
branch to the latest release of clang.
            Jack 


On Thu, May 29, 2014 at 9:46 AM, Andrey Bokhanko <[hidden email]> wrote:
All,

To clarify, what I meant is getting OMP runtime library (not clang-omp branch!) as a part of 3.5 release. I specifically referred to this thread: http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025 started by Jack. I thought he means libiomp when saying "openmp support" -- apparently, I was mistaken. Hence the confusion.

After so many talks with Chandler, I know very well that upstreaming full OpenMP support is a long way to go. :-)

Also, the whole desire to put the library into 3.5 release is a responce to the criticism expressed by Chandler in this message: http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html.

While we are on the topic, let me update on some other things happening in responce to other issues highlighted by Chandler:

- This library is not being developed as an active part of the LLVM
community, even if it is checked into SVN as part of the LLVM project and
under its license. See r197914 where there is a code drop of many months
worth of development with *no* change log, attribution, information, or
other participation in any part of the community.

This is changing, and many developers joining the whole OpenMP in clang support effort. I can say that Michael Wong and many of his colleagues from IBM are involved; Eric Stotzer and his colleagues from TI are involved; Barbara Chapman and her group from UofHouston is involved; Guansong Zhang from AMD is involved. Obviously, Hal Finkel, Chris Bergstrom, Steve Noonan and many others continue to be actively involved as well.

Take a look at the recent activity in openmp-dev mailing list. More to come.


- There has been essentially *zero* discussion with the rest of the clang
or llvm community about this library. There are separate mailing lists
which have nearly no traffic since the code drop.

Take a look at this month's messages: http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html. In general, as more people get interested, more traffic became generated. It's a chicken and egg problem...

- There has been no effort to make this library even work properly with
Clang as a host compiler. See the copious notes that only Clang 3.3 is
supported, and that not full featured.

It is buildable with clang now. Moreover, regular buildbots, with both gcc and clang, are running:

http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian

- The build system is totally disjoint from LLVM's, in fact it is an
entirely custom Perl build system that is unlike anything in use by the
LLVM project.

We started to work on improving build system. Stay tuned.

- There are *zero tests* in the open source repository!!!! This is even
called out in the original submission and on the primary website. We simply
*cannot* ship and link against a runtime library which has no tests!

University of Houston contributed OpenUH test suite: http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html. Sunita Chandrasekaran from the University works on integrating this suite into LLVM test system.

BTW, any advice with how to approach this would be *much* appreciated!

- No part of this library has gone through an LLVM release process either,
not even as a "new" or "beta" project.

Aha! So, you support inclusion of openmp (library, not compiler) in 3.5 as well? :-)

Andrey



On Thu, May 29, 2014 at 2:19 AM, Jack Howarth <[hidden email]> wrote:
     Andrey Bokhanko expressed interest in getting the clang-omp merge done in time for the 3.5 release but wants guidance on the process. I suggested starting with the creation a new clang-omp branch upstream rebased on clang trunk  for generation of merge patch. Unfortunately merging the  current changes from the clang-omp (based on clang 3.4) to a clang-omp (based on clang trunk)  looks very difficult as selective patches have been committed into clang trunk from clang-omp and don't appear to have been kept synchronized with the current changes from upstream. Does anyone know if these new files from previous commits out of clang-omp contain any local changes which haven't been back ported to clang-omp? It would seem that postponing this merge will just make the process harder as time goes on if selective merges from clang-omp into clang trunk continue in the interim. Hopefully the folks who did the original selective commits would help detangle this mess.
           Jack

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Philip Reames
In reply to this post by Chandler Carruth
On 05/29/2014 07:03 AM, Chandler Carruth wrote:

On Thu, May 29, 2014 at 6:46 AM, Andrey Bokhanko <[hidden email]> wrote:
While we are on the topic, let me update on some other things happening in responce to other issues highlighted by Chandler:

I just wanted to say, I am very happy to see progress starting here. =] All of my comments below are meant to help the effort move faster and deal better with the LLVM project.
What he said.  :)  Chandler is doing a better job expressing things the right way, so I'm going to defer to him. 

Andrey, thanks for the summary.  That was very helpful for those of us who haven't been following closely.

p.s. I'm actually thrilled with the idea of having openmp support in clang long term.  It just needs to be done right so that it can be maintained going into the future. 

Philip


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Hal Finkel
In reply to this post by Jack Howarth
----- Original Message -----

> From: "Jack Howarth" <[hidden email]>
> To: "Andrey Bokhanko" <[hidden email]>
> Cc: "cfe-dev" <[hidden email]>
> Sent: Thursday, May 29, 2014 10:36:47 AM
> Subject: Re: [cfe-dev] moving the clang-omp merge along
>
> Andrey,
> Thanks for the clarifications. Everyone I asked had been telling me
> that llvm.org openmp svn was *the*
> development repository for openmp but I thought that was unlikely.
> Where is that active tree at so we can
> monitor for changes that need to be migrated to the llvm.org copy of
> openmp? In particular, I am interested
> in finding the fix failure of…

Jack,

Let me try to clarify:

 - OpenMP language support

   - Intel has released its clang-omp branch to the public. On the clang-omp github page, there are also trunk-rebased repositories that, with some help from Intel, I personally maintain. Nevertheless, this code is not part of the Clang project. We have a process in place, involving extensive code reviews, to ensure implementation quality and maintainability, to help ensure multiple developers are familiar with the different areas of the codebase, etc. that the OpenMP implementation much undergo. This process, considering the limited number of qualified reviewers and the overall quantity of code, is operating at a reasonable pace. Personally, I would love for it to go much faster (recall, *I* maintain the trunk-rebased branch), but that is not necessarily in the best interest of the community. Whether this makes the 3.5 release in total, or takes until 3.6, in the long run, seems to matter very little. There has been steady (and now accelerating) progress toward contributing OpenMP support to Clang itself. Anyone who *needs* OpenMP support now can get it (albeit by applying some extra patches), and the pieces that are in the upstream Clang repository are of notably higher quality than the original code from Intel's clang-omp branch. In the end, everyone will benefit from the improved quality of the implementation.

 - OpenMP runtime library

   - This is the piece that is currently on llvm.org as a separate project. The openmp repository on llvm.org *is* *the* repository for this, at least as it exists as an open-source project. I know of no other. Obviously, other entities can have internal branches of any project, as Intel may have in this case, and these internal branches may contain bug fixes not yet contributed to the open source project. When this happens it is unfortunate, and it seems perfectly reasonable to push anyone with such internal fixes to contribute them sooner rather than later. Nevertheless, no one has been withholding otherwise-public code from you -- or, if they have, they've been withholding it from me too ;) If you'd like to push for such a code release, the openmp-dev list is the appropriate place for this, not here.

I hope this helps.

 -Hal

>
>
>
> Testing for "omp_taskyield":
> Generating sources .............. success
> Compiling soures ................ success
> Running test with 8 threads ..... success ...sh: line 1: 87638
> Segmentation fault: 11 ./bin/c/ctest_omp_taskyield >
> bin/c/ctest_omp_taskyield.out
> ... and verified with 139% certainty
>
>
> in the OpenMP3.1_Validation test suite on darwin. My understanding is
> that this is to be fixed in
> a new version of openmp but is unclear where to find that copy. Since
> the upstream developers of
> OpenMP3.1_Validation asked for it to be added to the openmp trunk in
> llvm.org , it would be nice
>
> to fix that for the copy currently checked into llvm.org 's openmp
> trunk.
> I assume you intend to eventually move the clang-omp development
> completely into clang trunk, no?
> That would be a worthwhile goal for clang 3.5 so that you don't have
> to constantly rebase the clang-omp
> branch to the latest release of clang.
> Jack
>
>
>
>
> On Thu, May 29, 2014 at 9:46 AM, Andrey Bokhanko <
> [hidden email] > wrote:
>
>
>
> All,
>
> To clarify, what I meant is getting OMP runtime library (not
> clang-omp branch!) as a part of 3.5 release. I specifically referred
> to this thread:
> http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025
> started by Jack. I thought he means libiomp when saying "openmp
> support" -- apparently, I was mistaken. Hence the confusion.
>
> After so many talks with Chandler, I know very well that upstreaming
> full OpenMP support is a long way to go. :-)
>
> Also, the whole desire to put the library into 3.5 release is a
> responce to the criticism expressed by Chandler in this message:
> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html
> .
>
> While we are on the topic, let me update on some other things
> happening in responce to other issues highlighted by Chandler:
>
> - This library is not being developed as an active part of the LLVM
> community, even if it is checked into SVN as part of the LLVM project
> and
> under its license. See r197914 where there is a code drop of many
> months
> worth of development with *no* change log, attribution, information,
> or
> other participation in any part of the community.
>
> This is changing, and many developers joining the whole OpenMP in
> clang support effort. I can say that Michael Wong and many of his
> colleagues from IBM are involved; Eric Stotzer and his colleagues
> from TI are involved; Barbara Chapman and her group from UofHouston
> is involved; Guansong Zhang from AMD is involved. Obviously, Hal
> Finkel, Chris Bergstrom, Steve Noonan and many others continue to be
> actively involved as well.
>
> Take a look at the recent activity in openmp-dev mailing list. More
> to come.
>
>
> - There has been essentially *zero* discussion with the rest of the
> clang
> or llvm community about this library. There are separate mailing
> lists
> which have nearly no traffic since the code drop.
>
> Take a look at this month's messages:
> http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html .
> In general, as more people get interested, more traffic became
> generated. It's a chicken and egg problem...
>
> - There has been no effort to make this library even work properly
> with
> Clang as a host compiler. See the copious notes that only Clang 3.3
> is
> supported, and that not full featured.
>
> It is buildable with clang now. Moreover, regular buildbots, with
> both gcc and clang, are running:
>
> http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
> http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian
>
> - The build system is totally disjoint from LLVM's, in fact it is an
> entirely custom Perl build system that is unlike anything in use by
> the
> LLVM project.
>
> We started to work on improving build system. Stay tuned.
>
> - There are *zero tests* in the open source repository!!!! This is
> even
> called out in the original submission and on the primary website. We
> simply
> *cannot* ship and link against a runtime library which has no tests!
>
> University of Houston contributed OpenUH test suite:
> http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html
> . Sunita Chandrasekaran from the University works on integrating
> this suite into LLVM test system.
>
> BTW, any advice with how to approach this would be *much*
> appreciated!
>
> - No part of this library has gone through an LLVM release process
> either,
> not even as a "new" or "beta" project.
>
> Aha! So, you support inclusion of openmp (library, not compiler) in
> 3.5 as well? :-)
>
> Andrey
>
>
>
>
>
>
> On Thu, May 29, 2014 at 2:19 AM, Jack Howarth <
> [hidden email] > wrote:
>
>
>
>
> Andrey Bokhanko expressed interest in getting the clang-omp merge
> done in time for the 3.5 release but wants guidance on the process.
> I suggested starting with the creation a new clang-omp branch
> upstream rebased on clang trunk for generation of merge patch.
> Unfortunately merging the current changes from the clang-omp (based
> on clang 3.4) to a clang-omp (based on clang trunk) looks very
> difficult as selective patches have been committed into clang trunk
> from clang-omp and don't appear to have been kept synchronized with
> the current changes from upstream. Does anyone know if these new
> files from previous commits out of clang-omp contain any local
> changes which haven't been back ported to clang-omp? It would seem
> that postponing this merge will just make the process harder as time
> goes on if selective merges from clang-omp into clang trunk continue
> in the interim. Hopefully the folks who did the original selective
> commits would help detangle this mess.
> Jack
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>

--
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Jack Howarth
Hal,
    I didn't mean to imply anyone was hiding fixes but whether there multiple repositories for openmp which just weren't well advertised. In particular, the existence of the https://www.openmprtl.org site and how that site figures in the general transmission of bug fixes between it and the various compiler projects that might use openmp.
               Jack


On Thu, May 29, 2014 at 1:03 PM, Hal Finkel <[hidden email]> wrote:
----- Original Message -----
> From: "Jack Howarth" <[hidden email]>
> To: "Andrey Bokhanko" <[hidden email]>
> Cc: "cfe-dev" <[hidden email]>
> Sent: Thursday, May 29, 2014 10:36:47 AM
> Subject: Re: [cfe-dev] moving the clang-omp merge along
>
> Andrey,
> Thanks for the clarifications. Everyone I asked had been telling me
> that llvm.org openmp svn was *the*
> development repository for openmp but I thought that was unlikely.
> Where is that active tree at so we can
> monitor for changes that need to be migrated to the llvm.org copy of
> openmp? In particular, I am interested
> in finding the fix failure of…

Jack,

Let me try to clarify:

 - OpenMP language support

   - Intel has released its clang-omp branch to the public. On the clang-omp github page, there are also trunk-rebased repositories that, with some help from Intel, I personally maintain. Nevertheless, this code is not part of the Clang project. We have a process in place, involving extensive code reviews, to ensure implementation quality and maintainability, to help ensure multiple developers are familiar with the different areas of the codebase, etc. that the OpenMP implementation much undergo. This process, considering the limited number of qualified reviewers and the overall quantity of code, is operating at a reasonable pace. Personally, I would love for it to go much faster (recall, *I* maintain the trunk-rebased branch), but that is not necessarily in the best interest of the community. Whether this makes the 3.5 release in total, or takes until 3.6, in the long run, seems to matter very little. There has been steady (and now accelerating) progress toward contributing OpenMP support to Clang itself. Anyone who *needs* OpenMP support now can get it (albeit by applying some extra patches), and the pieces that are in the upstream Clang repository are of notably higher quality than the original code from Intel's clang-omp branch. In the end, everyone will benefit from the improved quality of the implementation.

 - OpenMP runtime library

   - This is the piece that is currently on llvm.org as a separate project. The openmp repository on llvm.org *is* *the* repository for this, at least as it exists as an open-source project. I know of no other. Obviously, other entities can have internal branches of any project, as Intel may have in this case, and these internal branches may contain bug fixes not yet contributed to the open source project. When this happens it is unfortunate, and it seems perfectly reasonable to push anyone with such internal fixes to contribute them sooner rather than later. Nevertheless, no one has been withholding otherwise-public code from you -- or, if they have, they've been withholding it from me too ;) If you'd like to push for such a code release, the openmp-dev list is the appropriate place for this, not here.

I hope this helps.

 -Hal

>
>
>
> Testing for "omp_taskyield":
> Generating sources .............. success
> Compiling soures ................ success
> Running test with 8 threads ..... success ...sh: line 1: 87638
> Segmentation fault: 11 ./bin/c/ctest_omp_taskyield >
> bin/c/ctest_omp_taskyield.out
> ... and verified with 139% certainty
>
>
> in the OpenMP3.1_Validation test suite on darwin. My understanding is
> that this is to be fixed in
> a new version of openmp but is unclear where to find that copy. Since
> the upstream developers of
> OpenMP3.1_Validation asked for it to be added to the openmp trunk in
> llvm.org , it would be nice
>
> to fix that for the copy currently checked into llvm.org 's openmp
> trunk.
> I assume you intend to eventually move the clang-omp development
> completely into clang trunk, no?
> That would be a worthwhile goal for clang 3.5 so that you don't have
> to constantly rebase the clang-omp
> branch to the latest release of clang.
> Jack
>
>
>
>
> On Thu, May 29, 2014 at 9:46 AM, Andrey Bokhanko <
> [hidden email] > wrote:
>
>
>
> All,
>
> To clarify, what I meant is getting OMP runtime library (not
> clang-omp branch!) as a part of 3.5 release. I specifically referred
> to this thread:
> http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025
> started by Jack. I thought he means libiomp when saying "openmp
> support" -- apparently, I was mistaken. Hence the confusion.
>
> After so many talks with Chandler, I know very well that upstreaming
> full OpenMP support is a long way to go. :-)
>
> Also, the whole desire to put the library into 3.5 release is a
> responce to the criticism expressed by Chandler in this message:
> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html
> .
>
> While we are on the topic, let me update on some other things
> happening in responce to other issues highlighted by Chandler:
>
> - This library is not being developed as an active part of the LLVM
> community, even if it is checked into SVN as part of the LLVM project
> and
> under its license. See r197914 where there is a code drop of many
> months
> worth of development with *no* change log, attribution, information,
> or
> other participation in any part of the community.
>
> This is changing, and many developers joining the whole OpenMP in
> clang support effort. I can say that Michael Wong and many of his
> colleagues from IBM are involved; Eric Stotzer and his colleagues
> from TI are involved; Barbara Chapman and her group from UofHouston
> is involved; Guansong Zhang from AMD is involved. Obviously, Hal
> Finkel, Chris Bergstrom, Steve Noonan and many others continue to be
> actively involved as well.
>
> Take a look at the recent activity in openmp-dev mailing list. More
> to come.
>
>
> - There has been essentially *zero* discussion with the rest of the
> clang
> or llvm community about this library. There are separate mailing
> lists
> which have nearly no traffic since the code drop.
>
> Take a look at this month's messages:
> http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html .
> In general, as more people get interested, more traffic became
> generated. It's a chicken and egg problem...
>
> - There has been no effort to make this library even work properly
> with
> Clang as a host compiler. See the copious notes that only Clang 3.3
> is
> supported, and that not full featured.
>
> It is buildable with clang now. Moreover, regular buildbots, with
> both gcc and clang, are running:
>
> http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
> http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian
>
> - The build system is totally disjoint from LLVM's, in fact it is an
> entirely custom Perl build system that is unlike anything in use by
> the
> LLVM project.
>
> We started to work on improving build system. Stay tuned.
>
> - There are *zero tests* in the open source repository!!!! This is
> even
> called out in the original submission and on the primary website. We
> simply
> *cannot* ship and link against a runtime library which has no tests!
>
> University of Houston contributed OpenUH test suite:
> http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html
> . Sunita Chandrasekaran from the University works on integrating
> this suite into LLVM test system.
>
> BTW, any advice with how to approach this would be *much*
> appreciated!
>
> - No part of this library has gone through an LLVM release process
> either,
> not even as a "new" or "beta" project.
>
> Aha! So, you support inclusion of openmp (library, not compiler) in
> 3.5 as well? :-)
>
> Andrey
>
>
>
>
>
>
> On Thu, May 29, 2014 at 2:19 AM, Jack Howarth <
> [hidden email] > wrote:
>
>
>
>
> Andrey Bokhanko expressed interest in getting the clang-omp merge
> done in time for the 3.5 release but wants guidance on the process.
> I suggested starting with the creation a new clang-omp branch
> upstream rebased on clang trunk for generation of merge patch.
> Unfortunately merging the current changes from the clang-omp (based
> on clang 3.4) to a clang-omp (based on clang trunk) looks very
> difficult as selective patches have been committed into clang trunk
> from clang-omp and don't appear to have been kept synchronized with
> the current changes from upstream. Does anyone know if these new
> files from previous commits out of clang-omp contain any local
> changes which haven't been back ported to clang-omp? It would seem
> that postponing this merge will just make the process harder as time
> goes on if selective merges from clang-omp into clang trunk continue
> in the interim. Hopefully the folks who did the original selective
> commits would help detangle this mess.
> Jack
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>

--
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Hal Finkel

----- Original Message -----

> From: "Jack Howarth" <[hidden email]>
> To: "Hal Finkel" <[hidden email]>
> Cc: "cfe-dev" <[hidden email]>, "Andrey Bokhanko" <[hidden email]>
> Sent: Thursday, May 29, 2014 12:12:00 PM
> Subject: Re: [cfe-dev] moving the clang-omp merge along
>
>
> Hal,
> I didn't mean to imply anyone was hiding fixes but whether there
> multiple repositories for openmp which just weren't well advertised.
> In particular, the existence of the https://www.openmprtl.org site
> and how that site figures in the general transmission of bug fixes
> between it and the various compiler projects that might use openmp.

The releases on the openmprtl are managed by Intel, and the site existed before the library was contributed to the LLVM project, but certainly lag the code in the LLVM openmp repository at this point.

 -Hal

> Jack
>
>
>
> On Thu, May 29, 2014 at 1:03 PM, Hal Finkel < [hidden email] >
> wrote:
>
>
>
> ----- Original Message -----
> > From: "Jack Howarth" < [hidden email] >
> > To: "Andrey Bokhanko" < [hidden email] >
> > Cc: "cfe-dev" < [hidden email] >
> > Sent: Thursday, May 29, 2014 10:36:47 AM
> > Subject: Re: [cfe-dev] moving the clang-omp merge along
> >
> > Andrey,
> > Thanks for the clarifications. Everyone I asked had been telling me
> > that llvm.org openmp svn was *the*
> > development repository for openmp but I thought that was unlikely.
> > Where is that active tree at so we can
> > monitor for changes that need to be migrated to the llvm.org copy
> > of
> > openmp? In particular, I am interested
> > in finding the fix failure of…
>
> Jack,
>
> Let me try to clarify:
>
> - OpenMP language support
>
> - Intel has released its clang-omp branch to the public. On the
> clang-omp github page, there are also trunk-rebased repositories
> that, with some help from Intel, I personally maintain.
> Nevertheless, this code is not part of the Clang project. We have a
> process in place, involving extensive code reviews, to ensure
> implementation quality and maintainability, to help ensure multiple
> developers are familiar with the different areas of the codebase,
> etc. that the OpenMP implementation much undergo. This process,
> considering the limited number of qualified reviewers and the
> overall quantity of code, is operating at a reasonable pace.
> Personally, I would love for it to go much faster (recall, *I*
> maintain the trunk-rebased branch), but that is not necessarily in
> the best interest of the community. Whether this makes the 3.5
> release in total, or takes until 3.6, in the long run, seems to
> matter very little. There has been steady (and now accelerating)
> progress toward contributing OpenMP support to Clang itself. Anyone
> who *needs* OpenMP support now can get it (albeit by applying some
> extra patches), and the pieces that are in the upstream Clang
> repository are of notably higher quality than the original code from
> Intel's clang-omp branch. In the end, everyone will benefit from the
> improved quality of the implementation.
>
> - OpenMP runtime library
>
> - This is the piece that is currently on llvm.org as a separate
> project. The openmp repository on llvm.org *is* *the* repository for
> this, at least as it exists as an open-source project. I know of no
> other. Obviously, other entities can have internal branches of any
> project, as Intel may have in this case, and these internal branches
> may contain bug fixes not yet contributed to the open source
> project. When this happens it is unfortunate, and it seems perfectly
> reasonable to push anyone with such internal fixes to contribute
> them sooner rather than later. Nevertheless, no one has been
> withholding otherwise-public code from you -- or, if they have,
> they've been withholding it from me too ;) If you'd like to push for
> such a code release, the openmp-dev list is the appropriate place
> for this, not here.
>
> I hope this helps.
>
> -Hal
>
>
> >
> >
> >
> > Testing for "omp_taskyield":
> > Generating sources .............. success
> > Compiling soures ................ success
> > Running test with 8 threads ..... success ...sh: line 1: 87638
> > Segmentation fault: 11 ./bin/c/ctest_omp_taskyield >
> > bin/c/ctest_omp_taskyield.out
> > ... and verified with 139% certainty
> >
> >
> > in the OpenMP3.1_Validation test suite on darwin. My understanding
> > is
> > that this is to be fixed in
> > a new version of openmp but is unclear where to find that copy.
> > Since
> > the upstream developers of
> > OpenMP3.1_Validation asked for it to be added to the openmp trunk
> > in
> > llvm.org , it would be nice
>
>
> >
> > to fix that for the copy currently checked into llvm.org 's openmp
> > trunk.
> > I assume you intend to eventually move the clang-omp development
> > completely into clang trunk, no?
> > That would be a worthwhile goal for clang 3.5 so that you don't
> > have
> > to constantly rebase the clang-omp
> > branch to the latest release of clang.
> > Jack
> >
> >
> >
> >
> > On Thu, May 29, 2014 at 9:46 AM, Andrey Bokhanko <
> > [hidden email] > wrote:
> >
> >
> >
> > All,
> >
> > To clarify, what I meant is getting OMP runtime library (not
> > clang-omp branch!) as a part of 3.5 release. I specifically
> > referred
> > to this thread:
> > http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025
> > started by Jack. I thought he means libiomp when saying "openmp
> > support" -- apparently, I was mistaken. Hence the confusion.
> >
> > After so many talks with Chandler, I know very well that
> > upstreaming
> > full OpenMP support is a long way to go. :-)
> >
> > Also, the whole desire to put the library into 3.5 release is a
> > responce to the criticism expressed by Chandler in this message:
> > http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html
> > .
> >
> > While we are on the topic, let me update on some other things
> > happening in responce to other issues highlighted by Chandler:
> >
> > - This library is not being developed as an active part of the LLVM
> > community, even if it is checked into SVN as part of the LLVM
> > project
> > and
> > under its license. See r197914 where there is a code drop of many
> > months
> > worth of development with *no* change log, attribution,
> > information,
> > or
> > other participation in any part of the community.
> >
> > This is changing, and many developers joining the whole OpenMP in
> > clang support effort. I can say that Michael Wong and many of his
> > colleagues from IBM are involved; Eric Stotzer and his colleagues
> > from TI are involved; Barbara Chapman and her group from UofHouston
> > is involved; Guansong Zhang from AMD is involved. Obviously, Hal
> > Finkel, Chris Bergstrom, Steve Noonan and many others continue to
> > be
> > actively involved as well.
> >
> > Take a look at the recent activity in openmp-dev mailing list. More
> > to come.
> >
> >
> > - There has been essentially *zero* discussion with the rest of the
> > clang
> > or llvm community about this library. There are separate mailing
> > lists
> > which have nearly no traffic since the code drop.
> >
> > Take a look at this month's messages:
> > http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html
> > .
> > In general, as more people get interested, more traffic became
> > generated. It's a chicken and egg problem...
> >
> > - There has been no effort to make this library even work properly
> > with
> > Clang as a host compiler. See the copious notes that only Clang 3.3
> > is
> > supported, and that not full featured.
> >
> > It is buildable with clang now. Moreover, regular buildbots, with
> > both gcc and clang, are running:
> >
> > http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
> > http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian
> >
> > - The build system is totally disjoint from LLVM's, in fact it is
> > an
> > entirely custom Perl build system that is unlike anything in use by
> > the
> > LLVM project.
> >
> > We started to work on improving build system. Stay tuned.
> >
> > - There are *zero tests* in the open source repository!!!! This is
> > even
> > called out in the original submission and on the primary website.
> > We
> > simply
> > *cannot* ship and link against a runtime library which has no
> > tests!
> >
> > University of Houston contributed OpenUH test suite:
> > http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html
> > . Sunita Chandrasekaran from the University works on integrating
> > this suite into LLVM test system.
> >
> > BTW, any advice with how to approach this would be *much*
> > appreciated!
> >
> > - No part of this library has gone through an LLVM release process
> > either,
> > not even as a "new" or "beta" project.
> >
> > Aha! So, you support inclusion of openmp (library, not compiler) in
> > 3.5 as well? :-)
> >
> > Andrey
> >
> >
> >
> >
> >
> >
> > On Thu, May 29, 2014 at 2:19 AM, Jack Howarth <
> > [hidden email] > wrote:
> >
> >
> >
> >
> > Andrey Bokhanko expressed interest in getting the clang-omp merge
> > done in time for the 3.5 release but wants guidance on the process.
> > I suggested starting with the creation a new clang-omp branch
> > upstream rebased on clang trunk for generation of merge patch.
> > Unfortunately merging the current changes from the clang-omp (based
> > on clang 3.4) to a clang-omp (based on clang trunk) looks very
> > difficult as selective patches have been committed into clang trunk
> > from clang-omp and don't appear to have been kept synchronized with
> > the current changes from upstream. Does anyone know if these new
> > files from previous commits out of clang-omp contain any local
> > changes which haven't been back ported to clang-omp? It would seem
> > that postponing this merge will just make the process harder as
> > time
> > goes on if selective merges from clang-omp into clang trunk
> > continue
> > in the interim. Hopefully the folks who did the original selective
> > commits would help detangle this mess.
> > Jack
> >
> > _______________________________________________
> > cfe-dev mailing list
> > [hidden email]
> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> >
> >
> >
> >
> > _______________________________________________
> > cfe-dev mailing list
> > [hidden email]
> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> >
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
>

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Alexey Bataev
In reply to this post by Jack Howarth
Hi Chandler,

> 3) we need a good switch in Clang to use iomp (I know you or someone on the
> patch thread worked on this, did it land? if so, done!)
There is option -fopenmp=libiomp5 which makes clang to generate OpenMP
code for libiomp5. Currently the code is generated only for '#pragma omp
parallel' (just some initial codegen, you can check test in
clang/test/OpenMP/parallel_codegen.cpp). I'm working on CodeGen for
'#pragma omp threadprivate'. It will be ready soon.

Best regards,
Alexey Bataev
=============
Software Engineer
Intel Compiler Team
Intel Corp.

29.05.2014 18:39, [hidden email] пишет:

> Send cfe-dev mailing list submissions to
> [hidden email]
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> or, via email, send a message with subject or body 'help' to
> [hidden email]
>
> You can reach the person managing the list at
> [hidden email]
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of cfe-dev digest..."
>
>
> Today's Topics:
>
>     1. Re: moving the clang-omp merge along (Andrey Bokhanko)
>     2. Re: moving the clang-omp merge along (Chandler Carruth)
>     3. Re: moving the clang-omp merge along (Alp Toker)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 29 May 2014 17:46:27 +0400
> From: Andrey Bokhanko <[hidden email]>
> To: Jack Howarth <[hidden email]>, Chandler Carruth
> <[hidden email]>
> Cc: cfe-dev <[hidden email]>
> Subject: Re: [cfe-dev] moving the clang-omp merge along
> Message-ID:
> <CAJHoqafFFSLfy7cx6Uds2fBi6GhigSrhdM3xBW4aRPY1xHF=[hidden email]>
> Content-Type: text/plain; charset="utf-8"
>
> All,
>
> To clarify, what I meant is getting OMP runtime library (not clang-omp
> branch!) as a part of 3.5 release. I specifically referred to this thread:
> http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025
> started by Jack. I thought he means libiomp when saying "openmp support" --
> apparently, I was mistaken. Hence the confusion.
>
> After so many talks with Chandler, I know very well that upstreaming full
> OpenMP support is a long way to go. :-)
>
> Also, the whole desire to put the library into 3.5 release is a responce to
> the criticism expressed by Chandler in this message:
> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html
> .
>
> While we are on the topic, let me update on some other things happening in
> responce to other issues highlighted by Chandler:
>
> - This library is not being developed as an active part of the LLVM
> community, even if it is checked into SVN as part of the LLVM project and
> under its license. See r197914 where there is a code drop of many months
> worth of development with *no* change log, attribution, information, or
> other participation in any part of the community.
>
> This is changing, and many developers joining the whole OpenMP in clang
> support effort. I can say that Michael Wong and many of his colleagues from
> IBM are involved; Eric Stotzer and his colleagues from TI are involved;
> Barbara Chapman and her group from UofHouston is involved; Guansong Zhang
> from AMD is involved. Obviously, Hal Finkel, Chris Bergstrom, Steve Noonan
> and many others continue to be actively involved as well.
>
> Take a look at the recent activity in openmp-dev mailing list. More to come.
>
>
> - There has been essentially *zero* discussion with the rest of the clang
> or llvm community about this library. There are separate mailing lists
> which have nearly no traffic since the code drop.
>
> Take a look at this month's messages:
> http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html. In
> general, as more people get interested, more traffic became generated. It's
> a chicken and egg problem...
>
> - There has been no effort to make this library even work properly with
> Clang as a host compiler. See the copious notes that only Clang 3.3 is
> supported, and that not full featured.
>
> It is buildable with clang now. Moreover, regular buildbots, with both gcc
> and clang, are running:
>
> http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
> http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian
>
> - The build system is totally disjoint from LLVM's, in fact it is an
> entirely custom Perl build system that is unlike anything in use by the
> LLVM project.
>
> We started to work on improving build system. Stay tuned.
>
> - There are *zero tests* in the open source repository!!!! This is even
> called out in the original submission and on the primary website. We simply
> *cannot* ship and link against a runtime library which has no tests!
>
> University of Houston contributed OpenUH test suite:
> http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html.
> Sunita Chandrasekaran from the University works on integrating this suite
> into LLVM test system.
>
> BTW, any advice with how to approach this would be *much* appreciated!
>
> - No part of this library has gone through an LLVM release process either,
> not even as a "new" or "beta" project.
>
> Aha! So, you support inclusion of openmp (library, not compiler) in 3.5 as
> well? :-)
>
> Andrey
>
>
>
> On Thu, May 29, 2014 at 2:19 AM, Jack Howarth <
> [hidden email]> wrote:
>
>>       Andrey Bokhanko expressed interest in getting the clang-omp
>> merge done in time for the 3.5 release but wants guidance on the process. I
>> suggested starting with the creation a new clang-omp branch upstream
>> rebased on clang trunk  for generation of merge patch. Unfortunately
>> merging the  current changes from the clang-omp (based on clang 3.4) to a
>> clang-omp (based on clang trunk)  looks very difficult as selective patches
>> have been committed into clang trunk from clang-omp and don't appear to
>> have been kept synchronized with the current changes from upstream. Does
>> anyone know if these new files from previous commits out of clang-omp
>> contain any local changes which haven't been back ported to clang-omp? It
>> would seem that postponing this merge will just make the process harder as
>> time goes on if selective merges from clang-omp into clang trunk continue
>> in the interim. Hopefully the folks who did the original selective commits
>> would help detangle this mess.
>>             Jack
>>
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>
>>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.cs.uiuc.edu/pipermail/cfe-dev/attachments/20140529/f0d1442a/attachment-0001.html>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 29 May 2014 07:03:44 -0700
> From: Chandler Carruth <[hidden email]>
> To: Andrey Bokhanko <[hidden email]>
> Cc: cfe-dev <[hidden email]>
> Subject: Re: [cfe-dev] moving the clang-omp merge along
> Message-ID:
> <[hidden email]>
> Content-Type: text/plain; charset="utf-8"
>
> On Thu, May 29, 2014 at 6:46 AM, Andrey Bokhanko
> <[hidden email]>wrote:
>
>> While we are on the topic, let me update on some other things happening in
>> responce to other issues highlighted by Chandler:
>>
> I just wanted to say, I am very happy to see progress starting here. =] All
> of my comments below are meant to help the effort move faster and deal
> better with the LLVM project.
>
>
>> - This library is not being developed as an active part of the LLVM
>> community, even if it is checked into SVN as part of the LLVM project and
>> under its license. See r197914 where there is a code drop of many months
>> worth of development with *no* change log, attribution, information, or
>> other participation in any part of the community.
>>
>> This is changing, and many developers joining the whole OpenMP in clang
>> support effort. I can say that Michael Wong and many of his colleagues from
>> IBM are involved; Eric Stotzer and his colleagues from TI are involved;
>> Barbara Chapman and her group from UofHouston is involved; Guansong Zhang
>> from AMD is involved. Obviously, Hal Finkel, Chris Bergstrom, Steve Noonan
>> and many others continue to be actively involved as well.
>>
>> Take a look at the recent activity in openmp-dev mailing list. More to
>> come.
>>
> Pretty happy about this, and looking forward to more. Some things that
> would be helpful (IMO, others may disagree) to start looking into how/when
> to adopt or integrate:
>
> - Start more closely following the LLVM developer policy around code review
> and small, incremental patches and development.
>    - Maybe use code review tools like Phabricator? Dunno what would be
> needed to make this work well, probably some stuff...
> - Check that the codebase compiles with the same host toolchain baseline as
> the rest of the LLVM project[1]
> - Maybe work out a plan toward generally conforming with the coding
> standards[2] where applicable?
>
>
>> It is buildable with clang now. Moreover, regular buildbots, with both gcc
>> and clang, are running:
>>
>> http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
>> http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian
>>
> Just want to say, this in particular is fantastic.
>
>
>> - The build system is totally disjoint from LLVM's, in fact it is an
>> entirely custom Perl build system that is unlike anything in use by the
>> LLVM project.
>>
>> We started to work on improving build system. Stay tuned.
>>
> Very cool. I would look closely at the compiler-rt and libc++ CMake builds.
> Hopefully useful.
>
>
>> - There are *zero tests* in the open source repository!!!! This is even
>> called out in the original submission and on the primary website. We simply
>> *cannot* ship and link against a runtime library which has no tests!
>>
>> University of Houston contributed OpenUH test suite:
>> http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html.
>> Sunita Chandrasekaran from the University works on integrating this suite
>> into LLVM test system.
>>
>> BTW, any advice with how to approach this would be *much* appreciated!
>>
> I think the right way to go is something along the lines of the ASan (in
> compiler-rt) lit tests, but maybe others have a better idea. I agree that
> testing here is hard.
>
>
>> - No part of this library has gone through an LLVM release process either,
>> not even as a "new" or "beta" project.
>>
>> Aha! So, you support inclusion of openmp (library, not compiler) in 3.5 as
>> well? :-)
>>
> Hehe, I see what you did there.
>
> I do genuinely support it going through the release process, but I think
> that there are two things that need to be pretty solid first:
>
> 1) the build system work probably needs to be pretty solid. i'd be happy
> with support on par with compiler-rt or libc++ in CMake...
> 2) the test suite needs to be at least reasonably well integrated so
> release testers actually hit it and exercise the code
> 3) we need a good switch in Clang to use iomp (I know you or someone on the
> patch thread worked on this, did it land? if so, done!)
>
> Maybe others have other thoughts, but I think those three are my key ones.
>
> Anyways, thanks for the clarification and all the hard work on this stuff.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.cs.uiuc.edu/pipermail/cfe-dev/attachments/20140529/2c31f1bf/attachment-0001.html>
>
> ------------------------------
>
> Message: 3
> Date: Thu, 29 May 2014 17:39:32 +0300
> From: Alp Toker <[hidden email]>
> To: Andrey Bokhanko <[hidden email]>, Jack Howarth
> <[hidden email]>, Chandler Carruth
> <[hidden email]>
> Cc: cfe-dev <[hidden email]>
> Subject: Re: [cfe-dev] moving the clang-omp merge along
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Thanks for the summary Andrey.
>
> The underlying problem For the OpenMP *runtime* is really lack of
> visibility and sidelining on the openmp-dev list.
>
> It's absolutely critical to close down the low-volume openmp-dev list
> and fold the subscribers into one of the more active mailing lists,
> either llvm-dev or cfe-dev.
>
> Until that's done the patches won't get review from the mainstream LLVM
> developer community, build system experts won't join in etc.
>
> Alp.
>
>
>
> On 29/05/2014 16:46, Andrey Bokhanko wrote:
>> All,
>>
>> To clarify, what I meant is getting OMP runtime library (not clang-omp
>> branch!) as a part of 3.5 release. I specifically referred to this
>> thread:
>> http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025
>> started by Jack. I thought he means libiomp when saying "openmp
>> support" -- apparently, I was mistaken. Hence the confusion.
>>
>> After so many talks with Chandler, I know very well that upstreaming
>> full OpenMP support is a long way to go. :-)
>>
>> Also, the whole desire to put the library into 3.5 release is a
>> responce to the criticism expressed by Chandler in this message:
>> http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html.
>>
>> While we are on the topic, let me update on some other things
>> happening in responce to other issues highlighted by Chandler:
>>
>> - This library is not being developed as an active part of the LLVM
>> community, even if it is checked into SVN as part of the LLVM project and
>> under its license. See r197914 where there is a code drop of many months
>> worth of development with *no* change log, attribution, information, or
>> other participation in any part of the community.
>>
>> This is changing, and many developers joining the whole OpenMP in
>> clang support effort. I can say that Michael Wong and many of his
>> colleagues from IBM are involved; Eric Stotzer and his colleagues from
>> TI are involved; Barbara Chapman and her group from UofHouston is
>> involved; Guansong Zhang from AMD is involved. Obviously, Hal Finkel,
>> Chris Bergstrom, Steve Noonan and many others continue to be actively
>> involved as well.
>>
>> Take a look at the recent activity in openmp-dev mailing list. More to
>> come.
>>
>>
>> - There has been essentially *zero* discussion with the rest of the clang
>> or llvm community about this library. There are separate mailing lists
>> which have nearly no traffic since the code drop.
>>
>> Take a look at this month's messages:
>> http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html. In
>> general, as more people get interested, more traffic became generated.
>> It's a chicken and egg problem...
>>
>> - There has been no effort to make this library even work properly with
>> Clang as a host compiler. See the copious notes that only Clang 3.3 is
>> supported, and that not full featured.
>>
>> It is buildable with clang now. Moreover, regular buildbots, with both
>> gcc and clang, are running:
>>
>> http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
>> http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian
>>
>> - The build system is totally disjoint from LLVM's, in fact it is an
>> entirely custom Perl build system that is unlike anything in use by the
>> LLVM project.
>>
>> We started to work on improving build system. Stay tuned.
>>
>> - There are *zero tests* in the open source repository!!!! This is even
>> called out in the original submission and on the primary website. We
>> simply
>> *cannot* ship and link against a runtime library which has no tests!
>>
>> University of Houston contributed OpenUH test suite:
>> http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html. Sunita
>> Chandrasekaran from the University works on integrating this suite
>> into LLVM test system.
>>
>> BTW, any advice with how to approach this would be *much* appreciated!
>>
>> - No part of this library has gone through an LLVM release process either,
>> not even as a "new" or "beta" project.
>>
>> Aha! So, you support inclusion of openmp (library, not compiler) in
>> 3.5 as well? :-)
>>
>> Andrey
>>
>>
>>
>> On Thu, May 29, 2014 at 2:19 AM, Jack Howarth
>> <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>      Andrey Bokhanko expressed interest in getting the clang-omp
>>      merge done in time for the 3.5 release but wants guidance on the
>>      process. I suggested starting with the creation a new clang-omp
>>      branch upstream rebased on clang trunk  for generation of merge
>>      patch. Unfortunately merging the  current changes from the
>>      clang-omp (based on clang 3.4) to a clang-omp (based on clang
>>      trunk)  looks very difficult as selective patches have been
>>      committed into clang trunk from clang-omp and don't appear to have
>>      been kept synchronized with the current changes from upstream.
>>      Does anyone know if these new files from previous commits out of
>>      clang-omp contain any local changes which haven't been back ported
>>      to clang-omp? It would seem that postponing this merge will just
>>      make the process harder as time goes on if selective merges from
>>      clang-omp into clang trunk continue in the interim. Hopefully
>>      the folks who did the original selective commits would help
>>      detangle this mess.
>>                 Jack
>>
>>      _______________________________________________
>>      cfe-dev mailing list
>>      [hidden email] <mailto:[hidden email]>
>>      http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>
>>
>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> [hidden email]
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
--
Best regards,
Alexey Bataev
=============
Software Engineer
Intel Compiler Team
Intel Corp.
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Andrey Bokhanko
In reply to this post by Chandler Carruth
Chandler,

Thanks for the encouragement and the additional suggestions! -- we can always rely on you to keep us busy. :-)

> 3) we need a good switch in Clang to use iomp (I know you or someone on the patch thread worked on this, did it land? if so, done!)

Yes, this is done -- see Alexey's message.

Andrey



On Thu, May 29, 2014 at 6:03 PM, Chandler Carruth <[hidden email]> wrote:

On Thu, May 29, 2014 at 6:46 AM, Andrey Bokhanko <[hidden email]> wrote:
While we are on the topic, let me update on some other things happening in responce to other issues highlighted by Chandler:

I just wanted to say, I am very happy to see progress starting here. =] All of my comments below are meant to help the effort move faster and deal better with the LLVM project.
 
- This library is not being developed as an active part of the LLVM
community, even if it is checked into SVN as part of the LLVM project and
under its license. See r197914 where there is a code drop of many months
worth of development with *no* change log, attribution, information, or
other participation in any part of the community.

This is changing, and many developers joining the whole OpenMP in clang support effort. I can say that Michael Wong and many of his colleagues from IBM are involved; Eric Stotzer and his colleagues from TI are involved; Barbara Chapman and her group from UofHouston is involved; Guansong Zhang from AMD is involved. Obviously, Hal Finkel, Chris Bergstrom, Steve Noonan and many others continue to be actively involved as well.

Take a look at the recent activity in openmp-dev mailing list. More to come.

Pretty happy about this, and looking forward to more. Some things that would be helpful (IMO, others may disagree) to start looking into how/when to adopt or integrate:

- Start more closely following the LLVM developer policy around code review and small, incremental patches and development.
  - Maybe use code review tools like Phabricator? Dunno what would be needed to make this work well, probably some stuff...
- Check that the codebase compiles with the same host toolchain baseline as the rest of the LLVM project[1]
- Maybe work out a plan toward generally conforming with the coding standards[2] where applicable?
 

It is buildable with clang now. Moreover, regular buildbots, with both gcc and clang, are running:

http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian

Just want to say, this in particular is fantastic.


- The build system is totally disjoint from LLVM's, in fact it is an
entirely custom Perl build system that is unlike anything in use by the
LLVM project.

We started to work on improving build system. Stay tuned.

Very cool. I would look closely at the compiler-rt and libc++ CMake builds. Hopefully useful.


- There are *zero tests* in the open source repository!!!! This is even
called out in the original submission and on the primary website. We simply
*cannot* ship and link against a runtime library which has no tests!

University of Houston contributed OpenUH test suite: http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html. Sunita Chandrasekaran from the University works on integrating this suite into LLVM test system.

BTW, any advice with how to approach this would be *much* appreciated!

I think the right way to go is something along the lines of the ASan (in compiler-rt) lit tests, but maybe others have a better idea. I agree that testing here is hard.


- No part of this library has gone through an LLVM release process either,
not even as a "new" or "beta" project.

Aha! So, you support inclusion of openmp (library, not compiler) in 3.5 as well? :-)

Hehe, I see what you did there.

I do genuinely support it going through the release process, but I think that there are two things that need to be pretty solid first:

1) the build system work probably needs to be pretty solid. i'd be happy with support on par with compiler-rt or libc++ in CMake...
2) the test suite needs to be at least reasonably well integrated so release testers actually hit it and exercise the code
3) we need a good switch in Clang to use iomp (I know you or someone on the patch thread worked on this, did it land? if so, done!)

Maybe others have other thoughts, but I think those three are my key ones.

Anyways, thanks for the clarification and all the hard work on this stuff.


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: moving the clang-omp merge along

Andrey Bokhanko
In reply to this post by Alp Toker
Alp,

On Thu, May 29, 2014 at 5:01 AM, Alp Toker <[hidden email]> wrote:

If anything it would have been nice to have discussed the overall design of the OMP frontend earlier in development. The fact the patches are landing broadly as written in order to keep it moving is already something of a concession :-)

Andrey


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev