Quantcast

LLVM GSOC Projects Criteria Consultation (before 2/28)

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

LLVM GSOC Projects Criteria Consultation (before 2/28)

Ammarguellat, Zahira via cfe-dev
Hello all,

GSOC is around the corner, and the LLVM projects plans to participate again this year. For those who don’t know about GSOC, students are proposing a project that they will work on for 3 months. Amongst other, one goal for LLVM is to mentor students to become good developers and also contributors to the LLVM project (or user/advocate of LLVM for building other cool projects).

A key part of the process is about how do we select the projects. The way we’ve done it last year, is that the volunteer mentors shared an email thread and ultimately each one individually ranked the projects. We kept the average grading for each project to ranked them overall.

In order to make the process more transparent to student applicants, we want to formalize and announce the criteria for ranking and selection project.

The purpose of this email is to gather community feedback about the criterion that mentors should apply when ranking projects, for example:

- Should we favor a student which has past contributions to LLVM compared to a newcomer? Is it more important or as equally important as the quality of the proposal?
- How should we rank (if any) “research or unbounded projects” vs “project with an immediate application”? Should we strive to keep a balance?
- What about “projects that touch LLVM/Clang/…” vs “projects that are building something on top of LLVM”? (For example this project was first proposed to be selected by LLVM before being rejected and finally picked by the Julia project directly: https://summerofcode.withgoogle.com/archive/2016/projects/6197080536121344/ )?
- Should we ask that the work is done upstream all along? In the past there have been project developed on GitHub outside the community that have never been merged. The LLVM developer policy has a full section insisting on incremental development ( http://llvm.org/docs/DeveloperPolicy.html#incremental-development ).

Hopefully we should be able to provide a set of guidelines to student that would help them fill the best application, and avoid unfortunate surprise at the end of the process.

We should have this discussion before receiving the projects proposals, which opens on 2/28.

Best,

--
Mehdi


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [lldb-dev] LLVM GSOC Projects Criteria Consultation (before 2/28)

Ammarguellat, Zahira via cfe-dev


On 02/17/2017 12:16 AM, Mehdi Amini via lldb-dev wrote:
Hello all,

GSOC is around the corner, and the LLVM projects plans to participate again this year. For those who don’t know about GSOC, students are proposing a project that they will work on for 3 months. Amongst other, one goal for LLVM is to mentor students to become good developers and also contributors to the LLVM project (or user/advocate of LLVM for building other cool projects).

A key part of the process is about how do we select the projects. The way we’ve done it last year, is that the volunteer mentors shared an email thread and ultimately each one individually ranked the projects. We kept the average grading for each project to ranked them overall.

In order to make the process more transparent to student applicants, we want to formalize and announce the criteria for ranking and selection project.

The purpose of this email is to gather community feedback about the criterion that mentors should apply when ranking projects, for example:

- Should we favor a student which has past contributions to LLVM compared to a newcomer? Is it more important or as equally important as the quality of the proposal?

I think that the quality of the proposal is the most important thing, however, we should be realistic about the amount of time it will take to bring a newcomer up to speed on our coding conventions, quality standards, review process, etc. and factor that into the decision-making process.

- How should we rank (if any) “research or unbounded projects” vs “project with an immediate application”? Should we strive to keep a balance?

Given the short time period involved, I favor projects of more immediate utility. This does not mean that there can't be interesting open questions, but in our judgment, these should be answerable in a matter of weeks (i.e. choosing between clear alternatives, understood tuning parameters, and the like).

- What about “projects that touch LLVM/Clang/…” vs “projects that are building something on top of LLVM”? (For example this project was first proposed to be selected by LLVM before being rejected and finally picked by the Julia project directly: https://summerofcode.withgoogle.com/archive/2016/projects/6197080536121344/ )?

I think we need to base this on the overall benefit to the community. If working on a project built on top of LLVM will bring additional users, end up making LLVM more robust, etc. then there might be significant anticipated value. That having been said, in practice, justifying this value will probably be more difficult than for a project contributing to LLVM, Clang, etc.

- Should we ask that the work is done upstream all along? In the past there have been project developed on GitHub outside the community that have never been merged. The LLVM developer policy has a full section insisting on incremental development ( http://llvm.org/docs/DeveloperPolicy.html#incremental-development ).

I think we should insist on incremental upstream development.

Thanks again,
Hal


Hopefully we should be able to provide a set of guidelines to student that would help them fill the best application, and avoid unfortunate surprise at the end of the process.

We should have this discussion before receiving the projects proposals, which opens on 2/28.

Best,

--
Mehdi


_______________________________________________
lldb-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] LLVM GSOC Projects Criteria Consultation (before 2/28)

Ammarguellat, Zahira via cfe-dev
In reply to this post by Ammarguellat, Zahira via cfe-dev
On Thu, Feb 16, 2017 at 10:16 PM, Mehdi Amini via llvm-dev
<[hidden email]> wrote:

> Hello all,
>
> GSOC is around the corner, and the LLVM projects plans to participate again
> this year. For those who don’t know about GSOC, students are proposing a
> project that they will work on for 3 months. Amongst other, one goal for
> LLVM is to mentor students to become good developers and also contributors
> to the LLVM project (or user/advocate of LLVM for building other cool
> projects).
>
> A key part of the process is about how do we select the projects. The way
> we’ve done it last year, is that the volunteer mentors shared an email
> thread and ultimately each one individually ranked the projects. We kept the
> average grading for each project to ranked them overall.
>
> In order to make the process more transparent to student applicants, we want
> to formalize and announce the criteria for ranking and selection project.
>
> The purpose of this email is to gather community feedback about the
> criterion that mentors should apply when ranking projects, for example:
>
> - Should we favor a student which has past contributions to LLVM compared to
> a newcomer? Is it more important or as equally important as the quality of
> the proposal?

I really think that past contributions are *very* important and *a big plus*.
When I was involved in FreeBSD we ended up with people submitting
high-quality proposals, got accepted and then we realized at day 1 of
coding they didn't even set up a VM or knew how to build things. In
other words, I'm under the impression that students need to be well
aware of the development workflow and being able to show up some
involvement in the community.

> - How should we rank (if any) “research or unbounded projects” vs “project
> with an immediate application”? Should we strive to keep a balance?

Again, my opinion, but we should give high(er) priority to projects
which can produce something reasonable in 3 months.
If it's a research project, it's fine, but I think there should be a
bound (e.g. tune the thresholds for unrolling).

> - What about “projects that touch LLVM/Clang/…” vs “projects that are
> building something on top of LLVM”? (For example this project was first
> proposed to be selected by LLVM before being rejected and finally picked by
> the Julia project directly:
> https://summerofcode.withgoogle.com/archive/2016/projects/6197080536121344/
> )?
> - Should we ask that the work is done upstream all along? In the past there
> have been project developed on GitHub outside the community that have never
> been merged. The LLVM developer policy has a full section insisting on
> incremental development (
> http://llvm.org/docs/DeveloperPolicy.html#incremental-development ).
>

Yes, I think incremental patches are the way to go. If there are
bugfixes, etc.. they should be merged immediately (after code review).
If it's a larger chunk of work it should be in a state where others
can take over if the student stops being interested after the summer
(or we want to remove the thing altogether, e.g. a pass).

> Hopefully we should be able to provide a set of guidelines to student that
> would help them fill the best application, and avoid unfortunate surprise at
> the end of the process.
>
> We should have this discussion before receiving the projects proposals,
> which opens on 2/28.
>

--
Davide

"There are no solved problems; there are only problems that are more
or less solved" -- Henri Poincare
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [lldb-dev] LLVM GSOC Projects Criteria Consultation (before 2/28)

Ammarguellat, Zahira via cfe-dev
In reply to this post by Ammarguellat, Zahira via cfe-dev
On 2/17/17 1:16 AM, Mehdi Amini via lldb-dev wrote:
Hello all,

GSOC is around the corner, and the LLVM projects plans to participate again this year. For those who don’t know about GSOC, students are proposing a project that they will work on for 3 months. Amongst other, one goal for LLVM is to mentor students to become good developers and also contributors to the LLVM project (or user/advocate of LLVM for building other cool projects).

A key part of the process is about how do we select the projects. The way we’ve done it last year, is that the volunteer mentors shared an email thread and ultimately each one individually ranked the projects. We kept the average grading for each project to ranked them overall.

In order to make the process more transparent to student applicants, we want to formalize and announce the criteria for ranking and selection project.

The purpose of this email is to gather community feedback about the criterion that mentors should apply when ranking projects, for example:

- Should we favor a student which has past contributions to LLVM compared to a newcomer? Is it more important or as equally important as the quality of the proposal?

Historically, we have opted for students with previous LLVM experience because they are more likely to make progress and complete their project.  For most projects, this makes sense.  However, it does have the downside of attracting students that are already "on board" with LLVM; it doesn't introduce new developers to the LLVM community.

We therefore might want to devise projects that would fit beginner developers.  For example, a proposal that aims to fix a set of bugs might work.

- How should we rank (if any) “research or unbounded projects” vs “project with an immediate application”? Should we strive to keep a balance?

We should strive to keep a balance.  The important factor is getting students to either improve the existing LLVM software or to build cool tools using LLVM.  We want to grow the LLVM community, and both the software development and research communities are important.

I also believe that research projects and open-ended projects can provide long-term gains even if they don't provide immediate short-term benefits.  Remember that LLVM itself was once a research project.

If we keep a balance, then we will have a nice mix of short-term, low risk projects and projects that contribute to longer-term, higher risk goals.



- What about “projects that touch LLVM/Clang/…” vs “projects that are building something on top of LLVM”? (For example this project was first proposed to be selected by LLVM before being rejected and finally picked by the Julia project directly: https://summerofcode.withgoogle.com/archive/2016/projects/6197080536121344/ )?

We should encourage proposals that build something *with* LLVM/Clang/etc. as well as projects that improve the core compiler infrastructure.  Part of LLVM's appeal is that one can build *a lot* of tools with it.  We want to encourage students to use LLVM to build interesting things.

In some cases, we will have overlap with other communities (the Julia and FreeBSD communities come to mind).  I think that's fine.


- Should we ask that the work is done upstream all along? In the past there have been project developed on GitHub outside the community that have never been merged. The LLVM developer policy has a full section insisting on incremental development ( http://llvm.org/docs/DeveloperPolicy.html#incremental-development ).

It depends on the project.  Projects that enhance the core infrastructure should probably follow the developer policy and contribute upstream.  Projects that are "trying out" a new idea or are research-oriented should probably not worry about working upstream; why upstream if you don't even know if what you're trying to do is going to work?


Hopefully we should be able to provide a set of guidelines to student that would help them fill the best application, and avoid unfortunate surprise at the end of the process.

I think what actually makes a strong proposal is whether the student presents a good idea, whether the proposal convinces the reader that the student will be able to perform the work, and whether there is a mentor interested in the project.  If you have these three things, then you have a project that is more likely to achieve its goals and grow the LLVM community.

Regards,

John Criswell

-- 
John Criswell
Assistant Professor
Department of Computer Science, University of Rochester
http://www.cs.rochester.edu/u/criswell

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] LLVM GSOC Projects Criteria Consultation (before 2/28)

Ammarguellat, Zahira via cfe-dev
In reply to this post by Ammarguellat, Zahira via cfe-dev

On Feb 16, 2017, at 10:16 PM, Mehdi Amini via llvm-dev <[hidden email]> wrote:

Hello all,

GSOC is around the corner, and the LLVM projects plans to participate again this year. For those who don’t know about GSOC, students are proposing a project that they will work on for 3 months. Amongst other, one goal for LLVM is to mentor students to become good developers and also contributors to the LLVM project (or user/advocate of LLVM for building other cool projects).

A key part of the process is about how do we select the projects. The way we’ve done it last year, is that the volunteer mentors shared an email thread and ultimately each one individually ranked the projects. We kept the average grading for each project to ranked them overall.

In order to make the process more transparent to student applicants, we want to formalize and announce the criteria for ranking and selection project.

The purpose of this email is to gather community feedback about the criterion that mentors should apply when ranking projects, for example:

- Should we favor a student which has past contributions to LLVM compared to a newcomer? Is it more important or as equally important as the quality of the proposal?

While students with previous LLVM dev experience will be more productive, GSoC is a great way of attracting newcomers to the project! So if a proposal is scoped to be completed in time even by someone not familiar with LLVM, we should still accept it.

- How should we rank (if any) “research or unbounded projects” vs “project with an immediate application”? Should we strive to keep a balance?

My preference would be to give priority to projects that are on a trajectory to have practical benefit. 

- What about “projects that touch LLVM/Clang/…” vs “projects that are building something on top of LLVM”? (For example this project was first proposed to be selected by LLVM before being rejected and finally picked by the Julia project directly: https://summerofcode.withgoogle.com/archive/2016/projects/6197080536121344/ )?
- Should we ask that the work is done upstream all along? In the past there have been project developed on GitHub outside the community that have never been merged. The LLVM developer policy has a full section insisting on incremental development ( http://llvm.org/docs/DeveloperPolicy.html#incremental-development ).

An invaluable part of participating in GSoC is for students to learn about how open source projects work. Following the developer policy is one of the main aspects of development in these codebases. In addition, if a project is being developed on a branch it is likely not to get merged and the impact of the student, the mentor, as well as the GSoC budget will be minimal.

Hopefully we should be able to provide a set of guidelines to student that would help them fill the best application, and avoid unfortunate surprise at the end of the process.

We should have this discussion before receiving the projects proposals, which opens on 2/28.

Best,

--
Mehdi

_______________________________________________
LLVM Developers mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] LLVM GSOC Projects Criteria Consultation (before 2/28)

Ammarguellat, Zahira via cfe-dev
Agreed. I think it's worth thinking about what GSoC is actually about,
which according to the website: "Google Summer of Code is a global
program focused on introducing students to open source software
development."

Strongly favouring students with prior experience in community
involvement with LLVM is somewhat missing the point, and I think we
should be mindful of the differences in opportunities that some
students have had.

Amara

On 18 February 2017 at 02:10, Anna Zaks via llvm-dev
<[hidden email]> wrote:

>
> On Feb 16, 2017, at 10:16 PM, Mehdi Amini via llvm-dev
> <[hidden email]> wrote:
>
> Hello all,
>
> GSOC is around the corner, and the LLVM projects plans to participate again
> this year. For those who don’t know about GSOC, students are proposing a
> project that they will work on for 3 months. Amongst other, one goal for
> LLVM is to mentor students to become good developers and also contributors
> to the LLVM project (or user/advocate of LLVM for building other cool
> projects).
>
> A key part of the process is about how do we select the projects. The way
> we’ve done it last year, is that the volunteer mentors shared an email
> thread and ultimately each one individually ranked the projects. We kept the
> average grading for each project to ranked them overall.
>
> In order to make the process more transparent to student applicants, we want
> to formalize and announce the criteria for ranking and selection project.
>
> The purpose of this email is to gather community feedback about the
> criterion that mentors should apply when ranking projects, for example:
>
> - Should we favor a student which has past contributions to LLVM compared to
> a newcomer? Is it more important or as equally important as the quality of
> the proposal?
>
>
> While students with previous LLVM dev experience will be more productive,
> GSoC is a great way of attracting newcomers to the project! So if a proposal
> is scoped to be completed in time even by someone not familiar with LLVM, we
> should still accept it.
>
> - How should we rank (if any) “research or unbounded projects” vs “project
> with an immediate application”? Should we strive to keep a balance?
>
>
> My preference would be to give priority to projects that are on a trajectory
> to have practical benefit.
>
> - What about “projects that touch LLVM/Clang/…” vs “projects that are
> building something on top of LLVM”? (For example this project was first
> proposed to be selected by LLVM before being rejected and finally picked by
> the Julia project directly:
> https://summerofcode.withgoogle.com/archive/2016/projects/6197080536121344/
> )?
>
> - Should we ask that the work is done upstream all along? In the past there
> have been project developed on GitHub outside the community that have never
> been merged. The LLVM developer policy has a full section insisting on
> incremental development (
> http://llvm.org/docs/DeveloperPolicy.html#incremental-development ).
>
>
> An invaluable part of participating in GSoC is for students to learn about
> how open source projects work. Following the developer policy is one of the
> main aspects of development in these codebases. In addition, if a project is
> being developed on a branch it is likely not to get merged and the impact of
> the student, the mentor, as well as the GSoC budget will be minimal.
>
> Hopefully we should be able to provide a set of guidelines to student that
> would help them fill the best application, and avoid unfortunate surprise at
> the end of the process.
>
> We should have this discussion before receiving the projects proposals,
> which opens on 2/28.
>
> Best,
>
> --
> Mehdi
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> [hidden email]
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [llvm-dev] LLVM GSOC Projects Criteria Consultation (before 2/28)

Ammarguellat, Zahira via cfe-dev
In reply to this post by Ammarguellat, Zahira via cfe-dev
On 18 Feb 2017, at 02:10, Anna Zaks via llvm-dev <[hidden email]> wrote:
>
> While students with previous LLVM dev experience will be more productive, GSoC is a great way of attracting newcomers to the project! So if a proposal is scoped to be completed in time even by someone not familiar with LLVM, we should still accept it.

I’d also add, from having mentored GSoC projects for several open source projects, that there are two ways of considering success for a GSoC project:

1. The student completes the work and it’s incorporated into trunk.
2. The student becomes engaged with the project and continues to contribute after the end of the GSoC.

The former treats the GSoC as an expensive way of getting a particular feature added (yes, Google pays the student, but generally the mentor time comes from somewhere and that’s not free, it’s time that would otherwise be spent contributing to the project in another way).  The second treats the GSoC as a powerful recruitment tool.  I’ve had a couple of students that succeeded in the first way, but didn’t really benefit the project, because they left at the end of the GSoC.  I’ve also had a few that managed some proof-of-concept code that ended up showing that the entire GSoC project idea was the wrong approach, but then remained active contributors for years and ended up contributing far more (good) code than I’d have written if I’d spent the time that I spent mentoring them on hacking.

I would strongly encourage treating the GSoC as a way of engaging students.  I’d also hope that the Foundation’s Women in Compilers and Tools project will help encourage this kind of participation in a way that meshes well with the GSoC.  

David

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Loading...