On 4/22/2010 7:24 AM, Ross Gardler wrote:
I'd like to make a few comments about issues that have arisen during
the evaluation process for GSoC. I'm going to give my opinion on each,
please treat this as lazy consensus - do speak up if you wish to
disagree or add more items:
Thank's Ross,
A few comments inline.
Not enough visibility of the process
====================================
There will always be someone who doesn't read the stuff we send out.
Where this is the case I don't think we should worry ourselves.
Sending to PMCs (including the incubator PMC) is sufficient to reach
people. We don't want to send out to committers@ as the project as a
whole needs to be behind taking on a GSoC student.
Reaching PPMCs is more problematic, I think we should continue to rely
on incubator mentors taking the message to their projects if they feel
it is appropriate.
PROPOSAL
--------
Make it explicit that incubator mentors should pass the message on to
PPMCs if appropriate.
Marking experience mentors up
=============================
I really don't like the idea up to 2 points for having been a
successful mentor before, firstly it is error prone (e.g. both
Bertrand and Luciano have been mentors *and* admins, yet the admin
this year was unaware of that). Secondly, just because someone has
mentored a student in the past doesn't mean they will be better than
another mentor. Finally, mentoring a failing student is, in many ways,
more educational than mentoring a successful one.
We already have "Does the mentor show an understanding of how to
mentor a student? (0-4 points)" - I'm more interested in whether the
mentor knows what is expected. However, applying this score is difficult.
I think we should also have part of the process where the mentor puts in
a zero point comment with their evidence that they do understand how to
mentor a student. It will give the admins something to reference as
they probably have no personal understanding of the the mentor's
understanding or experience.
PROPOSAL
--------
Remove "Has the mentor had a successful student in the past (0-2
points)" from the admin rankings
Add some docs to the ranking process about what admins are looking for
with respect to "oes the mentor show an understanding of how to mentor
a student? (0-4 points)" (Noirins mail with the subject Admin
coordination to alexei.fedotov on code-awards wouild be a good
starting point)
+1 Two points is a lot and might discourage new mentors to engage in the
program.
Original Ideas are good
=======================
Past experience has shown that if a student proposes their own idea
and it is accepted the student is going to be strong.
PROPOSAL
--------
Add the following to the mentor ranking:
Is the project definition and idea originally the mentee's, the
mentor's or a collaborative effort? (0-2 points, 2 if mentee's idea, 1
if collaborative, 0 if mentor's)
I think this one is tricky. In a standards based product like Derby
there is not that much room for creativity in the initial "idea" and
projects are mostly based on existing Jira entries. Also really
everything is collaborative in the community and that should be
encouraged. I think I'd rather see this one left out.
Additional items -
1) I think 2 points for a live (non-email) interview is too much.
Opposing timezones can make this hard to coordinate and I think skills
in written communication is what is key to success. If we have to keep
it, make it clear that IRC is also ok for full points.
2) I think uneven ranking is still something of an issue, but I don't
know how to address it without putting a lot more work on the admins to
review the mentor ranking. I think Alexei's 0 point comments for
suggested adjustments or more information from students were good, but
if we all did that, it would mean a lot of comments for the admins to
review and negotiation that would take more time than we have.
And just a couple doc issues ...
3) Since the ranking process is public (and I am glad that it is), we
should advise students and mentors to look at it carefully and make sure
their proposal meets the criteria to the best of their ability.
4) Mentors should be encouraged to monitor their student's application
during the critical ranking period to spot any mistakes or injustice and
speak up at the time.
Kathey
