Will – What do you think? With only 26 presentations, do you think it would be reasonable to just ask each reviewer to review each one? One time that I was on one of these panels a couple years ago, we each reviewed the roughly dozen presentations that were submitted. Of course, people may not be able to spend that amount of time on this.
> On Apr 5, 2018, at 7:14 PM, Ron Wheeler <rwhee...@artifact-software.com> > wrote: > > We still need to manage the review process and make sure that it is > adequately staffed. > > The allocation of presentations to reviewers has to be managed to be sure > that the reviewers have the support that they need to do a proper review and > that the reviews get done. > > Ron > > >> On 05/04/2018 11:45 AM, Tutkowski, Mike wrote: >> Perfect…then, unless anyone has other opinions they’d like to share on the >> topic, let’s follow that approach. >> >> On 4/5/18, 9:43 AM, "Rafael Weingärtner" <rafaelweingart...@gmail.com> wrote: >> >> That is exactly it. >> On Thu, Apr 5, 2018 at 12:37 PM, Tutkowski, Mike >> <mike.tutkow...@netapp.com> >> wrote: >> > Hi Rafael, >> > >> > I think as long as we (the CloudStack Community) have the final say on >> how >> > we fill our allotted slots in the CloudStack track of ApacheCon in >> > Montreal, then it’s perfectly fine for us to leverage Apache’s normal >> > review process to gather all the feedback from the larger Apache >> Community. >> > >> > As you say, we could wait for the feedback to come in via that >> mechanism >> > and then, as per Will’s earlier comments, we could advertise on our >> users@ >> > and dev@ mailing lists when we plan to get together for a call and make >> > final decisions on the CFP. >> > >> > Is that, in fact, what you were thinking, Rafael? >> > >> > Talk to you soon, >> > Mike >> > >> > On 4/4/18, 2:58 PM, "Rafael Weingärtner" <rafaelweingart...@gmail.com> >> > wrote: >> > >> > I think everybody that “raised their hands here” already signed up >> to >> > review. >> > >> > Mike, what about if we only gathered the reviews from Apache main >> > review >> > system, and then we use that to decide which presentations will >> get in >> > CloudStack tracks? Then, we reduce the work on our side (we also >> remove >> > bias…). I do believe that the review from other peers from Apache >> > community >> > (even the one outside from our small community) will be fair and >> > technical >> > (meaning, without passion and or favoritism). >> > >> > Having said that, I think we only need a small group of PMCs to >> gather >> > the >> > results and out of the best ranked proposals, we pick the ones to >> our >> > tracks. >> > >> > What do you (Mike) and others think? >> > >> > >> > On Tue, Apr 3, 2018 at 5:07 PM, Tutkowski, Mike < >> > mike.tutkow...@netapp.com> >> > wrote: >> > >> > > Hi Ron, >> > > >> > > I don’t actually have insight into how many people have currently >> > signed >> > > up online to be CFP reviewers for ApacheCon. At present, I’m only >> > aware of >> > > those who have responded to this e-mail chain. >> > > >> > > We should be able to find out more in the coming weeks. We’re >> still >> > quite >> > > early in the process. >> > > >> > > Thanks for your feedback, >> > > Mike >> > > >> > > On 4/1/18, 9:18 AM, "Ron Wheeler" >> <rwhee...@artifact-software.com> >> > wrote: >> > > >> > > How many people have signed up to be reviewers? >> > > >> > > I don't think that scheduling is part of the review process >> and >> > that >> > > can >> > > be done by the person/team "organizing" ApacheCon on behalf >> of >> > the PMC. >> > > >> > > To me review is looking at content for >> > > - relevance >> > > - quality of the presentations (suggest fixes to content, >> > English, >> > > graphics, etc.) >> > > This should result in a consensus score >> > > - Perfect - ready for prime time >> > > - Needs minor changes as documented by the reviewers >> > > - Great topic but needs more work - perhaps a reviewer could >> > volunteer >> > > to work with the presenter to get it ready if chosen >> > > - Not recommended for topic or content reasons >> > > >> > > The reviewers could also make non-binding recommendations >> about >> > the >> > > balance between topics - marketing(why Cloudstack), >> > > Operations/implementation, Technical details, Roadmap, etc. >> > based on >> > > what they have seen. >> > > >> > > This should be used by the organizers to make the choices and >> > organize >> > > the program. >> > > The organizers have the final say on the choice of >> presentations >> > and >> > > schedule >> > > >> > > Reviewers are there to help the process not control it. >> > > >> > > I would be worried that you do not have enough reviewers >> rather >> > than >> > > too >> > > many. >> > > Then the work falls on the PMC and organizers. >> > > >> > > When planning meetings, I would recommend that you clearly >> > separate the >> > > roles and only invite the reviewers to the meetings about >> > review. Get >> > > the list of presentation to present to the reviewers and >> decide >> > if >> > > there >> > > are any instructions that you want to give to reviewers. >> > > I would recommend that you keep the organizing group small. >> > Membership >> > > should be set by the PMC and should be people that are >> committed >> > to the >> > > ApacheCon project and have the time. The committee can >> request >> > help for >> > > specific tasks from others in the community who are not on >> the >> > > committee. >> > > >> > > I would also recommend that organizers do not do reviews. >> They >> > should >> > > read the finalists but if they do reviews, there may be a >> > suggestion of >> > > favouring presentations that they reviewed. It also ensures >> that >> > the >> > > organizers are not getting heat from rejected presenters - >> "it >> > is the >> > > reviewers fault you did not get selected". >> > > >> > > My advice is to get as many reviewers as you can so that no >> one >> > is >> > > essential and each reviewer has a limited number of >> > presentations to >> > > review but each presentation gets reviewed by multiple >> people. >> > Also >> > > bear >> > > in mind that not all reviewers have the same ability to >> review >> > each >> > > presentation. >> > > Reviews should be anonymous and only the summary comments >> given >> > to the >> > > presenter. Reviewers of a presentation should be able to >> discuss >> > the >> > > presentation during the review to make sure that reviewers do >> > not feel >> > > isolated or get lost when they hit content that they don't >> > understand >> > > fully. >> > > >> > > >> > > >> > > Ron >> > > >> > > >> > > On 01/04/2018 12:20 AM, Tutkowski, Mike wrote: >> > > > Thanks for the feedback, Will! >> > > > >> > > > I agree with the approach you outlined. >> > > > >> > > > Thanks for being so involved in the process! Let’s chat >> with >> > Giles >> > > once he’s back to see if we can get your questions answered. >> > > > >> > > >> On Mar 31, 2018, at 10:14 PM, Will Stevens < >> > wstev...@cloudops.com> >> > > wrote: >> > > >> >> > > >> In the past the committee was chosen as a relatively small >> > group in >> > > order >> > > >> to make it easier to manage feedback. In order to make it >> > fair to >> > > everyone >> > > >> in the community, I would suggest that instead of doing it >> > with a >> > > small >> > > >> group, we do it out in the open on a scheduled call. >> > > >> >> > > >> We will have to get a list of the talks that are >> CloudStack >> > > specific from >> > > >> ApacheCon, but that should be possible. >> > > >> >> > > >> Once we have the talks selected, then a smaller number of >> us >> > can >> > > work on >> > > >> setting up the actual ordering and the details. >> > > >> >> > > >> I have been quite involved so far. Giles and I have been >> > > organizing the >> > > >> sponsors, website and dealing with ApacheCon so far. >> > Obviously, >> > > Mike is >> > > >> also working on this as well. >> > > >> >> > > >> I think we are headed in the right direction on this. >> > > >> >> > > >> Cheers, >> > > >> >> > > >> Will >> > > >> >> > > >> On Mar 31, 2018 11:49 PM, "Tutkowski, Mike" < >> > > mike.tutkow...@netapp.com> >> > > >> wrote: >> > > >> >> > > >> Hi Ron, >> > > >> >> > > >> I am definitely open to working this however makes the >> most >> > sense. >> > > >> >> > > >> It looks like Will’s e-mail indicates that the process I >> > suggested >> > > has been >> > > >> followed in the past (which is how I recall, as well). >> > > >> >> > > >> Let’s make sure I understood Will correctly. >> > > >> >> > > >> Will – Are you, in fact, indicating that what I was >> > suggesting is >> > > how we >> > > >> have reviewed the CFP in the past? If so, are you able to >> > address >> > > Ron’s >> > > >> concerns? >> > > >> >> > > >> Also, Will – I am not sure about a hackathon. Let’s chat >> with >> > Giles >> > > once >> > > >> he’s back from vacation since he’s been the most involved >> with >> > > organizing >> > > >> the CloudStack track within ApacheCon. >> > > >> >> > > >> Thanks! >> > > >> >> > > >> Mike >> > > >> >> > > >> >> > > >> On 3/31/18, 9:00 PM, "Ron Wheeler" < >> > rwhee...@artifact-software.com> >> > > wrote: >> > > >> >> > > >> I am not sure about your concern in that case. >> > > >> I am not sure why people not interested in Cloudstack >> > would >> > > volunteer as >> > > >> reviewers and want to pick bad presentations. >> > > >> >> > > >> I would be more worried that there are not enough good >> > > presentations >> > > >> proposed rather than some meritorious presentation >> will >> > get >> > > rejected due >> > > >> to "outsiders" voting it down in favour of less useful >> > > presentations. >> > > >> >> > > >> It may be tricky to get balance if that means taking >> "bad" >> > > proposals >> > > >> that can not be fixed that cover topics that are in >> areas >> > that >> > > are not >> > > >> otherwise covered at the expense of great >> presentations >> > that >> > > are in >> > > >> areas with many choices. >> > > >> >> > > >> We should wait to see how many presentations have to >> be >> > > rejected and the >> > > >> number of reviewers before getting too exercised over >> the >> > > loyalty of >> > > >> reviewers. >> > > >> >> > > >> Getting more reviewers is likely the most effective >> way >> > to see >> > > that a >> > > >> wider range of topics is covered. >> > > >> >> > > >> Ron >> > > >> >> > > >>> On 31/03/2018 7:15 PM, Tutkowski, Mike wrote: >> > > >>> Hi Ron, >> > > >>> >> > > >>> From what I understand, the CloudStack proposals will be >> > mixed in >> > > >> with all of the ApacheCon proposals. >> > > >>> In the past when I’ve participated in these CloudStack >> > panels to >> > > >> review proposals, we had to compare each proposal against >> the >> > > others to >> > > >> arrive at a balance of topics (i.e. not all networking >> > focused, not >> > > all >> > > >> XenServer focused, etc.) and to suggest improvements for >> > proposals >> > > that we >> > > >> did not accept for other reasons. >> > > >>> From what I understand (but Giles can comment further on >> > this), we >> > > >> have a track at ApacheCon and will need to fill it with X >> > number of >> > > >> presentations. To do this, it seems like a >> CloudStack-focused >> > panel >> > > would >> > > >> be a good approach, but I am definitely open to another >> > approach. >> > > We don’t >> > > >> want to exclude anyone (in or out of the CloudStack >> > Community) who >> > > might >> > > >> like to provide input. Anyone who is interested would, of >> > course, >> > > be free >> > > >> to join us in combing through the proposals. >> > > >>> We don’t need to get started on this right away. The CFP >> just >> > > closed >> > > >> yesterday. Let’s wait for feedback from Giles (who is >> > currently on >> > > >> vacation) and go from there. >> > > >>> Thanks! >> > > >>> Mike >> > > >>> >> > > >>> On 3/31/18, 6:59 AM, "Ron Wheeler" < >> > rwhee...@artifact-software.com >> > > > >> > > >> wrote: >> > > >>> Is this a real concern? >> > > >>> Why would a large number of Apache contributors who >> are >> > not >> > > >> interested >> > > >>> in Cloudstack (enough to outvote those "part of the >> > Cloudstack >> > > >>> community") get involved as reviewers >> > > >>> >> > > >>> Reviewing involves some commitment of time so I am >> hard >> > > pressed >> > > >> to guess >> > > >>> why some Apache contributor would volunteer to do >> the >> > work in >> > > >> order to >> > > >>> veto a presentation that they have not yet seen or >> have >> > no >> > > >> interest in >> > > >>> seeing. >> > > >>> >> > > >>> Are we guaranteed a fixed number of hours of >> > presentations or >> > > is >> > > >> the >> > > >>> review process part of the allocation of overall >> time? >> > > >>> >> > > >>> On what basis can some group veto a presentation? >> > > >>> That would seem to be a very strong action and I >> would >> > hope >> > > that >> > > >> it >> > > >>> requires a strong reason. >> > > >>> >> > > >>> OTOH if a large??? number of Apache contributors >> > (regardless >> > > of >> > > >> their >> > > >>> affiliation) say that a presentation has serious >> issues >> > or >> > > very >> > > >> limited >> > > >>> interest, that would seem to be a red flag that the >> > > presentation >> > > >>> requires improvement or needs to be dropped in >> favour of >> > > another >> > > >>> Cloudstack presentation, if it can not be fixed. >> > > >>> >> > > >>> We should also be aware that this is an opportunity >> to >> > > "market" >> > > >>> Cloudstack to the broader Apache community. >> > > >>> Outside reviewers might have valuable input into how >> > > >> presentations can >> > > >>> attract new adopters or be clearer to the broader >> DevOps >> > > >> community. >> > > >>> We also need to remember that we do have an active >> > community >> > > and >> > > >> other >> > > >>> opportunities during the year to present >> presentations >> > that do >> > > >> not get >> > > >>> selected for this conference. >> > > >>> >> > > >>> If their is a real fear that a lot of "outsiders" >> are >> > going to >> > > >> disrupt >> > > >>> the review process, a more reasonable response would >> > seem to >> > > be >> > > >> to get >> > > >>> more reviewers from the community. >> > > >>> >> > > >>> I have volunteered already. >> > > >>> >> > > >>> Ron >> > > >>> >> > > >>>> On 30/03/2018 11:11 PM, Tutkowski, Mike wrote: >> > > >>>> Hi Rafael, >> > > >>>> >> > > >>>> It’s a little bit tricky in our particular situation. >> Allow >> > me >> > > >> to explain: >> > > >>>> As you are likely aware, the CloudStack Collaboration >> > > >> Conference will be held as a track in the larger ApacheCon >> > > conference in >> > > >> Montreal this coming September. >> > > >>>> It is true, as you say, that anyone who wishes to do so >> can >> > > >> contribute to reviewing the CFP for ApacheCon. >> > > >>>> What is a bit of a concern, however, is that we might >> get >> > > >> certain CloudStack CFP proposals vetoed by people who are >> > not, per >> > > se, a >> > > >> part of our community. >> > > >>>> That being the case, I have contacted the organizers for >> > > >> ApacheCon to see if there is some way we can section off >> the >> > > CloudStack CFP >> > > >> from the larger ApacheCon CFP for review purposes. >> > > >>>> Assuming we can do this, the panel that I am proposing >> here >> > > >> would handle this review task. >> > > >>>> I hope that helps clarify the situation. >> > > >>>> >> > > >>>> Thanks! >> > > >>>> Mike >> > > >>>> >> > > >>>> On 3/30/18, 8:38 AM, "Rafael Weingärtner" < >> > > >> rafaelweingart...@gmail.com> wrote: >> > > >>>> Are we going to have a separated review process? >> > > >>>> >> > > >>>> I thought anybody could go here [1] and apply for a >> > > >> reviewer position and >> > > >>>> start reviewing. Well, that is what I did. I have >> > already >> > > >> reviewed some >> > > >>>> CloudStack proposals (of course I did not review >> > mines). >> > > >> After asking to >> > > >>>> review presentations, Rich has giving me access to >> the >> > > >> system. I thought >> > > >>>> everybody interest in helping was going to do the >> same. >> > > >>>> >> > > >>>> [1] >> > > >> https://cfp.apachecon.com/conference.html?apachecon- >> > > north-america-2018 >> > > >>>> >> > > >>>> On Wed, Mar 28, 2018 at 4:05 AM, Swen - swen.io < >> > > >> m...@swen.io> wrote: >> > > >>>>> Hi Mike, >> > > >>>>> >> > > >>>>> congrats! >> > > >>>>> >> > > >>>>> I can help sort through presentations. >> > > >>>>> >> > > >>>>> Best regards, >> > > >>>>> Swen >> > > >>>>> >> > > >>>>> -----Ursprüngliche Nachricht----- >> > > >>>>> Von: Tutkowski, Mike [mailto:mike.tutkow...@netapp.com] >> > > >>>>> Gesendet: Dienstag, 27. März 2018 21:40 >> > > >>>>> An: dev@cloudstack.apache.org; >> > > >> us...@cloudstack.apache.org >> > > >>>>> Betreff: Committee to Sort through CCC Presentation >> > > >> Submissions >> > > >>>>> Hi everyone, >> > > >>>>> >> > > >>>>> As you may be aware, this coming September in Montreal, >> > > >> the CloudStack >> > > >>>>> Community will be hosting the CloudStack Collaboration >> > > >> Conference: >> > > >>>>> http://ca.cloudstackcollab.org/ >> > > >>>>> >> > > >>>>> Even though the event is six months away, we are on a >> > > >> tight schedule with >> > > >>>>> regards to the Call For Participation (CFP): >> > > >>>>> >> > > >>>>> https://www.apachecon.com/acna18/schedule.html >> > > >>>>> >> > > >>>>> If you are interested in submitting a talk, please do >> > > >> so before March 30th. >> > > >>>>> That being said, as usual, we will have need of a small >> > > >> committee to sort >> > > >>>>> through these presentation submissions. >> > > >>>>> >> > > >>>>> If you are interested in helping out in this process, >> > > >> please reply to this >> > > >>>>> message. >> > > >>>>> >> > > >>>>> Thanks! >> > > >>>>> Mike >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>> >> > > >>>> -- >> > > >>>> Rafael Weingärtner >> > > >>>> >> > > >>>> >> > > >>> >> > > >> > > -- >> > > Ron Wheeler >> > > President >> > > Artifact Software Inc >> > > email: rwhee...@artifact-software.com >> > > skype: ronaldmwheeler >> > > phone: 866-970-2435, ext 102 >> > > >> > > >> > > >> > > >> > >> > >> > -- >> > Rafael Weingärtner >> > >> > >> > >> -- >> Rafael Weingärtner >> > > -- > Ron Wheeler > President > Artifact Software Inc > email: rwhee...@artifact-software.com > skype: ronaldmwheeler > phone: 866-970-2435, ext 102 >
