Re: Seeking committers' help to review on SS PR
I don't see any objections on that thread. You're a committer and have reviews from other knowledgeable people in this area. Do you have any reason to believe it's controversial, like, changes semantics or APIs? Were there related discussions elsewhere that expressed any concern? >From a glance, OK it's introducing a new idea of state schema and validation; would it conflict with any other possible approaches, have any limits if this is enshrined as supported functionality? There's always some cost to introducing yet more code to support, but, this doesn't look intrusive or large. The "don't review your own PR" idea isn't hard-and-fast. I don't think anyone needs to block for anything like this long if you have other capable reviews and you are a committer, if you don't see that it impacts other code meaningfully in a way that really demands review from others, and in good faith judge that it is worthwhile. I think you are the one de facto expert on that code and indeed you can't block yourself for 1.5 years or else nothing substantial would happen. On Mon, Nov 23, 2020 at 1:18 AM Jungtaek Lim wrote: > Hi devs, > > I have been struggling to find reviewers who are committers, to get my PR > [1] for SPARK-27237 [2] reviewed. The PR was submitted on Mar. 2019 (1.5 > years ago), and somehow it got two approvals from contributors working on > the SS area, but still doesn't get any committers' traction to review. > (I can review others' SS PRs and I'm trying to unblock other SS area > contributors, but I can't self review my SS PRs. Not sure it's technically > possible, but fully sure it's not encouraged.) > > Could I please ask help to unblock this before feature freeze for Spark > 3.1 is happening? Submitted 1.5 years ago and continues struggling for > including it in Spark 3.2 (another half of a year) doesn't make sense to me. > > In addition, is there a way to unblock me to work for meaningful features > instead of being stuck with small improvements? I have something in my > backlog but I'd rather not want to continue struggling with new PRs. > > Thanks, > Jungtaek Lim (HeartSaVioR) > > 1. https://github.com/apache/spark/pull/24173 > 2. https://issues.apache.org/jira/browse/SPARK-27237 >
Re: Seeking committers' help to review on SS PR
I'll go take a look. While I would generally agree with Sean that it would be appropriate in this case to commit, I'm very hesitant to set that precedent. I'd prefer to stick with "review then commit" and, if needed, relax that constraint for parts of the project that can't get reviewers for a certain period of time. We did that in another community where there weren't many reviewers and we wanted to get more people involved, but we put a time limit on it and set expectations to prevent any perception of abuse. I would support doing that in SS. Thanks for being so patient on that PR. I'm sorry that you had to wait so long. On Mon, Nov 23, 2020 at 7:11 AM Sean Owen wrote: > I don't see any objections on that thread. You're a committer and have > reviews from other knowledgeable people in this area. Do you have any > reason to believe it's controversial, like, changes semantics or APIs? Were > there related discussions elsewhere that expressed any concern? > > From a glance, OK it's introducing a new idea of state schema and > validation; would it conflict with any other possible approaches, have any > limits if this is enshrined as supported functionality? There's always some > cost to introducing yet more code to support, but, this doesn't look > intrusive or large. > > The "don't review your own PR" idea isn't hard-and-fast. I don't think > anyone needs to block for anything like this long if you have other capable > reviews and you are a committer, if you don't see that it impacts other > code meaningfully in a way that really demands review from others, and in > good faith judge that it is worthwhile. I think you are the one de facto > expert on that code and indeed you can't block yourself for 1.5 years or > else nothing substantial would happen. > > > > On Mon, Nov 23, 2020 at 1:18 AM Jungtaek Lim > wrote: > >> Hi devs, >> >> I have been struggling to find reviewers who are committers, to get my PR >> [1] for SPARK-27237 [2] reviewed. The PR was submitted on Mar. 2019 (1.5 >> years ago), and somehow it got two approvals from contributors working on >> the SS area, but still doesn't get any committers' traction to review. >> (I can review others' SS PRs and I'm trying to unblock other SS area >> contributors, but I can't self review my SS PRs. Not sure it's technically >> possible, but fully sure it's not encouraged.) >> >> Could I please ask help to unblock this before feature freeze for Spark >> 3.1 is happening? Submitted 1.5 years ago and continues struggling for >> including it in Spark 3.2 (another half of a year) doesn't make sense to me. >> >> In addition, is there a way to unblock me to work for meaningful features >> instead of being stuck with small improvements? I have something in my >> backlog but I'd rather not want to continue struggling with new PRs. >> >> Thanks, >> Jungtaek Lim (HeartSaVioR) >> >> 1. https://github.com/apache/spark/pull/24173 >> 2. https://issues.apache.org/jira/browse/SPARK-27237 >> > -- Ryan Blue Software Engineer Netflix
Re: Seeking committers' help to review on SS PR
Yes, agree, and that time limit is probably a lot shorter than 1.5 years. I think these ultimately come down to judgment, and am affirming the judgment that this amounts to 'reviewed'. On Mon, Nov 23, 2020 at 11:40 AM Ryan Blue wrote: > I'll go take a look. > > While I would generally agree with Sean that it would be appropriate in > this case to commit, I'm very hesitant to set that precedent. I'd prefer to > stick with "review then commit" and, if needed, relax that constraint for > parts of the project that can't get reviewers for a certain period of time. > We did that in another community where there weren't many reviewers and we > wanted to get more people involved, but we put a time limit on it and set > expectations to prevent any perception of abuse. I would support doing that > in SS. > > Thanks for being so patient on that PR. I'm sorry that you had to wait so > long. > > On Mon, Nov 23, 2020 at 7:11 AM Sean Owen wrote: > >> I don't see any objections on that thread. You're a committer and have >> reviews from other knowledgeable people in this area. Do you have any >> reason to believe it's controversial, like, changes semantics or APIs? Were >> there related discussions elsewhere that expressed any concern? >> >> From a glance, OK it's introducing a new idea of state schema and >> validation; would it conflict with any other possible approaches, have any >> limits if this is enshrined as supported functionality? There's always some >> cost to introducing yet more code to support, but, this doesn't look >> intrusive or large. >> >> The "don't review your own PR" idea isn't hard-and-fast. I don't think >> anyone needs to block for anything like this long if you have other capable >> reviews and you are a committer, if you don't see that it impacts other >> code meaningfully in a way that really demands review from others, and in >> good faith judge that it is worthwhile. I think you are the one de facto >> expert on that code and indeed you can't block yourself for 1.5 years or >> else nothing substantial would happen. >> >> >> >> On Mon, Nov 23, 2020 at 1:18 AM Jungtaek Lim < >> kabhwan.opensou...@gmail.com> wrote: >> >>> Hi devs, >>> >>> I have been struggling to find reviewers who are committers, to get my >>> PR [1] for SPARK-27237 [2] reviewed. The PR was submitted on Mar. 2019 (1.5 >>> years ago), and somehow it got two approvals from contributors working on >>> the SS area, but still doesn't get any committers' traction to review. >>> (I can review others' SS PRs and I'm trying to unblock other SS area >>> contributors, but I can't self review my SS PRs. Not sure it's technically >>> possible, but fully sure it's not encouraged.) >>> >>> Could I please ask help to unblock this before feature freeze for Spark >>> 3.1 is happening? Submitted 1.5 years ago and continues struggling for >>> including it in Spark 3.2 (another half of a year) doesn't make sense to me. >>> >>> In addition, is there a way to unblock me to work for meaningful >>> features instead of being stuck with small improvements? I have something >>> in my backlog but I'd rather not want to continue struggling with new PRs. >>> >>> Thanks, >>> Jungtaek Lim (HeartSaVioR) >>> >>> 1. https://github.com/apache/spark/pull/24173 >>> 2. https://issues.apache.org/jira/browse/SPARK-27237 >>> >> > > -- > Ryan Blue > Software Engineer > Netflix >
Re: jenkins downtime tomorrow evening/weekend
the third most terrifying event in the world, a massive jenkins plugin update is happening in a couple of hours. i'm going to restart jenkins and start working out any bugs/issues that pop up. this could be short, or quite long. i'm guessing somewhere in the middle. no new builds will be kicked off starting now. in parallel, i'm about to start porting my ansible to ubuntu 20 and testing that on two freshly reinstalled workers. the ultimate goal is to get the PRB running on ubuntu 20... the sbt tests will also likely be broken as i've never been able to work on ubuntu 16, 18 or 20. shane On Sat, Nov 21, 2020 at 4:23 PM shane knapp ☠ wrote: > somehow that went pretty smoothly, tho i've got a bunch of plugins to deal > with... we're back up and building w/a shiny new UI. :) > > On Sat, Nov 21, 2020 at 3:52 PM shane knapp ☠ wrote: > >> this is starting now >> >> On Thu, Nov 19, 2020 at 4:34 PM shane knapp ☠ >> wrote: >> >>> i'm going to be upgrading jenkins to something more reasonable, and >>> there will definitely be some downtime as i get things sorted. >>> >>> we should be back up and building by monday. >>> >>> shane >>> -- >>> Shane Knapp >>> Computer Guy / Voice of Reason >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>> https://rise.cs.berkeley.edu >>> >> >> >> -- >> Shane Knapp >> Computer Guy / Voice of Reason >> UC Berkeley EECS Research / RISELab Staff Technical Lead >> https://rise.cs.berkeley.edu >> > > > -- > Shane Knapp > Computer Guy / Voice of Reason > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu > -- Shane Knapp Computer Guy / Voice of Reason UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
Re: jenkins downtime tomorrow evening/weekend
Thank you, Shane! On Mon, Nov 23, 2020 at 2:12 PM shane knapp ☠ wrote: > the third most terrifying event in the world, a massive jenkins plugin > update is happening in a couple of hours. i'm going to restart jenkins and > start working out any bugs/issues that pop up. > > this could be short, or quite long. i'm guessing somewhere in the > middle. no new builds will be kicked off starting now. > > in parallel, i'm about to start porting my ansible to ubuntu 20 and > testing that on two freshly reinstalled workers. the ultimate goal is to > get the PRB running on ubuntu 20... the sbt tests will also likely be > broken as i've never been able to work on ubuntu 16, 18 or 20. > > shane > > On Sat, Nov 21, 2020 at 4:23 PM shane knapp ☠ wrote: > >> somehow that went pretty smoothly, tho i've got a bunch of plugins to >> deal with... we're back up and building w/a shiny new UI. :) >> >> On Sat, Nov 21, 2020 at 3:52 PM shane knapp ☠ >> wrote: >> >>> this is starting now >>> >>> On Thu, Nov 19, 2020 at 4:34 PM shane knapp ☠ >>> wrote: >>> i'm going to be upgrading jenkins to something more reasonable, and there will definitely be some downtime as i get things sorted. we should be back up and building by monday. shane -- Shane Knapp Computer Guy / Voice of Reason UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu >>> >>> >>> -- >>> Shane Knapp >>> Computer Guy / Voice of Reason >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>> https://rise.cs.berkeley.edu >>> >> >> >> -- >> Shane Knapp >> Computer Guy / Voice of Reason >> UC Berkeley EECS Research / RISELab Staff Technical Lead >> https://rise.cs.berkeley.edu >> > > > -- > Shane Knapp > Computer Guy / Voice of Reason > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu > --
Re: jenkins downtime tomorrow evening/weekend
it seems that the plugin upgrade went as smoothly as it could have... i still have a bunch of stack traces to filter through and see if anything is really broken but it's looking pretty good and things are building. if you see any bad behavior from jenkins, don't hesitate to file a jira and ping me here. also, my backlog of things i need to install will be addressed this week. the ansible is coming along nicely! On Mon, Nov 23, 2020 at 2:11 PM shane knapp ☠ wrote: > the third most terrifying event in the world, a massive jenkins plugin > update is happening in a couple of hours. i'm going to restart jenkins and > start working out any bugs/issues that pop up. > > this could be short, or quite long. i'm guessing somewhere in the > middle. no new builds will be kicked off starting now. > > in parallel, i'm about to start porting my ansible to ubuntu 20 and > testing that on two freshly reinstalled workers. the ultimate goal is to > get the PRB running on ubuntu 20... the sbt tests will also likely be > broken as i've never been able to work on ubuntu 16, 18 or 20. > > shane > > On Sat, Nov 21, 2020 at 4:23 PM shane knapp ☠ wrote: > >> somehow that went pretty smoothly, tho i've got a bunch of plugins to >> deal with... we're back up and building w/a shiny new UI. :) >> >> On Sat, Nov 21, 2020 at 3:52 PM shane knapp ☠ >> wrote: >> >>> this is starting now >>> >>> On Thu, Nov 19, 2020 at 4:34 PM shane knapp ☠ >>> wrote: >>> i'm going to be upgrading jenkins to something more reasonable, and there will definitely be some downtime as i get things sorted. we should be back up and building by monday. shane -- Shane Knapp Computer Guy / Voice of Reason UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu >>> >>> >>> -- >>> Shane Knapp >>> Computer Guy / Voice of Reason >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>> https://rise.cs.berkeley.edu >>> >> >> >> -- >> Shane Knapp >> Computer Guy / Voice of Reason >> UC Berkeley EECS Research / RISELab Staff Technical Lead >> https://rise.cs.berkeley.edu >> > > > -- > Shane Knapp > Computer Guy / Voice of Reason > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu > -- Shane Knapp Computer Guy / Voice of Reason UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
