Yes, I agree thats its a valid concern and leads to individual contributors giving up on new ideas or major improvements.
On Tue, 26 Feb 2019 at 15:24, Jungtaek Lim <kabh...@gmail.com> wrote: > Adding one more, it implicitly leads individual contributors to give up > with challenging major things and just focus on minor things, which would > even help on project, but not in the long run. We don't have roadmap put > into wall and let whole community share the load together, so individual > contributors have lots of risks on putting major efforts - shouldn't > conflict to what others have been doing privately, should be accepted after > putting numerous effort to design and have POC. > > 2019년 2월 27일 (수) 오전 8:14, Jungtaek Lim <kabh...@gmail.com>님이 작성: > >> Thanks Sean, as always, to share your thought quickly! >> >> I agree most of points, except "they add a lot of code and complexity >> relative to benefit", since no one can weigh on something before at least >> taking quick review. IMHO if someone would think so, better to speak (I >> know it's hard and being a chance to be blamed, but better than giving >> meaningless hope, yeah I admit I might be another one to do that in another >> project) and see how others will weigh, rather than let it put aside and >> ignore. >> >> I guess my target is already simple and targeted since I've only >> mentioned SS area - there're not much committers who can review SS area. >> Thing to consider is, I have PRs in other areas as well, and I don't have >> issue on these areas. The reason of posting it to dev mailing list instead >> of periodically ping in Github PR is, 1) ping in PR just doesn't work 2) >> let others - hopefully PMC members - indicate a lack on activity on SS >> area, and lead some action. >> >> 2019년 2월 27일 (수) 오전 7:57, Sean Owen <sro...@gmail.com>님이 작성: >> >>> Those aren't bad changes, but they add a lot of code and complexity >>> relative to benefit. I think it's positive that you've gotten people >>> to spend time reviewing them, quite a lot. I don't know whether they >>> should be merged. This isn't a 'bug' though; not all changes should be >>> committed. Simple and targeted is much easier to say yes to, because >>> you implicitly here ask a lot of people to assume responsibility for >>> your change. >>> >>> On Tue, Feb 26, 2019 at 4:38 PM Jungtaek Lim <kabh...@gmail.com> wrote: >>> > >>> > Hi devs, >>> > >>> > sorry to bring this again to mailing list, but you know, ping in >>> Github PR just doesn't work. >>> > >>> > I have long-stand (created in last year) PRs on SS area which already >>> got over 100 comments (so community and me already put lots of efforts) but >>> no progress in point of view for being merged unfortunately lack of >>> committers' attention. >>> > >>> > - SPARK-20568 [1] : Provide option to clean up completed files in >>> streaming query >>> > - SPARK-25151 [2] : Apply Apache Commons Pool to KafkaDataConsumer >>> > >>> > According to my experiences on previous PRs (including other areas), >>> it won't take more than 1 months regardless of size of code diff to merge >>> once committer(s) gave a focus on PR and reviewed. >>> > >>> > Thanks, >>> > Jungtaek Lim (HeartSaVioR) >>> > >>> > ps. I may agree all committers in SS area could be busy (It might >>> clearly represent SS area lacks committers), but I may not agree they're >>> involved in DSv2 and DSv2 is the first thing to focus. I haven't seen >>> anyone in participants on DSv2 discussions, and most of PRs in SS area is >>> parallel to DSv2 so I'm wondering why we try to couple SS area with DSv2 >>> and restrict its evolution. >>> > >>> > ps2. Some of above is the part of previous mail thread regarding "Plan >>> on Structured Streaming in next major/minor release?" [3] >>> > >>> > I'm sure I still would like to address other items in the list (or >>> new), but without fast feedback it would not be possible. (Maintaining >>> multiple of long-lasting PRs make contributors very tired, and sometimes >>> worse than giving -1 and providing reason to reject.) >>> > >>> > 1. https://github.com/apache/spark/pull/22952 >>> > 2. https://github.com/apache/spark/pull/22138 >>> > 3. >>> https://lists.apache.org/thread.html/e6c8a530c998c4a2bb12b167f815d3726d155ce722047957e32689df@%3Cdev.spark.apache.org%3E >>> >>
