Would SAMZA-697 worth adding to 0.10.0 as well? Guozhang
On Thu, Jul 30, 2015 at 6:45 PM, Yan Fang <yanfang...@gmail.com> wrote: > For SAMZA-747, may ping Naveen or Chris. :) > > They have the permission to publish to maven. From the discuss > <https://github.com/facebook/rocksdb/issues/606>, they seem ready for the > release. > > Thanks, > > Fang, Yan > yanfang...@gmail.com > > On Thu, Jul 30, 2015 at 5:35 PM, Navina Ramesh > <nram...@linkedin.com.invalid > > wrote: > > > Ok. I think it got confusing because you are talking about tickets NOT to > > be included in the release :) > > > > Got it, now. +1 for this list of exclusions! > > SAMZA-723 (StreamAppender bug) and SAMZA-747 (rocksdb) should be in > 0.10.0. > > I think we don't have an ETA on the fix for SAMZA-747? > > > > Thanks! > > Navina > > > > On Thu, Jul 30, 2015 at 5:26 PM, Yi Pan <nickpa...@gmail.com> wrote: > > > > > Uh... wrong math all the day today... :( > > > > > > Let me re-try: > > > > > > 29/32 tickets are to be moved to later (i.e. excluded from 0.10.0). > > > > > > 2/32 tickets (including SAMZA-723) will be included > > > > > > 1/32 ticket (SAMZA-689) is up for discussion and I am leaning toward > mark > > > it as won't fix. > > > > > > -Yi > > > > > > On Thu, Jul 30, 2015 at 5:24 PM, Yi Pan <nickpa...@gmail.com> wrote: > > > > > > > Hi, Navina, > > > > > > > > The 29/30 tickets are to be excluded from 0.10.0 (i.e. moved to > > 0.11.0), > > > > the three tickets are either to be included in 0.10.0, or won't fix. > > > > > > > > -Yi > > > > > > > > On Thu, Jul 30, 2015 at 5:10 PM, Navina Ramesh < > > > > nram...@linkedin.com.invalid> wrote: > > > > > > > >> Hi Yi, > > > >> Thanks for summarizing. But why are we excluding SAMZA-723 from the > > > >> current > > > >> release ? Doesn't this break the existing StreamAppender > functionality > > > in > > > >> 0.9? > > > >> > > > >> Thanks! > > > >> Navina > > > >> > > > >> On Thu, Jul 30, 2015 at 4:55 PM, Yi Pan <nickpa...@gmail.com> > wrote: > > > >> > > > >> > Sorry, hit the send button too fast. Let me correct the summary > > > section: > > > >> > > > > >> > 29/32 tickets Status=open/reopen : > > > >> > > > > >> > > > > >> > > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC > > > >> > > > > >> > Three tickets excluded: > > > >> > - SAMZA-689, @Chinmay, were you able to re-produce it? I am > thinking > > > of > > > >> > mark it as won't fix since 0.10.0 will remove the checkpoint topic > > > >> > altogether. > > > >> > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to include > > the > > > >> fix > > > >> > that breaks Samza 0.10.0 build on Linux boxes > > > >> > - SAMZA-723, stream appender deadlock issue > > > >> > > > > >> > Thanks! > > > >> > > > > >> > -Yi > > > >> > > > > >> > > > > >> > On Thu, Jul 30, 2015 at 4:52 PM, Yi Pan <nickpa...@gmail.com> > > wrote: > > > >> > > > > >> > > Hi, all, > > > >> > > > > > >> > > Thanks a lot for helping out to select the features planned in > > > 0.10.0. > > > >> > > > > > >> > > Based on the above discussion, I am proposing to move the > > following > > > >> > > tickets later (i.e. 0.11.0). > > > >> > > > > > >> > > 30/32 tickets Status=open/reopen (exception for SAMZA-723: > stream > > > >> > appender > > > >> > > deadlock issue) : > > > >> > > > > > >> > > > > >> > > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20fixVersion%20%3D%200.10.0%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC > > > >> > > > > > >> > > Two tickets excluded: > > > >> > > - SAMZA-689, @Chinmay, were you able to re-produce it? I am > > thinking > > > >> of > > > >> > > mark it as won't fix since 0.10.0 will remove the checkpoint > topic > > > >> > > altogether. > > > >> > > - SAMZA-747, we will have to upgrade to RocksDB 3.11.1 to > include > > > the > > > >> fix > > > >> > > that breaks Samza 0.10.0 build on Linux boxes > > > >> > > > > > >> > > Anything that I missed? > > > >> > > > > > >> > > Thanks! > > > >> > > > > > >> > > -Yi > > > >> > > > > > >> > > > > > >> > > On Wed, Jul 29, 2015 at 9:57 AM, Navina Ramesh < > > > >> > > nram...@linkedin.com.invalid> wrote: > > > >> > > > > > >> > >> +1 for StreamAppender bug fix as a mandatory in 0.10. > > > >> > >> > > > >> > >> @Yan: will review SAMZA-676 today and test it out locally. > > > >> > >> > > > >> > >> Thanks! > > > >> > >> Navina > > > >> > >> > > > >> > >> On Wed, Jul 29, 2015 at 8:38 AM, Chinmay Soman < > > > >> > chinmay.cere...@gmail.com > > > >> > >> > > > > >> > >> wrote: > > > >> > >> > > > >> > >> > I can take care of SAMZA-340, SAMZA-683 and will follow up > > with > > > >> Luis > > > >> > >> for > > > >> > >> > SAMZA-401,2,3,4 > > > >> > >> > > > > >> > >> > On Wed, Jul 29, 2015 at 12:10 AM, Dan <danharve...@gmail.com > > > > > >> wrote: > > > >> > >> > > > > >> > >> > > I agree SAMZA-741 for the ElasticSearch producer should be > in > > > >> too so > > > >> > >> > we've > > > >> > >> > > got a better API as part of that release. > > > >> > >> > > > > > >> > >> > > - Dan > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > On 29 July 2015 at 07:51, Yan Fang <yanfang...@gmail.com> > > > wrote: > > > >> > >> > > > > > >> > >> > > > Actually, I also want to include a few patch-available > > > >> features, > > > >> > >> > > > especially: > > > >> > >> > > > > > > >> > >> > > > 1. broadcast stream (SAMZA-676) > > > >> > >> > > > - waiting for review > > > >> > >> > > > > > > >> > >> > > > 2. graphite support (SAMZA-340) > > > >> > >> > > > 3. meter and histogram (SAMZA-683) > > > >> > >> > > > 4. utility (SAMZA-401) > > > >> > >> > > > - 2,3,4 belong to Luis, if he does not have time to > > > update, > > > >> > >> since > > > >> > >> > > they > > > >> > >> > > > only need some small changes, we can edit it and get +1 > > from > > > >> > another > > > >> > >> > > > committer. > > > >> > >> > > > > > > >> > >> > > > 5. hdfs producer (SAMZA-693) > > > >> > >> > > > - I am reviewing. > > > >> > >> > > > > > > >> > >> > > > 6. upgrade yarn to 2.7.1 (SAMZA-563) > > > >> > >> > > > - though I am reviewing, this ticket is negotiable if > we > > > >> want > > > >> > to > > > >> > >> put > > > >> > >> > > > into the 0.10.0 release. If we do not, I think, when > users > > > >> enable > > > >> > >> the > > > >> > >> > > > worker-persisting and container-persisting features, > Samza > > > will > > > >> > not > > > >> > >> be > > > >> > >> > > able > > > >> > >> > > > to handle it. (Some classes are only available after yarn > > > 2.5.0 > > > >> > >> while > > > >> > >> > > Samza > > > >> > >> > > > currently only support yarn 2.4.0) > > > >> > >> > > > > > > >> > >> > > > 7. others: scrooge, class loader isolation, etc. > > > >> > >> > > > - those are waiting for reviewing too. > > > >> > >> > > > > > > >> > >> > > > My opinion is that, if we can clean up all the > > > patch-available > > > >> > >> tickets, > > > >> > >> > > it > > > >> > >> > > > will be great. Most of them have been already reviewed > more > > > >> than > > > >> > >> once. > > > >> > >> > > So I > > > >> > >> > > > think it should not be very time-consuming to have them > in > > > the > > > >> > >> 0.10.0 > > > >> > >> > > > release. > > > >> > >> > > > > > > >> > >> > > > What do you think? > > > >> > >> > > > > > > >> > >> > > > Of course, another must-have is the bug-fix of the Stream > > > >> > Appender. > > > >> > >> :) > > > >> > >> > > > > > > >> > >> > > > Thanks, > > > >> > >> > > > > > > >> > >> > > > Fang, Yan > > > >> > >> > > > yanfang...@gmail.com > > > >> > >> > > > > > > >> > >> > > > On Tue, Jul 28, 2015 at 10:27 PM, Roger Hoover < > > > >> > >> roger.hoo...@gmail.com > > > >> > >> > > > > > >> > >> > > > wrote: > > > >> > >> > > > > > > >> > >> > > > > Thanks, Yi. > > > >> > >> > > > > > > > >> > >> > > > > I propose that we also include SAMZA-741 for > > Elasticsearch > > > >> > >> versioning > > > >> > >> > > > > support with the new ES producer. I think it's very > > close > > > to > > > >> > >> being > > > >> > >> > > > merged. > > > >> > >> > > > > > > > >> > >> > > > > Roger > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan < > > > >> nickpa...@gmail.com> > > > >> > >> > wrote: > > > >> > >> > > > > > > > >> > >> > > > > > Hi, all, > > > >> > >> > > > > > > > > >> > >> > > > > > I want to start the discussion on the release > schedule > > > for > > > >> > >> 0.10.0. > > > >> > >> > > > There > > > >> > >> > > > > > are a few important features that we plan to release > in > > > >> 0.10.0 > > > >> > >> and > > > >> > >> > I > > > >> > >> > > > want > > > >> > >> > > > > > to start this thread s.t. we can agree on what to > > include > > > >> in > > > >> > >> 0.10.0 > > > >> > >> > > > > > release. > > > >> > >> > > > > > > > > >> > >> > > > > > There are the following main features added in > 0.10.0: > > > >> > >> > > > > > - RocksDB TTL support > > > >> > >> > > > > > - Add CoordinatorStream and disable CheckpointManager > > > >> > >> > > > > > - Elasticsearch Producer > > > >> > >> > > > > > - Host affinity > > > >> > >> > > > > > And other 0.10.0 tickets: > > > >> > >> > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > > > > >> > > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20status%20in%20(%22In%20Progress%22%2C%20Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%200.10.0 > > > >> > >> > > > > > > > > >> > >> > > > > > I propose to cut a 0.10.0 release after we get the > > > >> following > > > >> > >> issues > > > >> > >> > > > > > resolved: > > > >> > >> > > > > > - SAMZA-615: Migrate checkpoint from checkpoint topic > > to > > > >> > >> > Coordinator > > > >> > >> > > > > stream > > > >> > >> > > > > > - SAMZA-617: YARN host affinity in Samza > > > >> > >> > > > > > > > > >> > >> > > > > > Thoughts? > > > >> > >> > > > > > > > > >> > >> > > > > > Thanks! > > > >> > >> > > > > > > > > >> > >> > > > > > -Yi > > > >> > >> > > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > >> > >> > > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > -- > > > >> > >> > Thanks and regards > > > >> > >> > > > > >> > >> > Chinmay Soman > > > >> > >> > > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> -- > > > >> > >> Navina R. > > > >> > >> > > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> -- > > > >> Navina R. > > > >> > > > > > > > > > > > > > > > > > > > -- > > Navina R. > > > -- -- Guozhang
