Since we have decided to only support security on the new clients, the security feature will only be available after the new consumer is done. So, for the next release, we probably should just target finishing the new consumer as the main feature. If security can be done in the same release, that's just a bonus.
Thanks, Jun On Thu, Mar 12, 2015 at 6:37 AM, Harsha <ka...@harsha.io> wrote: > I would like to include ssl/sasl > 1) Kafka-1684 (Patch posted for a review) > 2) Kafka-1686 (Patch depends on kafka-1684) > 3) Kafka-1688 (work is in progress) > Thanks, > Harsha > On Thu, Mar 12, 2015, at 04:35 AM, Guozhang Wang wrote: > > Gwen, > > > > Just for clarification, you were suggesting we should or should not > > include > > MM improvement in 0.8.3? I personally would prefer it (KAFKA-1650 and > > KAFKA-1997) to go into 0.8.3. > > > > I see Joe has made a pass over the tickets and mark them 0.8.3. We can > > probably do another pass and consider adding: > > > > 1) Purgatory improvement (KAFKA-1989). > > 2) Compression improvement (KAFKA-527). > > 3) Some unit test failures (KAFKA-1501, I think we are pretty close in > > getting it fixed). > > 4) any other tickets? > > > > Guozhang > > > > On Wed, Mar 11, 2015 at 9:40 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > > > With regard to mm, I was kind of assuming just based on the amount of > work > > > that that would go in for sure, but yeah I agree it is important. > > > > > > -Jay > > > > > > On Wed, Mar 11, 2015 at 9:39 PM, Jay Kreps <jay.kr...@gmail.com> > wrote: > > > > > > > What I was trying to say was let's do a real release whenever either > > > > consumer or authn is done whichever happens first (or both if they > can > > > > happen close together)--not sure which is more likely to slip. > > > > > > > > WRT the beta thing I think the question for people is whether the > beta > > > > period was helpful or not in getting a more stable release? We could > > > either > > > > do a beta release again or we could just do a normal release and > call the > > > > consumer feature "experimental" or whatever...basically something to > get > > > it > > > > in peoples hands before it is supposed to work perfectly and never > change > > > > again. > > > > > > > > -Jay > > > > > > > > > > > > On Wed, Mar 11, 2015 at 9:27 PM, Gwen Shapira <gshap...@cloudera.com > > > > > > wrote: > > > > > > > >> So basically you are suggesting - lets do a beta release whenever we > > > >> feel the new consumer is done? > > > >> > > > >> This can definitely work. > > > >> > > > >> I'd prefer holding for MM improvements too. IMO, its not just more > > > >> improvements like flush() and compression optimization. > > > >> Current MirrorMaker can lose data, which makes it pretty useless for > > > >> its job. We hear lots of requests for robust MM from our customers, > so > > > >> I can imagine its pretty important to the Kafka community (unless I > > > >> have a completely skewed sample). > > > >> > > > >> Gwen > > > >> > > > >> > > > >> > > > >> On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <jay.kr...@gmail.com> > wrote: > > > >> > Yeah the real question is always what will we block on? > > > >> > > > > >> > I don't think we should try to hold back smaller changes. In this > > > >> bucket I > > > >> > would include most things you described: mm improvements, replica > > > >> > assignment tool improvements, flush, purgatory improvements, > > > compression > > > >> > optimization, etc. Likely these will all get done in time as well > as > > > >> many > > > >> > things that kind of pop up from users but probably aren't worth > doing > > > a > > > >> > release for on their own. If one of them slips that fine. I also > don't > > > >> > think we should try to hold back work that is done if it isn't on > a > > > >> list. > > > >> > > > > >> > I would consider either SSL+SASL or the consumer worthy of a > release > > > on > > > >> its > > > >> > own. If they finish close to the same time that is great. We can > maybe > > > >> just > > > >> > assess as these evolve where the other one is at and make a call > > > >> whether it > > > >> > will be one or both? > > > >> > > > > >> > -Jay > > > >> > > > > >> > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira < > gshap...@cloudera.com> > > > >> wrote: > > > >> > > > > >> >> If we are going in terms of features, I can see the following > > > features > > > >> >> getting in in the next month or two: > > > >> >> > > > >> >> * New consumer > > > >> >> * Improved Mirror Maker (I've seen tons of interest) > > > >> >> * Centralized admin requests (aka KIP-4) > > > >> >> * Nicer replica-reassignment tool > > > >> >> * SSL (and perhaps also SASL)? > > > >> >> > > > >> >> I think this collection will make a nice release. Perhaps we can > cap > > > >> >> it there and focus (as a community) on getting these in, we can > have > > > a > > > >> >> release without too much scope creep in the > not-very-distant-future? > > > >> >> Even just 3 out of these 5 will still make a nice incremental > > > >> >> improvement. > > > >> >> > > > >> >> Gwen > > > >> >> > > > >> >> > > > >> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <jay.kr...@gmail.com> > > > >> wrote: > > > >> >> > Yeah I'd be in favor of a quicker, smaller release but I think > as > > > >> long as > > > >> >> > we have these big things in flight we should probably keep the > > > >> release > > > >> >> > criteria feature-based rather than time-based, though (e.g. > "when X > > > >> >> works" > > > >> >> > not "every other month). > > > >> >> > > > > >> >> > Ideally the next release would have at least a "beta" version > of > > > the > > > >> new > > > >> >> > consumer. I think having a new hunk of code like that > available but > > > >> >> marked > > > >> >> > as "beta" is maybe a good way to go, as it gets it into peoples > > > >> hands for > > > >> >> > testing. This way we can declare the API not fully locked down > > > until > > > >> the > > > >> >> > final release too, since mostly users only look at stuff after > we > > > >> release > > > >> >> > it. Maybe we can try to construct a schedule around this? > > > >> >> > > > > >> >> > -Jay > > > >> >> > > > > >> >> > > > > >> >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein < > joe.st...@stealth.ly> > > > >> wrote: > > > >> >> > > > > >> >> >> There hasn't been any public discussion about the 0.8.3 > release > > > >> plan. > > > >> >> >> > > > >> >> >> There seems to be a lot of work in flight, work with patches > and > > > >> review > > > >> >> >> that could/should get committed but now just pending KIPS, > work > > > >> without > > > >> >> KIP > > > >> >> >> but that is in trunk already (e.g. the new Consumer) that > would be > > > >> the > > > >> >> the > > > >> >> >> release but missing the KIP for the release... > > > >> >> >> > > > >> >> >> What does this mean for the 0.8.3 release? What are we trying > to > > > >> get out > > > >> >> >> and when? > > > >> >> >> > > > >> >> >> Also looking at > > > >> >> >> > > > >> > https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan > > > >> >> >> there > > > >> >> >> seems to be things we are getting earlier (which is great of > > > >> course) so > > > >> >> are > > > >> >> >> we going to try to up the version and go with 0.9.0? > > > >> >> >> > > > >> >> >> 0.8.2.0 ended up getting very bloated and that delayed it much > > > >> longer > > > >> >> than > > > >> >> >> we had originally communicated to the community and want to > make > > > >> sure we > > > >> >> >> take that feedback from the community and try to improve upon > it. > > > >> >> >> > > > >> >> >> Thanks! > > > >> >> >> > > > >> >> >> ~ Joe Stein > > > >> >> >> - - - - - - - - - - - - - - - - - > > > >> >> >> > > > >> >> >> http://www.stealth.ly > > > >> >> >> - - - - - - - - - - - - - - - - - > > > >> >> >> > > > >> >> > > > >> > > > > > > > > > > > > > > > > > > > -- > > -- Guozhang >
