Re last beta, I think it was extremely useful. We identified big issues wrt API versioning and third-party client compatibility. Even if the server code doesnt get deployed widely, I think the beta period is still a good signal to third party client devs to rev up tests and make any updates necessary to support the new version in advance of a full release.
Dana On Mar 12, 2015 2:37 PM, "Jun Rao" <j...@confluent.io> wrote: > Since we have decided to only support security on the new clients, the > security feature will only be available after the new consumer is done. So, > for the next release, we probably should just target finishing the new > consumer as the main feature. If security can be done in the same release, > that's just a bonus. > > Thanks, > > Jun > > On Thu, Mar 12, 2015 at 6:37 AM, Harsha <ka...@harsha.io> wrote: > > > I would like to include ssl/sasl > > 1) Kafka-1684 (Patch posted for a review) > > 2) Kafka-1686 (Patch depends on kafka-1684) > > 3) Kafka-1688 (work is in progress) > > Thanks, > > Harsha > > On Thu, Mar 12, 2015, at 04:35 AM, Guozhang Wang wrote: > > > Gwen, > > > > > > Just for clarification, you were suggesting we should or should not > > > include > > > MM improvement in 0.8.3? I personally would prefer it (KAFKA-1650 and > > > KAFKA-1997) to go into 0.8.3. > > > > > > I see Joe has made a pass over the tickets and mark them 0.8.3. We can > > > probably do another pass and consider adding: > > > > > > 1) Purgatory improvement (KAFKA-1989). > > > 2) Compression improvement (KAFKA-527). > > > 3) Some unit test failures (KAFKA-1501, I think we are pretty close in > > > getting it fixed). > > > 4) any other tickets? > > > > > > Guozhang > > > > > > On Wed, Mar 11, 2015 at 9:40 PM, Jay Kreps <jay.kr...@gmail.com> > wrote: > > > > > > > With regard to mm, I was kind of assuming just based on the amount of > > work > > > > that that would go in for sure, but yeah I agree it is important. > > > > > > > > -Jay > > > > > > > > On Wed, Mar 11, 2015 at 9:39 PM, Jay Kreps <jay.kr...@gmail.com> > > wrote: > > > > > > > > > What I was trying to say was let's do a real release whenever > either > > > > > consumer or authn is done whichever happens first (or both if they > > can > > > > > happen close together)--not sure which is more likely to slip. > > > > > > > > > > WRT the beta thing I think the question for people is whether the > > beta > > > > > period was helpful or not in getting a more stable release? We > could > > > > either > > > > > do a beta release again or we could just do a normal release and > > call the > > > > > consumer feature "experimental" or whatever...basically something > to > > get > > > > it > > > > > in peoples hands before it is supposed to work perfectly and never > > change > > > > > again. > > > > > > > > > > -Jay > > > > > > > > > > > > > > > On Wed, Mar 11, 2015 at 9:27 PM, Gwen Shapira < > gshap...@cloudera.com > > > > > > > > wrote: > > > > > > > > > >> So basically you are suggesting - lets do a beta release whenever > we > > > > >> feel the new consumer is done? > > > > >> > > > > >> This can definitely work. > > > > >> > > > > >> I'd prefer holding for MM improvements too. IMO, its not just more > > > > >> improvements like flush() and compression optimization. > > > > >> Current MirrorMaker can lose data, which makes it pretty useless > for > > > > >> its job. We hear lots of requests for robust MM from our > customers, > > so > > > > >> I can imagine its pretty important to the Kafka community (unless > I > > > > >> have a completely skewed sample). > > > > >> > > > > >> Gwen > > > > >> > > > > >> > > > > >> > > > > >> On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <jay.kr...@gmail.com> > > wrote: > > > > >> > Yeah the real question is always what will we block on? > > > > >> > > > > > >> > I don't think we should try to hold back smaller changes. In > this > > > > >> bucket I > > > > >> > would include most things you described: mm improvements, > replica > > > > >> > assignment tool improvements, flush, purgatory improvements, > > > > compression > > > > >> > optimization, etc. Likely these will all get done in time as > well > > as > > > > >> many > > > > >> > things that kind of pop up from users but probably aren't worth > > doing > > > > a > > > > >> > release for on their own. If one of them slips that fine. I also > > don't > > > > >> > think we should try to hold back work that is done if it isn't > on > > a > > > > >> list. > > > > >> > > > > > >> > I would consider either SSL+SASL or the consumer worthy of a > > release > > > > on > > > > >> its > > > > >> > own. If they finish close to the same time that is great. We can > > maybe > > > > >> just > > > > >> > assess as these evolve where the other one is at and make a call > > > > >> whether it > > > > >> > will be one or both? > > > > >> > > > > > >> > -Jay > > > > >> > > > > > >> > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira < > > gshap...@cloudera.com> > > > > >> wrote: > > > > >> > > > > > >> >> If we are going in terms of features, I can see the following > > > > features > > > > >> >> getting in in the next month or two: > > > > >> >> > > > > >> >> * New consumer > > > > >> >> * Improved Mirror Maker (I've seen tons of interest) > > > > >> >> * Centralized admin requests (aka KIP-4) > > > > >> >> * Nicer replica-reassignment tool > > > > >> >> * SSL (and perhaps also SASL)? > > > > >> >> > > > > >> >> I think this collection will make a nice release. Perhaps we > can > > cap > > > > >> >> it there and focus (as a community) on getting these in, we can > > have > > > > a > > > > >> >> release without too much scope creep in the > > not-very-distant-future? > > > > >> >> Even just 3 out of these 5 will still make a nice incremental > > > > >> >> improvement. > > > > >> >> > > > > >> >> Gwen > > > > >> >> > > > > >> >> > > > > >> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps < > jay.kr...@gmail.com> > > > > >> wrote: > > > > >> >> > Yeah I'd be in favor of a quicker, smaller release but I > think > > as > > > > >> long as > > > > >> >> > we have these big things in flight we should probably keep > the > > > > >> release > > > > >> >> > criteria feature-based rather than time-based, though (e.g. > > "when X > > > > >> >> works" > > > > >> >> > not "every other month). > > > > >> >> > > > > > >> >> > Ideally the next release would have at least a "beta" version > > of > > > > the > > > > >> new > > > > >> >> > consumer. I think having a new hunk of code like that > > available but > > > > >> >> marked > > > > >> >> > as "beta" is maybe a good way to go, as it gets it into > peoples > > > > >> hands for > > > > >> >> > testing. This way we can declare the API not fully locked > down > > > > until > > > > >> the > > > > >> >> > final release too, since mostly users only look at stuff > after > > we > > > > >> release > > > > >> >> > it. Maybe we can try to construct a schedule around this? > > > > >> >> > > > > > >> >> > -Jay > > > > >> >> > > > > > >> >> > > > > > >> >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein < > > joe.st...@stealth.ly> > > > > >> wrote: > > > > >> >> > > > > > >> >> >> There hasn't been any public discussion about the 0.8.3 > > release > > > > >> plan. > > > > >> >> >> > > > > >> >> >> There seems to be a lot of work in flight, work with patches > > and > > > > >> review > > > > >> >> >> that could/should get committed but now just pending KIPS, > > work > > > > >> without > > > > >> >> KIP > > > > >> >> >> but that is in trunk already (e.g. the new Consumer) that > > would be > > > > >> the > > > > >> >> the > > > > >> >> >> release but missing the KIP for the release... > > > > >> >> >> > > > > >> >> >> What does this mean for the 0.8.3 release? What are we > trying > > to > > > > >> get out > > > > >> >> >> and when? > > > > >> >> >> > > > > >> >> >> Also looking at > > > > >> >> >> > > > > >> > > https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan > > > > >> >> >> there > > > > >> >> >> seems to be things we are getting earlier (which is great of > > > > >> course) so > > > > >> >> are > > > > >> >> >> we going to try to up the version and go with 0.9.0? > > > > >> >> >> > > > > >> >> >> 0.8.2.0 ended up getting very bloated and that delayed it > much > > > > >> longer > > > > >> >> than > > > > >> >> >> we had originally communicated to the community and want to > > make > > > > >> sure we > > > > >> >> >> take that feedback from the community and try to improve > upon > > it. > > > > >> >> >> > > > > >> >> >> Thanks! > > > > >> >> >> > > > > >> >> >> ~ Joe Stein > > > > >> >> >> - - - - - - - - - - - - - - - - - > > > > >> >> >> > > > > >> >> >> http://www.stealth.ly > > > > >> >> >> - - - - - - - - - - - - - - - - - > > > > >> >> >> > > > > >> >> > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > >
