Re: [DISCUSS] KIP-378: Enable Dependency Injection for Kafka Streams handlers

[Damian Guy]

Hi Wladimir,

Of the two approaches in the KIP - i feel the second approach is cleaner.
However, am i correct in assuming that you want to have the
`ConfiguredStreamsFactory` as a ctor arg in `StreamsConfig` so that Spring
can inject this for you?

Otherwise you could just put the ApplicationContext as a property in the
config and then use that via the configure method of the appropriate
handler to get your actual handler.

Thanks,
Damian

On Tue, 9 Oct 2018 at 01:55, Guozhang Wang  wrote:

> John, thanks for the explanation, now it makes much more sense to me.
>
> As for the concrete approach, to me it seems the first option requires less
> changes than the second (ConfiguredStreamsFactory based) approach, whereas
> the second one requires an additional interface that is overlapping with
> the AbstractConfig.
>
> I'm aware that in KafkaProducer / KafkaConsumer we do not have public
> constructors for taking a ProducerConfig or ConsumerConfig directly, and
> anyone using Spring can share how you've worked around it by far? If it is
> very awkward I'm not against just adding the XXXConfigs to the constructors
> directly.
>
> Guozhang
>
> On Fri, Oct 5, 2018 at 1:48 PM, John Roesler  wrote:
>
> > Hi Wladimir,
> >
> > Thanks for the KIP!
> >
> > As I mentioned in the PR discussion, I personally prefer not to recommend
> > overriding StreamsConfig for this purpose.
> >
> > It seems like a person wishing to create a DI shim would have to acquire
> > quite a deep understanding of the class and its usage to figure out what
> > exactly to override to accomplish their goals without breaking
> everything.
> > I'm honestly impressed with the method you came up with to create your
> > Spring/Streams shim.
> >
> > I think we can make to path for the next person smoother by going with
> > something more akin to the ConfiguredStreamsFactory. This is a
> constrained
> > interface that tells you exactly what you have to implement to create
> such
> > a shim.
> >
> > A few thoughts:
> > 1. it seems like we can keep all the deprecated constructors still
> > deprecated
> >
> > 2. we could add just one additional constructor to each of KafkaStreams
> and
> > TopologyTestDriver to still take a Properties, but also your new
> > ConfiguredStreamsFactory
> >
> > 3. I don't know if I'm sold on the name ConfiguredStreamsFactory, since
> it
> > does not produce configured streams. Instead, it produces configured
> > instances... How about ConfiguredInstanceFactory?
> >
> > 4. if I understand the usage correctly, it's actually a pretty small
> number
> > of classes that we actually make via getConfiguredInstance. Offhand, I
> can
> > think of the key/value Serdes, the deserialization exception handler, and
> > the production exception handler.
> > Perhaps, instead of maintaining a generic "class instantiator", we could
> > explore a factory interface that just has methods for creating exactly
> the
> > kinds of things we need to create. In fact, we already have something
> like
> > this: org.apache.kafka.streams.KafkaClientSupplier . Do you think we
> could
> > just add some more methods to that interface (and maybe rename it)
> instead?
> >
> > Thanks,
> > -John
> >
> > On Fri, Oct 5, 2018 at 3:31 PM John Roesler  wrote:
> >
> > > Hi Guozhang,
> > >
> > > I'm going to drop in a little extra context from the preliminary PR
> > > discussion (https://github.com/apache/kafka/pull/5344).
> > >
> > > The issue isn't that it's impossible to use Streams within a Spring
> app,
> > > just that the interplay between our style of construction/configuration
> > and
> > > Spring's is somewhat awkward compared to the normal experience with
> > > dependency injection.
> > >
> > > I'm guessing users of dependency injection would not like the approach
> > you
> > > offered. I believe it's commonly considered an antipattern when using
> DI
> > > frameworks to pass the injector directly into the class being
> > constructed.
> > > Wladimir has also offered an alternative usage within the current
> > framework
> > > of injecting pre-constructed dependencies into the Properties, and then
> > > retrieving and casting them inside the configured class.
> > >
> > > It seems like this KIP is more about offering a more elegant interface
> to
> > > DI users.
> > >
> > > One of the points that Wladimir raised on his PR discussion was the
> > desire
> > > to configure the classes in a typesafe way in the constructor (thus
> > > allowing the use of immutable classes).
> > >
> > > With this KIP, it would be possible for a DI user to:
> > > 1. register a Streams-Spring or Streams-Guice (etc) "plugin" (via
> either
> > > of the mechanisms he proposed)
> > > 2. simply make the Serdes, exception handlers, etc, available on the
> > class
> > > path with the DI annotations
> > > 3. start the app
> > >
> > > There's no need to mess with passing dependencies (or the injector)
> > > through the properties.
> > >
> > > Sorry for "injecting" myself into your discussion, but it 

Re: [VOTE] KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde

[Damian Guy]

Thanks +1 binding

On Sun, 7 Oct 2018 at 03:16, Chia-Ping Tsai  wrote:

> (just bump)
>
> We have following votes:
>
> non-binding
> 1. John Roesler
> 2. vito jeng
> 3. Richard Yu
> 4.  Bill Bejeck
> 5. Satish Duggana
>
> binding
> 1. Matthias J. Sax
> 2. Ismael Juma
>
> it needs one more ticket :)
>
> Cheers,
> Chia-Ping
>
> On 2018/07/05 14:45:01, Chia-Ping Tsai  wrote:
> > hi all,
> >
> > I would like to start voting on "KIP-331 Add default implementation to
> close() and configure() for Serializer, Deserializer and Serde"
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-331+Add+default+implementation+to+close%28%29+and+configure%28%29+for+Serializer%2C+Deserializer+and+Serde
> >
> > Cheers,
> > Chia-Ping
> >
>

Re: Vote for KIP-393 (Fix time windowed serde to deserialize changelog topic)

[Damian Guy]

+1 (binding)

On Wed, 12 Dec 2018 at 13:27, Adam Bellemare 
wrote:

> +1 (non-binding) from me. Looks like a pretty clear-cut case.
>
>
>
> On Tue, Dec 11, 2018 at 1:11 AM Shawn Nguyen 
> wrote:
>
> > Thanks for the feedback Guozhang! I updated the KIP.
> >
> > In the meantime, could I ask for additional binding votes/approval on
> this
> > KIP proposal?
> >
> > Shawn
> >
> > On Thu, Dec 6, 2018 at 1:22 PM Liquan Pei  wrote:
> >
> > > +1 (non-binding)
> > >
> > > On Wed, Dec 5, 2018 at 4:51 PM Guozhang Wang 
> wrote:
> > >
> > >> Hello Shawn,
> > >>
> > >> Thanks for the writeup. I've made a pass over it and here are some
> minor
> > >> comments:
> > >>
> > >> 1) As we discussed in the PR:
> https://github.com/apache/kafka/pull/5307
> > ,
> > >> the public APIs that we will add is
> > >>
> > >> In WindowedSerdes:
> > >> ```
> > >> static public  Serde>
> > timeWindowedChangelogSerdeFrom(final
> > >> Class type, final long windowSize)
> > >> ```
> > >>
> > >> In TimeWindowedSerde
> > >> ```
> > >> TimeWindowedSerde forChangelog(final boolean);
> > >> ```
> > >>
> > >> Other classes such as WindowedKeySchema are internal classes for
> > >> implementation details and hence do not need to be listed in the wiki
> as
> > >> public APIs.
> > >>
> > >>
> > >> 2) The wiki doc may reads a bit confusing for audience who are not
> > >> familiar
> > >> with the PR, since we mentioned the "forChangelog()" function and the
> > >> "isChangelog" parameter without clear definitions, but only explained
> > what
> > >> it is later in the docs as java code examples. I think rephrasing the
> > >> early
> > >> paragraphs to explain a bit more why we will add a new internal field
> > >> along
> > >> with a setter, its semantics (its default value and how
> deserialization
> > >> will be different depending on that) would be better.
> > >>
> > >> Otherwise, I'm +1 on the KIP, thanks!
> > >>
> > >>
> > >> Guozhang
> > >>
> > >>
> > >> On Wed, Dec 5, 2018 at 8:18 AM Shawn Nguyen 
> > >> wrote:
> > >>
> > >> > Hey all,
> > >> >
> > >> > I wanted to start a vote on approval of KIP-393
> > >> > <
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-393%3A+Time+windowed+serde+to+properly+deserialize+changelog+input+topic
> > >> > >
> > >> > to
> > >> > fix the current time windowed serde for properly deserializing
> > changelog
> > >> > input topics. Let me know what you guys think.
> > >> >
> > >> > Thanks,
> > >> > Shawn
> > >> >
> > >>
> > >>
> > >> --
> > >> -- Guozhang
> > >>
> > >
> > >
> > > --
> > > Liquan Pei
> > > Software Engineer, Confluent Inc
> > >
> >
>

Re: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL Factory

[Damian Guy]

Hi Clement,

You should start a separate thread for the vote, i.e., one with a subject
of [VOTE] KIP-383 ...

Looks like you haven't done this?

On Tue, 18 Dec 2018 at 17:33, Harsha  wrote:

> Yes. +1 binding.
>
> Thanks,
> Harsha
>
> On Mon, Dec 17, 2018, at 5:21 AM, Pellerin, Clement wrote:
> > I'm new here. Is this vote binding or not?
> >
> > -Original Message-
> > From: Harsha [mailto:ka...@harsha.io]
> > Sent: Saturday, December 15, 2018 1:59 PM
> > To: dev@kafka.apache.org
> > Subject: Re: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL
> Factory
> >
> > Overall LGTM. +1.
> >
> > Thanks,
> > Harsha
>

Re: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL Factory

[Damian Guy]

must be my gmail playing up. This appears to be the DISCUSS thread to me...

On Thu, 20 Dec 2018 at 18:42, Harsha  wrote:

> Damian,
>This is the VOTE thread. There is a DISCUSS thread which
> concluded in it.
>
> -Harsha
>
>
> On Wed, Dec 19, 2018, at 5:04 AM, Pellerin, Clement wrote:
> > I did that and nobody came.
> > https://lists.apache.org/list.html?dev@kafka.apache.org:lte=1M:kip-383
> > I don't understand why this feature is not more popular.
> > It's the solution to one Jira and a work-around for a handful more Jiras.
> >
> > -Original Message-
> > From: Damian Guy [mailto:damian@gmail.com]
> > Sent: Wednesday, December 19, 2018 7:38 AM
> > To: dev
> > Subject: Re: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL
> Factory
> >
> > Hi Clement,
> >
> > You should start a separate thread for the vote, i.e., one with a subject
> > of [VOTE] KIP-383 ...
> >
> > Looks like you haven't done this?
>

Re: [VOTE] KIP-414: Expose Embedded ClientIds in Kafka Streams

[Damian Guy]

+1

On Fri, 11 Jan 2019 at 05:09, John Roesler  wrote:

> Hi Guozhang,
>
> It sounds reasonable to me. I'm +1 (nonbinding).
>
> -John
>
> On Tue, Jan 8, 2019 at 8:51 PM Guozhang Wang  wrote:
>
> > Hello folks,
> >
> > I'd like to start a voting process for the following KIP:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-414%3A+Expose+Embedded+ClientIds+in+Kafka+Streams
> >
> > It is a pretty straight-forward and small augment to Stream's public
> > ThreadMetadata interface, as an outcome of the discussion on KIP-345.
> Hence
> > I think we can skip the DISCUSS thread and move on to voting directly. If
> > people have any questions about the context of KIP-414 or KIP-345, please
> > feel free to read these two wiki pages and let me know.
> >
> >
> > -- Guozhang
> >
>

Re: [VOTE] - KIP-213 (new vote) - Simplified and revised.

[Damian Guy]

+1 binding

On Thu, 10 Jan 2019 at 16:57, Bill Bejeck  wrote:

> +1 from me.  Great job on the KIP.
>
> -Bill
>
> On Thu, Jan 10, 2019 at 11:35 AM John Roesler  wrote:
>
> > It's a +1 (nonbinding) from me as well.
> >
> > Thanks for sticking with this, Adam!
> > -John
> >
> > On Wed, Jan 9, 2019 at 6:22 PM Guozhang Wang  wrote:
> >
> > > Hello Adam,
> > >
> > > I'm +1 on the current proposal, thanks!
> > >
> > >
> > > Guozhang
> > >
> > > On Mon, Jan 7, 2019 at 6:13 AM Adam Bellemare <
> adam.bellem...@gmail.com>
> > > wrote:
> > >
> > > > Hi All
> > > >
> > > > I would like to call a new vote on KIP-213. The design has changed
> > > > substantially. Perhaps more importantly, the KIP and associated
> > > > documentation has been greatly simplified. I know this KIP has been
> on
> > > the
> > > > mailing list for a long time, but the help from John Roesler and
> > Guozhang
> > > > Wang have helped put it into a much better state. I would appreciate
> > any
> > > > feedback or votes.
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-213+Support+non-key+joining+in+KTable
> > > >
> > > >
> > > >
> > > > Thank you
> > > >
> > > > Adam Bellemare
> > > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>

Re: [VOTE] KIP-379: Multiple Consumer Group Management

[Damian Guy]

+1 (binding) thanks

On Tue, 15 Jan 2019 at 09:43, Alex D  wrote:

> Hello guys,
>
> 2 votes from Jason, Gwen
> Any binding votes?
>
>
> Hello M Manna,
> If migrating to JSON makes sense, you can post a new KIP for this purpose.
>
> Thanks,
> Alex Dunayevsky
>
> On Tue, 15 Jan 2019, 04:24 Jason Gustafson 
> > +1 Thanks for the KIP!
> >
> > -Jason
> >
> > On Mon, Jan 14, 2019 at 5:15 PM Gwen Shapira  wrote:
> >
> > > I was also wondering about that. I think its for compatibility with
> > > the existing output.
> > >
> > > We can have a separate KIP to add JSON output.
> > >
> > > On Mon, Jan 14, 2019 at 7:55 AM M. Manna  wrote:
> > > >
> > > > Hi,
> > > >
> > > > Thanks for this KIP. Could you kindly clarify why CSV format is
> useful,
> > > but
> > > > not anything else?
> > > >
> > > > CSV format is ancient and the only reason it keep existing is various
> > > > application is because legacy applications aren't moving away from
> > using
> > > > them. Would you would consider JSON or YAML?
> > > >
> > > > Also, if you think about the kafka-reassign-partitions - it's also
> > using
> > > > JSON, not CSV. That is my only point. However, if majority feels that
> > > it's
> > > > not
> > > > an issue I believe it's a team decision after all :).
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > On Mon, 14 Jan 2019 at 15:06, Gwen Shapira 
> wrote:
> > > >
> > > > > +1. Thanks, that will be very helpful.
> > > > >
> > > > > On Mon, Jan 14, 2019, 4:43 AM Alex D  > wrote:
> > > > >
> > > > > > Hello guys,
> > > > > >
> > > > > > We need your VOTES for the KIP-379: Multiple Consumer Group
> > > Management.
> > > > > >
> > > > > > KIP-379:
> > > > > >
> > > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-379%3A+Multiple+Consumer+Group+Management
> > > > > >
> > > > > > PR: https://github.com/apache/kafka/pull/5726/
> > > > > >
> > > > > > Let's start the voting session.
> > > > > >
> > > > > > Thank you,
> > > > > >
> > > > > > Alex Dunayevsky
> > > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Gwen Shapira
> > > Product Manager | Confluent
> > > 650.450.2760 | @gwenshap
> > > Follow us: Twitter | blog
> > >
> >
>

Re: [VOTE] KIP-258: Allow to Store Record Timestamps in RocksDB

[Damian Guy]

+1

On Wed, 16 Jan 2019 at 05:09, Patrik Kleindl  wrote:

> +1 (non-binding)
> Thanks too
> Best regards
> Patrik
>
> > Am 16.01.2019 um 03:30 schrieb Bill Bejeck :
> >
> > Thanks for the KIP Matthias.
> >
> > +1
> >
> > -Bill
> >
> > On Tue, Jan 15, 2019 at 7:33 PM Matthias J. Sax 
> > wrote:
> >
> >> Hi,
> >>
> >> I would like to start the vote for KIP-258:
> >>
> >>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB
> >>
> >> The KIP adds new stores that allow to store record timestamps next to
> >> key and value. Additionally, we will allow to upgrade exiting stores to
> >> the new stores; this will allow us to use the new stores in the DSL with
> >> a smooth upgrade path.
> >>
> >>
> >> -Matthias
> >>
> >>
>

Re: [DISCUSS] KIP-292: Add transformValues() method to KTable

[Damian Guy]

Initially i thought materializing a store would be overkill, but from a
consistency point of view it makes sense to add an overload that takes a
`Materialized` and only create the store if that is supplied.

On Sun, 6 May 2018 at 17:52 Matthias J. Sax  wrote:

> Andy,
>
> thanks for the KIP. I don't have any further comments.
>
> My 2cents about Guozhang's questions: as I like consistent behavior, I
> think transfromValues() should behave the same way as filter() and
> mapValues().
>
>
> -Matthias
>
> On 5/2/18 2:24 PM, Guozhang Wang wrote:
> > Hello Andy,
> >
> > Thanks for the KIP. The motivation and the general proposal looks good to
> > me. I think in KTable it is indeed valuable to add the functions that
> does
> > not change key, such as mapValues, transformValues, and filter.
> >
> > There are a few meta comments I have about the semantics of the newly
> added
> > functions:
> >
> > 1) For the resulted KTable, how should its "queryableStoreName()" be
> > returning?
> >
> > 2) More specifically, how do we decide if the resulted KTable is to be
> > materialized or not? E.g. if there is no store names provided then it is
> > likely that the resulted KTable is not materialized, or at least not
> > logically materialized and not be queryable. What if there is at least
> one
> > state store provided? Will any of them be provided as the materialized
> > store, or should we still add a Materialized parameter for this purpose?
> >
> > 3) For its internal implementations, how should the key/value serde,
> > sendOldValues flag etc be inherited from its parent processor node?
> >
> >
> > Guozhang
> >
> >
> > On Wed, May 2, 2018 at 12:43 PM, Andy Coates  wrote:
> >
> >> Hi everyone,
> >>
> >> I would like to start a discussion for KIP 292. I would appreciate it if
> >> you could review and provide feedback.
> >>
> >> KIP: KIP-292: Add transformValues() method to KTable
> >>  >> 292%3A+Add+transformValues%28%29+method+to+KTable>
> >> Jira: KAFKA-6849 
> >>
> >>PR: #4959 
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Andy
> >>
> >
> >
> >
>
>

Re: [DISCUSS] KIP-292: Add transformValues() method to KTable

[Damian Guy]

I'm a +1 for Guozhang's suggestion

On Fri, 11 May 2018 at 10:20 Andy Coates  wrote:

> Makes sense to me.  What do others think?
>
> On 11 May 2018 at 10:13, Guozhang Wang  wrote:
>
> > Hi folks,
> >
> > While looking into the overloaded functions, I'm wondering if we can save
> > the transformers without key, i.e. only add two overloaded functions:
> >
> >  KTable transformValues(final
> ValueTransformerWithKeySupplier > super K, ? super V, ? extends VR> valueTransformerSupplier,
> >final String... stateStoreNames);
> >
> >  KTable transformValues(final
> ValueTransformerWithKeySupplier > super K, ? super V, ? extends VR> valueTransformerSupplier,
> >final Materialized > KeyValueStore> materialized,
> >final String... stateStoreNames);
> >
> > The reason is that, in KIP-149 we've added the overloaded functions
> > `withKey`, which should be covering the case without key already because
> if
> > users do not really need the key, they can just take it as a dummy
> > parameter. We did not deprecate the old ones since some of them have just
> > been added one version back. But if we agree that by the end of the day
> we
> > would only maintain the overloaded value functions "with key" only, then
> we
> > should not add the ones without keys any more in new KIPs.
> >
> > WDYT?
> >
> >
> > Guozhang
> >
> >
> > On Fri, May 11, 2018 at 9:42 AM, Andy Coates  wrote:
> >
> > > Sorry for my lack of response - I've been out of action with a bad back
> > for
> > > a few days!
> > >
> > > I originally had the `Materialized` overloads added to the API. I'll
> > update
> > > the KIP / PR with these again. In terms of semantics, as Matthias
> > suggests,
> > > these should be consistent with filter() and mapValues(), etc.
> > >
> > > On 8 May 2018 at 17:59, Guozhang Wang  wrote:
> > >
> > > > To follow on Matthias and Damian's comments here:
> > > >
> > > > If we are going to add the overload functions as
> > > >
> > > > ```
> > > >  KTable transformValues(final ValueTransformerSupplier > super
> > > > V,
> > > > ? extends VR> valueTransformerSupplier,
> > > >final String... stateStoreNames,
> > > >final Materialized > > > VR, KeyValueStore materialized);
> > > >
> > > >  KTable transformValues(final
> ValueTransformerWithKeySupplie
> > > r > > > super K, ? super V, ? extends VR> valueTransformerSupplier,
> > > >final String... stateStoreNames,
> > > >  final Materialized > > > VR, KeyValueStore materialized);
> > > > ```
> > > >
> > > > Then are we going to still only allow the valueTransofmer.init() /
> > > > process() to be able to access N stores, with N stores specified with
> > the
> > > > stateStoreNames, but not the one specified in materialized.name()?
> > > > Personally I think it should be the case as the materialized store
> > should
> > > > be managed by the Streams library itself, but we should probably help
> > > users
> > > > to understand if they have some stores used for the same purpose
> > (storing
> > > > the value that are going to be sent to the downstream changelog
> stream
> > of
> > > > KTable), they should save that store and not creating by themselves
> as
> > it
> > > > will be auto created by the Streams library.
> > > >
> > > >
> > > > Guozhang
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, May 8, 2018 at 7:45 AM, Damian Guy 
> > wrote:
> > > >
> > > > > Initially i thought materializing a store would be overkill, but
> > from a
> > > > > consistency point of view it makes sense to add an overload that
> > takes
> > > a
> > > > > `Materialized` and only create the store if that is supplied.
> > > > >
> > > > > On Sun, 6 May 2018 at 17:52 Matthias J. Sax  >
> > > > wrote:
> > > > >
> > > > > > Andy,
> > > > > >
> > > > > > 

Re: [VOTE] KIP-206: Add support for UUID serialization and deserialization

[Damian Guy]

+1 (binding)

On Fri, 4 May 2018 at 08:55 Rajini Sivaram  wrote:

> Hi Brandon,
>
> +1 (binding)
>
> Thanks for the KIP!
>
> Regards,
>
> Rajini
>
> On Fri, May 4, 2018 at 2:14 PM, Brandon Kirchner <
> brandon.kirch...@gmail.com
> > wrote:
>
> > I'd like to resurrect this one last time, any chance we can get some more
> > binding votes and move this forward?
> >
> > Brandon K.
> >
> > On Fri, Feb 2, 2018 at 11:35 AM, Colin McCabe 
> wrote:
> >
> > > Hi Brandon,
> > >
> > > I think people are generally busy working on the upcoming release now.
> > > Sorry for the inconvenience.
> > >
> > > best,
> > >
> > >
> > > On Fri, Feb 2, 2018, at 07:33, Brandon Kirchner wrote:
> > > > I'd really like this to get 2 more binding votes. If that doesn't
> > happen,
> > > > how / can this still move forward? Not sure what the procedure is...
> > > >
> > > > Brandon K.
> > > >
> > > > On Tue, Jan 30, 2018 at 9:56 AM, Mickael Maison <
> > > mickael.mai...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 (non binding)
> > > > > Thanks for the KIP
> > > > >
> > > > > On Tue, Jan 30, 2018 at 9:49 AM, Manikumar <
> > manikumar.re...@gmail.com>
> > > > > wrote:
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > On Tue, Jan 30, 2018 at 11:50 AM, Ewen Cheslack-Postava <
> > > > > e...@confluent.io>
> > > > > > wrote:
> > > > > >
> > > > > >> +1 (binding)
> > > > > >>
> > > > > >> On Fri, Jan 26, 2018 at 9:16 AM, Colin McCabe <
> cmcc...@apache.org
> > >
> > > > > wrote:
> > > > > >>
> > > > > >> > +1 (non-binding)
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> > On Fri, Jan 26, 2018, at 08:29, Ted Yu wrote:
> > > > > >> > > +1
> > > > > >> > >
> > > > > >> > > On Fri, Jan 26, 2018 at 7:00 AM, Brandon Kirchner <
> > > > > >> > > brandon.kirch...@gmail.com> wrote:
> > > > > >> > >
> > > > > >> > > > Hi all,
> > > > > >> > > >
> > > > > >> > > > I would like to (re)start the voting process for KIP-206:
> > > > > >> > > >
> > > > > >> > > > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > >> > > > 206%3A+Add+support+for+UUID+serialization+and+
> > deserialization
> > > > > >> > > >  > > > > >> > > > 206%3A+Add+support+for+UUID+serialization+and+
> > > deserialization>*
> > > > > >> > > >
> > > > > >> > > > The KIP adds a UUID serializer and deserializer. Possible
> > > > > >> > implementation
> > > > > >> > > > can be seen here --
> > > > > >> > > >
> > > > > >> > > > https://github.com/apache/kafka/pull/4438
> > > > > >> > > >
> > > > > >> > > > Original discussion and voting thread can be seen here --
> > > > > >> > > > http://search-hadoop.com/m/Kafka/uyzND1dlgePJY7l9?subj=+
> > > > > >> > > > DISCUSS+KIP+206+Add+support+for+UUID+serialization+and+
> > > > > >> deserialization
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > Thanks!
> > > > > >> > > > Brandon K.
> > > > > >> > > >
> > > > > >> >
> > > > > >>
> > > > >
> > >
> >
>

Re: [VOTE] KIP-292: Add KTable#transformValues() method in Kafka Streams DSL

[Damian Guy]

+1

On Fri, 11 May 2018 at 14:30 Matthias J. Sax  wrote:

> +1 (binding)
>
>
> On 5/11/18 2:17 PM, Ted Yu wrote:
> > +1
> >
> > On Fri, May 11, 2018 at 2:16 PM, Guozhang Wang 
> wrote:
> >
> >> +1 (binding)
> >>
> >> Also slightly changed the title beyond changing [DISCUSS] to [VOTE]
> since
> >> Gmail will collapse these two threads into one, so people may not
> realize
> >> there is a voting started already.
> >>
> >>
> >> Guozhang
> >>
> >>
> >> On Fri, May 11, 2018 at 2:07 PM, Andy Coates  wrote:
> >>
> >>> Hi all,
> >>>
> >>> I would like to start the vote on KIP-292: Add transformValues() method
> >> to
> >>> KTable
> >>>
> >>> The link to this KIP is here:
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>> 292%3A+Add+transformValues%28%29+method+to+KTable
> >>>
> >>> The discussion thread is here:
> >>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201805.mbox/%
> >>> 3CCAO1RfpHyhEEwkwad5yxd0rLgPEK-PJ4y0%2B0y%3DwidHP0uOtToJQ%
> >>> 40mail.gmail.com%3E
> >>>
> >>> Thanks,
> >>>
> >>> Andy
> >>>
> >>
> >>
> >>
> >> --
> >> -- Guozhang
> >>
> >
>
>

Re: [VOTE] KIP-244: Add Record Header support to Kafka Streams

[Damian Guy]

Thanks. +1 (binding)

On Tue, 15 May 2018 at 01:04 Jorge Esteban Quilcate Otoya <
quilcate.jo...@gmail.com> wrote:

> @Guozhang added. Thanks!
>
> El mar., 15 may. 2018 a las 5:50, Matthias J. Sax ( >)
> escribió:
>
> > +1 (binding)
> >
> > Thanks a lot for the KIP!
> >
> > -Matthias
> >
> > On 5/14/18 10:17 AM, Guozhang Wang wrote:
> > > +1 from me
> > >
> > > One more comment on the wiki: while reviewing the PR I realized that
> in `
> > > MockProcessorContext.java
> > > <
> >
> https://github.com/apache/kafka/pull/4955/files#diff-d5440e7338f775230019a86e6bcacccb
> > >`
> > > we are also adding one additional API plus modifying the existing
> > > `setRecordMetadata` API. Since this class is part of the public
> > test-utils
> > > package we should claim it in the wiki as well.
> > >
> > >
> > > Guozhang
> > >
> > > On Mon, May 14, 2018 at 8:43 AM, Ted Yu  wrote:
> > >
> > >> +1
> > >>
> > >> On Mon, May 14, 2018 at 8:31 AM, Jorge Esteban Quilcate Otoya <
> > >> quilcate.jo...@gmail.com> wrote:
> > >>
> > >>> Hi everyone,
> > >>>
> > >>> I would like to start a vote on KIP-244: Add Record Header support to
> > >> Kafka
> > >>> Streams
> > >>>
> > >>> KIP wiki page:
> > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >>> 244%3A+Add+Record+Header+support+to+Kafka+Streams+Processor+API
> > >>>
> > >>> The discussion thread is here:
> > >>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201805.
> > >>> mbox/%3CCAC3UcJvrgcBfe6%3DiW6%2BuTWsLB%2B4CsHgRmDx9TvCzJQrWvfg7_w%
> > >>> 40mail.gmail.com%3E
> > >>>
> > >>> Cheers,
> > >>> Jorge.
> > >>>
> > >>
> > >
> > >
> > >
> >
> >
>

Re: [VOTE] KIP-295: Add Streams Configuration Allowing for Optional Topology Optimization

[Damian Guy]

+1
On Tue, 15 May 2018 at 15:04, Ted Yu  wrote:

> +1
>  Original message From: Guozhang Wang 
> Date: 5/15/18  2:34 PM  (GMT-08:00) To: dev@kafka.apache.org Subject: Re:
> [VOTE] KIP-295: Add Streams Configuration Allowing for Optional Topology
> Optimization
> +1 (binding).
>
> On Tue, May 15, 2018 at 2:16 PM, Matthias J. Sax 
> wrote:
>
> > +1 (binding)
> >
> >
> > On 5/15/18 1:45 PM, Bill Bejeck wrote:
> > > Hi all,
> > >
> > > I'd like to start a vote on KIP-295: Add Streams Configuration Allowing
> > for
> > > Optional Topology Optimization.
> > >
> > > KIP wiki page:
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 295%3A+Add+Streams+Configuration+Allowing+for+
> > Optional+Topology+Optimization
> > >
> > > Discussion thread:
> > > https://www.mail-archive.com/dev@kafka.apache.org/msg87593.html
> > >
> > > Thanks,
> > > Bill
> > >
> >
> >
>
>
> --
> -- Guozhang
>

Re: [DISCUSS] KIP-303: Add Dynamic Routing in Streams Sink

[Damian Guy]

Overall i'm a +1 on this, but i'm not a big fan of using the KeyValueMapper
to choose the topic. It is a bit counter-intuitve to me. I'd prefer to add
a class specifically for it and possibly pass in the RecordContext

On Wed, 16 May 2018 at 13:22 Guozhang Wang  wrote:

> Hello folks,
>
> Please let me know if you have further feedbacks; if there is no more
> feedbacks I'm going to start the voting thread soon.
>
>
> Guozhang
>
>
> On Wed, May 16, 2018 at 8:31 AM, Guozhang Wang  wrote:
>
> > I have thought about exposing record context as well, and in the end I
> > decided to piggy-back it with KIP-159. And if we want to indeed reuse the
> > class it will be:
> >
> > ```
> > public interface RichKeyValueMapper {
> > VR apply(final K key, final V value, final RecordContext
> > recordContext);
> > }
> > ```
> >
> >
> >
> > Guozhang
> >
> >
> > On Tue, May 15, 2018 at 10:04 PM, Matthias J. Sax  >
> > wrote:
> >
> >> Just my 2 cents:
> >>
> >> I am fine with `KeyValueMapper` (+1 for code reusage) -- the JavaDocs
> >> will explain what the `KeyValueMapper` is supposed to do, ie, extract
> >> and return the sink topic name from the key-value pair.
> >>
> >> A side remark though: do we think that accessing key/value is
> >> sufficient? Or should we provide access to the full metadata? We could
> >> also do this with KIP-159 of course -- but this would come earliest in
> >> 2.1. As an alternative we could add a `TopicNameExtractor` to expose the
> >> whole record context. The advantage would be, that we don't need to
> >> change it via KIP-159 later again. WDYT?
> >>
> >> -Matthias
> >>
> >> On 5/15/18 5:57 PM, Bill Bejeck wrote:
> >> > Thanks for the KIP Guozhang, it's a +1 for me.
> >> >
> >> > As for re-using the KeyValueMapper for choosing the topic, I am on the
> >> > fence, a more explicitly named class would be more clear, but I'm not
> >> sure
> >> > it's worth a new class that will primarily perform the same actions as
> >> the
> >> > KeyValueMapper.
> >> >
> >> > Thanks,
> >> > Bill
> >> >
> >> > On Tue, May 15, 2018 at 5:52 PM, Guozhang Wang 
> >> wrote:
> >> >
> >> >> Hello John:
> >> >>
> >> >> * As for the type superclass, it is mainly for allowing super class
> >> serdes.
> >> >> More details can be found here:
> >> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >> >> 100+-+Relax+Type+constraints+in+Kafka+Streams+API
> >> >>
> >> >> * I may have slight preference on reusing existing classes but I
> think
> >> most
> >> >> of my rationales are quite subjective. Personally I do not find `self
> >> >> documenting` worth a new class but I can be convinced if people have
> >> other
> >> >> motivations doing it :)
> >> >>
> >> >>
> >> >> Guozhang
> >> >>
> >> >>
> >> >> On Tue, May 15, 2018 at 11:19 AM, John Roesler 
> >> wrote:
> >> >>
> >> >>> Thanks for the KIP, Guozhang.
> >> >>>
> >> >>> It looks good overall to me; I just have one question:
> >> >>> * Why do we bound the generics of KVMapper in KStream to be
> >> >> superclass-of-K
> >> >>> and superclass-of-V instead of exactly K and V, as in Topology? I
> >> might
> >> >> be
> >> >>> thinking about it wrong, but that seems backwards to me. If
> anything,
> >> >>> bounding to be a subclass-of-K or subclass-of-V would seem right to
> >> me.
> >> >>>
> >> >>> One extra thought: I agree that KVMapper
> >> is an
> >> >>> applicable type for extracting the topic name, but I wonder what the
> >> >> value
> >> >>> of reusing the KVMapper for this purpose is. Would defining a new
> >> class,
> >> >>> say TopicNameExtractor, provide the same functionality while
> >> being a
> >> >>> bit more self-documenting?
> >> >>>
> >> >>> Thanks,
> >> >>> -John
> >> >>>
> >> >>> On Tue, May 15, 2018 at 12:32 AM, Guozhang Wang  >
> >> >>> wrote:
> >> >>>
> >>  Hello folks,
> >> 
> >>  I'd like to start a discussion on adding dynamic routing
> >> functionality
> >> >> in
> >>  Streams sink node. I.e. users do not need to specify the topic name
> >> at
> >>  compilation time but can dynamically determine which topic to send
> to
> >> >>> based
> >>  on each record's key value pairs. Please find a KIP here:
> >> 
> >>  https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>  303%3A+Add+Dynamic+Routing+in+Streams+Sink
> >> 
> >>  Any feedbacks are highly appreciated.
> >> 
> >>  Thanks!
> >> 
> >>  -- Guozhang
> >> 
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> -- Guozhang
> >> >>
> >> >
> >>
> >>
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> -- Guozhang
>

Re: [DISCUSS] KIP-303: Add Dynamic Routing in Streams Sink

[Damian Guy]

Hi Guozhang, yes i think it would make sense to add this now as having the
additional record context would be valuable. Though i'm happy either way.

On Wed, 16 May 2018 at 23:07 Guozhang Wang  wrote:

> Hi Damian,
>
> My current plan is to add the "RichKeyValueMapper" when getting in KIP-159
> to include the record context in this dynamic routing feature. So just to
> clarify: are you more concerning that if we are going to do that anyways in
> the future, we should not add overloaded functions with "KeyValueMapper" as
> of now since they will be subsumed soon?
>
>
> Guozhang
>
>
> On Wed, May 16, 2018 at 2:59 PM, Damian Guy  wrote:
>
>> Overall i'm a +1 on this, but i'm not a big fan of using the
>> KeyValueMapper
>> to choose the topic. It is a bit counter-intuitve to me. I'd prefer to add
>> a class specifically for it and possibly pass in the RecordContext
>>
>> On Wed, 16 May 2018 at 13:22 Guozhang Wang  wrote:
>>
>> > Hello folks,
>> >
>> > Please let me know if you have further feedbacks; if there is no more
>> > feedbacks I'm going to start the voting thread soon.
>> >
>> >
>> > Guozhang
>> >
>> >
>> > On Wed, May 16, 2018 at 8:31 AM, Guozhang Wang 
>> wrote:
>> >
>> > > I have thought about exposing record context as well, and in the end I
>> > > decided to piggy-back it with KIP-159. And if we want to indeed reuse
>> the
>> > > class it will be:
>> > >
>> > > ```
>> > > public interface RichKeyValueMapper {
>> > > VR apply(final K key, final V value, final RecordContext
>> > > recordContext);
>> > > }
>> > > ```
>> > >
>> > >
>> > >
>> > > Guozhang
>> > >
>> > >
>> > > On Tue, May 15, 2018 at 10:04 PM, Matthias J. Sax <
>> matth...@confluent.io
>> > >
>> > > wrote:
>> > >
>> > >> Just my 2 cents:
>> > >>
>> > >> I am fine with `KeyValueMapper` (+1 for code reusage) -- the JavaDocs
>> > >> will explain what the `KeyValueMapper` is supposed to do, ie, extract
>> > >> and return the sink topic name from the key-value pair.
>> > >>
>> > >> A side remark though: do we think that accessing key/value is
>> > >> sufficient? Or should we provide access to the full metadata? We
>> could
>> > >> also do this with KIP-159 of course -- but this would come earliest
>> in
>> > >> 2.1. As an alternative we could add a `TopicNameExtractor` to expose
>> the
>> > >> whole record context. The advantage would be, that we don't need to
>> > >> change it via KIP-159 later again. WDYT?
>> > >>
>> > >> -Matthias
>> > >>
>> > >> On 5/15/18 5:57 PM, Bill Bejeck wrote:
>> > >> > Thanks for the KIP Guozhang, it's a +1 for me.
>> > >> >
>> > >> > As for re-using the KeyValueMapper for choosing the topic, I am on
>> the
>> > >> > fence, a more explicitly named class would be more clear, but I'm
>> not
>> > >> sure
>> > >> > it's worth a new class that will primarily perform the same
>> actions as
>> > >> the
>> > >> > KeyValueMapper.
>> > >> >
>> > >> > Thanks,
>> > >> > Bill
>> > >> >
>> > >> > On Tue, May 15, 2018 at 5:52 PM, Guozhang Wang > >
>> > >> wrote:
>> > >> >
>> > >> >> Hello John:
>> > >> >>
>> > >> >> * As for the type superclass, it is mainly for allowing super
>> class
>> > >> serdes.
>> > >> >> More details can be found here:
>> > >> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > >> >> 100+-+Relax+Type+constraints+in+Kafka+Streams+API
>> > >> >>
>> > >> >> * I may have slight preference on reusing existing classes but I
>> > think
>> > >> most
>> > >> >> of my rationales are quite subjective. Personally I do not find
>> `self
>> > >> >> documenting` worth a new class but I can be convinced if people
>> have
>> > >> other
>> > >> >> motivations doing it :)

Re: [VOTE] KIP-303: Add Dynamic Routing Support in Kafka Streams' Topology Sink

[Damian Guy]

Thanks Guozhang. +1

Note: the link to the KIP is broken

On Thu, 17 May 2018 at 11:38 Guozhang Wang  wrote:

> Hello folks,
>
> I'd like to start a voting thread on adding dynamic routing functionality
> in Streams sink node. Please find a KIP here:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 303%3A+Add+Dynamic+Routing+in+Streams+Sink
>
>
> And the PR itself ready for review as well under KAFKA-4936:
>
> https://github.com/apache/kafka/pull/5018
>
>
>
> Thanks!
> -- Guozhang
>

Re: [VOTE] KIP-277 - Fine Grained ACL for CreateTopics API

[Damian Guy]

+1 (binding)

Thanks

On Mon, 21 May 2018 at 11:26 Edoardo Comar  wrote:

> Hi, KIP freeze is tomorrow. Anyone else would like to cast their vote ?
>
> voting so far :
> binding: Guozhang, Rajini
> non-binding: Ted, Stephane, Vahid, Colin, Manikumar, Matt
>
> On 18 May 2018 at 09:38, Edoardo Comar  wrote:
> > bumping again as this KIP is still missing one binding vote ...
> >
> > voting so far :
> > binding: Guozhang, Rajini
> > non-binding: Ted, Stephane, Vahid, Colin, Manikumar, Matt
> >
> > Edo
> >
> > On 17 May 2018 at 03:15, Matt Farmer  wrote:
> >> +1 (non-binding)
> >>
> >> On Tue, May 15, 2018 at 4:26 AM, Edoardo Comar 
> wrote:
> >>
> >>> Hi,
> >>> bumping the thread as the current vote count for this KIP is
> >>> 2 binding +1
> >>> 5 non-binding +1
> >>>
> >>> thanks, Edo
> >>>
> >>> On 8 May 2018 at 16:14, Edoardo Comar  wrote:
> >>> > Hi,
> >>> > bumping the thread as the current vote count for this KIP is
> >>> > 2 binding +1
> >>> > 5 non-binding +1
> >>> >
> >>> > so still missing a binding vote please
> >>> >
> >>> > thanks,
> >>> > Edo
> >>> >
> >>> >
> >>> > On 30 April 2018 at 12:49, Manikumar 
> wrote:
> >>> >>
> >>> >> +1 (non-binding)
> >>> >>
> >>> >> Thanks
> >>> >>
> >>> >> On Thu, Apr 26, 2018 at 9:59 PM, Colin McCabe 
> >>> wrote:
> >>> >>
> >>> >> > +1 (non-binding)
> >>> >> >
> >>> >> > best,
> >>> >> > Colin
> >>> >> >
> >>> >> >
> >>> >> > On Wed, Apr 25, 2018, at 02:45, Edoardo Comar wrote:
> >>> >> > > Hi,
> >>> >> > >
> >>> >> > > The discuss thread on KIP-277 (
> >>> >> > > https://www.mail-archive.com/dev@kafka.apache.org/msg86540.html
> )
> >>> >> > > seems to have been fruitful and concerns have been addressed,
> please
> >>> >> > allow
> >>> >> > > me start a vote on it:
> >>> >> > >
> >>> >> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>> >> > 277+-+Fine+Grained+ACL+for+CreateTopics+API
> >>> >> > >
> >>> >> > > I will update the small PR to the latest KIP semantics if the
> vote
> >>> >> > passes
> >>> >> > > (as I hope :-).
> >>> >> > >
> >>> >> > > cheers
> >>> >> > > Edo
> >>> >> > > --
> >>> >> > >
> >>> >> > > Edoardo Comar
> >>> >> > >
> >>> >> > > IBM Message Hub
> >>> >> > >
> >>> >> > > IBM UK Ltd, Hursley Park, SO21 2JN
> >>> >> > > Unless stated otherwise above:
> >>> >> > > IBM United Kingdom Limited - Registered in England and Wales
> with
> >>> >> > > number
> >>> >> > > 741598.
> >>> >> > > Registered office: PO Box 41, North Harbour, Portsmouth,
> Hampshire
> >>> PO6
> >>> >> > 3AU
> >>> >> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > "When the people fear their government, there is tyranny; when the
> >>> > government fears the people, there is liberty." [Thomas Jefferson]
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> "When the people fear their government, there is tyranny; when the
> >>> government fears the people, there is liberty." [Thomas Jefferson]
> >>>
> >
> >
> >
> > --
> > "When the people fear their government, there is tyranny; when the
> > government fears the people, there is liberty." [Thomas Jefferson]
>
>
>
> --
> "When the people fear their government, there is tyranny; when the
> government fears the people, there is liberty." [Thomas Jefferson]
>

Re: [VOTE] KIP-290: Support for wildcard suffixed ACLs

[Damian Guy]

+1 (binding)

On Sat, 19 May 2018 at 03:51 Piyush Vijay  wrote:

> Hi everyone,
>
> I would like to start a vote for
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-290%3A+Support+for+wildcard+suffixed+ACLs
> .
>
> The KIP proposes a way to support wildcard-suffixed resource names in Kafka
> ACLs.
>
> The main challenge was to support it in a backward compatible way because
> resources like consumer groups don't have defined naming convention and can
> have '*' in their names.
>
> Please take a look.
>
> Thanks
>
> Piyush Vijay
>

Re: [VOTE] KIP-255: OAuth Authentication via SASL/OAUTHBEARER

[Damian Guy]

+1 (binding)

Thanks

On Mon, 21 May 2018 at 04:59 Ron Dagostino  wrote:

> Hi Committers.  One more binding affirmative vote is required if KIP 255
> is to have a chance of being included in the 2.0.0 release.  Please vote
> today.
>
> Ron
>
> > On May 18, 2018, at 9:27 PM, Ron Dagostino  wrote:
> >
> > Hi committers.  KIP 255 still needs 1 more binding vote.  Currently
> there are two binding + 1 votes, from Rajini and Jun, and three non-binding
> +1 votes, from Mickael, Manikumar, and myself.
> >
> > Please vote by the Monday deadline.
> >
> > Ron
> >
> >> On Thu, May 17, 2018 at 10:59 AM, Ron Dagostino 
> wrote:
> >> Hi Jun.  I've updated the KIP to add a new section titled "Summary for
> Production Use" that includes this information along with a consolidated
> set of references to the applicable specifications.  Thanks for the
> questions.
> >>
> >> *We still need another binding vote* (currently there are two binding +
> 1 votes, from Rajini and Jun, and three non-binding +1 votes, from Mickael,
> Manikumar, and myself).
> >>
> >> Please vote before the May 22nd KIP Freeze deadline so this KIP can be
> included in the 2.0.0 release.
> >>
> >> A pull request is available and includes additional commits reflecting
> initial review comments: https://github.com/apache/kafka/pull/4994
> >>
> >> Ron
> >>
> >>> On Wed, May 16, 2018 at 8:09 PM, Jun Rao  wrote:
> >>> Hi, Ron,
> >>>
> >>> Thanks. I understand now. It may be useful to add a reference to JWT
> in the
> >>> KIP.
> >>>
> >>> Jun
> >>>
> >>> On Tue, May 15, 2018 at 6:51 PM, Ron Dagostino 
> wrote:
> >>>
> >>> > Hi Jun.  I think you are getting at the fact that OAuth 2 is a
> flexible
> >>> > framework that allows different installations to do things
> differently.  It
> >>> > is true that the principal name in Kafka could come from any claim
> in the
> >>> > token.  Most of the time it would come from the 'sub' claim, but it
> could
> >>> > certainly come from another claim, or it could be only indirectly
> based on
> >>> > a claim value (maybe certain text would be trimmed or
> prefixed/suffixed).
> >>> > The point, which I think you are getting at, is that because the
> framework
> >>> > is flexible we need to accommodate that flexibility.  The callback
> handler
> >>> > class defined by the listener.name.sasl_ssl.oauthbearer.sasl.server.
> >>> > callback.handler.class configuration value gives us the required
> >>> > flexibility.  As an example, I have an implementation that leverages
> a
> >>> > popular open source JOSE library to parse the compact serialization,
> >>> > retrieve the public key if it has not yet been retrieved, verify the
> >>> > digital signature, and map the 'sub' claim to the OAuthBearerToken's
> >>> > principal name (which becomes the SASL authorization ID, which
> becomes the
> >>> > Kafka principal name).  I could just as easily have mapped a
> different
> >>> > claim to the OAuthBearerToken's principal name, manipulated the
> 'sub' claim
> >>> > value in some way, etc.  I write the callback handler code, so I
> complete
> >>> > flexibility to do whatever my OAuth 2 installation requires me to do.
> >>> >
> >>> > Ron
> >>> >
> >>> > On Tue, May 15, 2018 at 1:39 PM, Jun Rao  wrote:
> >>> >
> >>> > > Hi, Ron,
> >>> > >
> >>> > > Thanks for the reply. I understood your answers to #2 and #3.
> >>> > >
> >>> > > For #1, will the server map all clients' principal name to the
> value
> >>> > > associated with "sub" claim? How do we support mapping different
> clients
> >>> > to
> >>> > > different principal names?
> >>> > >
> >>> > > Jun
> >>> > >
> >>> > > On Mon, May 14, 2018 at 7:02 PM, Ron Dagostino 
> >>> > wrote:
> >>> > >
> >>> > > > Hi Jun.  Thanks for the +1 vote.
> >>> > > >
> >>> > > > Regarding the first question about token claims, yes, you have it
> >>> > correct
> >>> > > > about translating the OAuth token to a principle name via a JAAS
> module
> >>> > > > option in the default unsecured case.  Specifically, the OAuth
> SASL
> >>> > > Server
> >>> > > > implementation is responsible for setting the authorization ID,
> and it
> >>> > > gets
> >>> > > > the authorization ID from the OAuthBearerToken's principalName()
> >>> > method.
> >>> > > > The listener.name.sasl_ssl.oauthbearer.sasl.server.
> >>> > > callback.handler.class
> >>> > > > is responsible for handling an instance of
> OAuthBearerValidatorCallback
> >>> > > to
> >>> > > > accept a token compact serialization from the client and return
> an
> >>> > > instance
> >>> > > > of OAuthBearerToken (assuming the compact serialization
> validates), and
> >>> > > in
> >>> > > > the default unsecured case the builtin unsecured validator
> callback
> >>> > > handler
> >>> > > > defines the OAuthBearerToken.principalName() method to return the
> >>> > 'sub'
> >>> > > > claim value by default (with the actual claim it uses being
> >>> > configurable
> >>> > > > via the unsecuredValidatorPrincipalClaimName JAAS module
> option).  So
> >>> > > that
> >>> > > > is h

Re: [VOTE] KIP-303: Add Dynamic Routing Support in Kafka Streams' Topology Sink

[Damian Guy]

+1

On Tue, 22 May 2018 at 00:12, Gwen Shapira  wrote:

> Recasting my +1
>
> On Mon, May 21, 2018 at 3:38 PM, Guozhang Wang  wrote:
>
> > Thanks Matthias, will do.
> >
> > On Mon, May 21, 2018 at 3:35 PM, Matthias J. Sax 
> > wrote:
> >
> > > Thanks. I am fine with changing the `StreamPartitioner` interface
> > directly.
> > >
> > > Can you add the idea bout `DynamicStreamPartitioner` to "rejected
> > > alternative" section. Thx.
> > >
> > >
> > > recasting +1 (binding)
> > >
> > >
> > > -Matthias
> > >
> > >
> > > On 5/21/18 3:04 PM, Guozhang Wang wrote:
> > > > Hello Matthias,
> > > >
> > > > I've tried it out on the PR, the implementation should be fine but
> one
> > > > concern I had is that, as you may also realize users of
> > > > DynamicStreamPartitioner needs to implement two interface functions,
> > with
> > > > and without the topic name if it is extending from StreamPartitioner;
> > we
> > > > could also let it to not extend from StreamPartition so it has one
> > > function
> > > > only but then we'd need Produced to have two functions allowing
> > > > StreamPartitioner and DynamicStreamPartitioner. Thinking about the
> pros
> > > and
> > > > cons I'm think it may be better to just change the interface of
> > > > StreamPartitioner itself, since even without dynamic routing,
> allowing
> > > the
> > > > topic name could let users to give one partitioner implementation
> that
> > > > branch on the topic names other than having one partitioner per
> topic.
> > > >
> > > >
> > > > Guozhang
> > > >
> > > >
> > > > On Mon, May 21, 2018 at 11:56 AM, Matthias J. Sax <
> > matth...@confluent.io
> > > >
> > > > wrote:
> > > >
> > > >> I think that the risk of the change is moderate as I expect most
> > people
> > > >> to use the DefaultStreamPartitioner.
> > > >>
> > > >> However, there would still be possibility to define a new interface
> > > >> instead of changing the old:
> > > >>
> > > >>> public interface DynamicStreamPartitioner {
> > > >>> Integer partition(String topic, K key, V value, int
> > numPartitions);
> > > >>> }
> > > >>
> > > >> The newly added methods `Topology#addSink` and `KStream#to` would
> take
> > > >> this new interface instead of the old.
> > > >>
> > > >> Maybe `DynamicStreamPartitioner` must extend `StreamPartitioner` to
> > make
> > > >> runtime code work though...
> > > >>
> > > >> WDYT?
> > > >>
> > > >> -Matthias
> > > >>
> > > >> On 5/21/18 11:47 AM, Guozhang Wang wrote:
> > > >>> Hello everyone,
> > > >>>
> > > >>> While implementing the PR for this KIP I realized there is once
> place
> > > >> which
> > > >>> we should consider modifying on public APIs as well:
> > > >>> StreamPartitioner#partition, to add the topic name string. Note it
> > will
> > > >> be
> > > >>> a incompatible change that requires users who have customized
> > > >>> StreamPartitioner implementations.
> > > >>>
> > > >>> I've updated the wiki page of KIP-303, please recast your vote on
> > this
> > > >>> thread. Thanks!!!
> > > >>>
> > > >>>
> > > >>> Guozhang
> > > >>>
> > > >>>
> > > >>> On Thu, May 17, 2018 at 3:15 PM, John Roesler 
> > > wrote:
> > > >>>
> > >  +1 non-binding
> > > 
> > >  On Thu, May 17, 2018 at 4:44 PM, Matthias J. Sax <
> > > matth...@confluent.io
> > > >>>
> > >  wrote:
> > > 
> > > > +1 (binding)
> > > >
> > > >
> > > > On 5/17/18 12:18 PM, Ted Yu wrote:
> > > >> +1
> > > >>  Original message From: Gwen Shapira <
> > >  g...@confluent.io>
> > > > Date: 5/17/18  11:53 AM  (GMT-08:00) To: dev <
> dev@kafka.apache.org
> > >
> > > > Subject: Re: [VOTE] KIP-303: Add Dynamic Routing Support in Kafka
> > >  Streams'
> > > > Topology Sink
> > > >> Yay, its about time :)
> > > >>
> > > >> +1
> > > >>
> > > >> On Thu, May 17, 2018 at 12:38 PM, Guozhang Wang <
> > wangg...@gmail.com
> > > >
> > > > wrote:
> > > >>
> > > >>> Hello folks,
> > > >>>
> > > >>> I'd like to start a voting thread on adding dynamic routing
> > > > functionality
> > > >>> in Streams sink node. Please find a KIP here:
> > > >>>
> > > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > >>> 303%3A+Add+Dynamic+Routing+in+Streams+Sink
> > > >>>
> > > >>>
> > > >>> And the PR itself ready for review as well under KAFKA-4936:
> > > >>>
> > > >>> https://github.com/apache/kafka/pull/5018
> > > >>>
> > > >>>
> > > >>>
> > > >>> Thanks!
> > > >>> -- Guozhang
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >
> > > >
> > > 
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >>
> > > >
> > > >
> > >
> > >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter  | blog
> 
>

Re: [DISCUSS] KIP-307: Allow to define custom processor names with KStreams DSL

[Damian Guy]

Hi Florian,

Thanks for the KIP. What about KTable and other DSL interfaces? Will they
not want to be able to do the same thing?
It would be good to see a complete set of the public API changes.

Cheers,
Damian

On Wed, 30 May 2018 at 19:45 Guozhang Wang  wrote:

> Hello Florian,
>
> Thanks for the KIP. I have some meta feedbacks on the proposal:
>
> 1. You mentioned that this `Processed` object will be added to a new
> overloaded variant of all the stateless operators, what about the stateful
> operators? Would like to hear your opinions if you have thought about that:
> note for stateful operators they will usually be mapped to multiple
> processor node names, so we probably need to come up with some ways to
> define all their names.
>
> 2. I share the same concern with Bill as for adding lots of new overload
> functions into the stateless operators, as we have just spent quite some
> effort in trimming them since 1.0.0 release. If the goal is to just provide
> some "hints" on the generated processor node names, not strictly enforcing
> the exact names that to be generated, then how about we just add a new
> function to `KStream` and `KTable` classes like: "as(Processed)", with the
> semantics as "the latest operators that generate this KStream / KTable will
> be named accordingly to this hint".
>
> The only caveat, is that for all operators like `KStream#to` and
> `KStream#print` that returns void, this alternative would not work. But for
> the current operators:
>
> a. KStream#print,
> b. KStream#foreach,
> c. KStream#to,
> d. KStream#process
>
> I personally felt that except `KStream#process` users would not usually
> bother to override their names, and for `KStream#process` we could add an
> overload variant with the additional Processed object.
>
>
> 3. In your example, the processor names are still added with a suffix like
> "
> -00", is this intentional? If yes, why (I thought with user
> specified processor name hints we will not add suffix to distinguish
> different nodes of the same type any more)?
>
>
> Guozhang
>
>
> On Tue, May 29, 2018 at 6:47 AM, Bill Bejeck  wrote:
>
> > Hi Florian,
> >
> > Thanks for the KIP.  I think being able to add more context to the
> > processor names would be useful.
> >
> > I like the idea of adding a "withProcessorName" to Produced, Consumed and
> > Joined.
> >
> > But instead of adding the "Processed" parameter to a large percentage of
> > the methods, which would result in overloaded methods (which we removed
> > quite a bit with KIP-182) what do you think of adding a method
> > to the AbstractStream class "withName(String processorName)"? BTW I"m not
> > married to the method name, it's the best I can do off the top of my
> head.
> >
> > For the methods that return void, we'd have to add a parameter, but that
> > would at least cut down on the number of overloaded methods in the API.
> >
> > Just my 2 cents.
> >
> > Thanks,
> > Bill
> >
> > On Sun, May 27, 2018 at 4:13 PM, Florian Hussonnois <
> fhussonn...@gmail.com
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > I would like to start a new discussion on following KIP :
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 307%3A+Allow+to+define+custom+processor+names+with+KStreams+DSL
> > >
> > > This is still a draft.
> > >
> > > Looking forward for your feedback.
> > > --
> > > Florian HUSSONNOIS
> > >
> >
>
>
>
> --
> -- Guozhang
>

Re: [DISCUSS] KIP-313: Add KStream.flatTransform and KStream.flatTransformValues

[Damian Guy]

Thanks for the KIP. LGTM

On Mon, 11 Jun 2018 at 00:44 Matthias J. Sax  wrote:

> Thanks for the KIP.
>
> I don't have any comments at this pint. Overall I am +1 on the KIP.
>
>
> -Matthias
>
> On 6/8/18 2:56 PM, Bruno Cadonna wrote:
> > Hi list,
> >
> > I created KIP-313 [1] for JIRA issue KAFKA-4217 [2] and I would like to
> > put the KIP up for discussion.
> >
> > Best regards,
> > Bruno
> >
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues
> >
> > [2] https://issues.apache.org/jira/browse/KAFKA-4217
> >
>
>

Re: [VOTE] KIP-312: Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties

[Damian Guy]

Thanks Bill! +1

On Mon, 25 Jun 2018 at 18:57 Ted Yu  wrote:

> +1
>
> On Mon, Jun 25, 2018 at 9:45 AM, Guozhang Wang  wrote:
>
> > +1.
> >
> > On Mon, Jun 25, 2018 at 8:12 AM, Matthias J. Sax 
> > wrote:
> >
> > > +1 (binding)
> > >
> > > On 6/25/18 6:11 AM, Bill Bejeck wrote:
> > > > All,
> > > > I'd like to start a vote for this KIP now.
> > > >
> > > > Thanks,
> > > > Bill
> > > >
> > >
> > >
> >
> >
> > --
> > -- Guozhang
> >
>

Re: [VOTE] KIP-324: Add method to get metrics() in AdminClient

[Damian Guy]

+1 (binding)

Thanks

On Wed, 27 Jun 2018 at 18:50 Bill Bejeck  wrote:

> +1
>
> -Bill
>
> On Wed, Jun 27, 2018 at 12:47 PM Manikumar 
> wrote:
>
> > +1 (non-binding)
> >
> > Thanks.
> >
> > On Wed, Jun 27, 2018 at 10:15 PM Matthias J. Sax 
> > wrote:
> >
> > > +1 (binding)
> > >
> > > On 6/26/18 2:33 PM, Guozhang Wang wrote:
> > > > +1. Thanks.
> > > >
> > > > On Tue, Jun 26, 2018 at 2:31 PM, Yishun Guan 
> > wrote:
> > > >
> > > >> Hi All,
> > > >>
> > > >> I am starting a vote on this KIP:
> > > >>
> > > >> https://cwiki.apache.org/confluence/x/lQg0BQ
> > > >>
> > > >> Thanks,
> > > >> Yishun
> > > >>
> > > >
> > > >
> > > >
> > >
> > >
> >
>

Re: [VOTE] KIP-319: Replace numSegments to segmentInterval in Streams window configurations

[Damian Guy]

+1

On Tue, 26 Jun 2018 at 17:22 Bill Bejeck  wrote:

> +1
>
> On Mon, Jun 25, 2018 at 11:07 PM Matthias J. Sax 
> wrote:
>
> > +1 (binding)
> >
> > On 6/25/18 3:00 PM, Guozhang Wang wrote:
> > > +1
> > >
> > > On Mon, Jun 25, 2018 at 2:58 PM, Ted Yu  wrote:
> > >
> > >> +1
> > >>
> > >> On Mon, Jun 25, 2018 at 2:56 PM, John Roesler 
> > wrote:
> > >>
> > >>> Hello All,
> > >>>
> > >>> Thanks for the discussion on KIP-319. I'd now like to start the
> voting.
> > >>>
> > >>> As a reminder, KIP-319 proposes a fix to an issue I identified in
> > >>> KAFKA-7080. Specifically, the issue is that we're creating
> > >>> CachingWindowStore with the *number of segments* instead of the
> > *segment
> > >>> size*.
> > >>>
> > >>> Here's the jira: https://issues.apache.org/jira/browse/KAFKA-7080
> > >>> Here's the KIP: https://cwiki.apache.org/confluence/x/mQU0BQ
> > >>>
> > >>> Additionally, here's a draft PR for clarity:
> > >>> https://github.com/apache/kafka/pull/5257
> > >>>
> > >>> Thanks,
> > >>> -John
> > >>>
> > >>
> > >
> > >
> > >
> >
> >
>

Re: [VOTE] KIP-330: Add retentionPeriod in SessionBytesStoreSupplier

[Damian Guy]

+1

On Thu, 28 Jun 2018 at 02:16 Ted Yu  wrote:

> +1
>
> On Wed, Jun 27, 2018 at 4:40 PM, Bill Bejeck  wrote:
>
> > +1
> >
> > -Bill
> >
> > On Wed, Jun 27, 2018 at 7:39 PM Guozhang Wang 
> wrote:
> >
> > > Hello folks,
> > >
> > > I'd like to start a voting thread on KIP-330. I've intentionally
> skipped
> > > the discuss phase since it is a pretty straight-forward public API
> change
> > > and should actually be added since day one. The bug fix of KAFKA-7071
> > > helped us to discover this overlook.
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>

Re: [VOTE] KIP-313: Add KStream.flatTransform and KStream.flatTransformValues

[Damian Guy]

+1

On Mon, 25 Jun 2018 at 02:16 Matthias J. Sax  wrote:

> +1 (binding)
>
>
> -Matthias
>
> On 6/22/18 10:25 AM, Bill Bejeck wrote:
> > Thanks for the KIP, +1.
> >
> > -Bill
> >
> > On Fri, Jun 22, 2018 at 1:08 PM Ted Yu  wrote:
> >
> >> +1
> >>
> >> On Fri, Jun 22, 2018 at 2:50 AM, Bruno Cadonna 
> wrote:
> >>
> >>> Hi list,
> >>>
> >>> I would like to voting on this KIP.
> >>>
> >>> I created a first PR[1] that adds flatTransform. Once I get some
> >>> feedback, I will start work on flatTransformValues.
> >>>
> >>> Best regards,
> >>> Bruno
> >>>
> >>> [1] https://github.com/apache/kafka/pull/5273
> >>>
> >>
> >
>
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Damian Guy]

; Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to act as a stream processor,
>
> consuming an input stream from one or more topics and producing an
>
> output stream to one or more output topics, effectively transforming the
>
> input streams to output streams.
>
>
>
> ** The Connector API allows building and running reusable producers or
>
> consumers that connect Kafka topics to existing applications or data
>
> systems. For example, a connector to a relational database might
>
> capture every change to a table.
>
>
>
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
>
>
> ** Building real-time streaming data pipelines that reliably get data
>
> between systems or applications.
>
>
>
> ** Building real-time streaming applications that transform or react
>
> to the streams of data.
>
>
>
>
>
>
>
> Apache Kafka is in use at large and small companies worldwide, including
>
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
>
>
>
>
>
> A big thank you for the following 131 contributors to this release!
>
>
>
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
>
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>
> Yaswanth Kumar, ying-zheng, Yu
>
>
>
>
>
>
>
> We welcome your help and feedback. For more information on how to
>
> report problems, and to get involved, visit the project website at
>
> https://kafka.apache.org/
>
>
>
>
>
> Thank you!
>
>
>
>
>
> Regards,
>
>
>
> Rajini
>

Re: [Vote] KIP-321: Update TopologyDescription to better represent Source and Sink Nodes

[Damian Guy]

Hi Nishanth,

I have one nit on the KIP. I think the topicNameExtractor method should
return Optional rather than null.
Sorry I'm late here.

Thanks,
Damian

On Tue, 31 Jul 2018 at 01:14 Nishanth Pradeep  wrote:

> We need one more binding vote.
>
> Binding Votes:
>
>- Matthias J. Sax
>- Guozhang Wong
>
> Community Votes:
>
>- Bill Bejeck
>- Ted Yu
>
> Best,
> Nishanth Pradeep
>
> On Fri, Jul 27, 2018 at 10:02 AM Bill Bejeck  wrote:
>
> > Thanks for the KIP!
> >
> > +1
> >
> > -Bill
> >
> > On Thu, Jul 26, 2018 at 2:39 AM Guozhang Wang 
> wrote:
> >
> > > +1
> > >
> > > On Wed, Jul 25, 2018 at 11:13 PM, Matthias J. Sax <
> matth...@confluent.io
> > >
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > -Matthias
> > > >
> > > > On 7/25/18 7:47 PM, Ted Yu wrote:
> > > > > +1
> > > > >
> > > > > On Wed, Jul 25, 2018 at 7:24 PM Nishanth Pradeep <
> > > nishanth...@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Hello,
> > > > >>
> > > > >> I'm calling a vote for KIP-321:
> > > > >>
> > > > >>
> > > > >>
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-321%3A+Update+
> > > > TopologyDescription+to+better+represent+Source+and+Sink+Nodes
> > > > >>
> > > > >> Best,
> > > > >> Nishanth Pradeep
> > > > >>
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>

Re: [Vote] KIP-321: Update TopologyDescription to better represent Source and Sink Nodes

[Damian Guy]

Ewen - no as I don't believe they are never null. Whereas the
topicNameExtractor method returns null if it is the default extractor or
the extractor. So i think this would be better to be optional as it is
optionally returning a TopicNameExtractor

On Tue, 31 Jul 2018 at 23:01 Ewen Cheslack-Postava 
wrote:

> Generally +1 (binding)
>
> It would be helpful to just provide the full, updated interfaces in the
> KIP and mark things as new with comments if needed. I had to go back and
> read the discussion thread to make sure I was understanding the intent
> correctly.
>
> Damian -- if we make that Optional, shouldn't the methods on Source also
> be Optional types?
>
> -Ewen
>
> On Mon, Jul 30, 2018 at 11:13 PM Damian Guy  wrote:
>
>> Hi Nishanth,
>>
>> I have one nit on the KIP. I think the topicNameExtractor method should
>> return Optional rather than null.
>> Sorry I'm late here.
>>
>> Thanks,
>> Damian
>>
>> On Tue, 31 Jul 2018 at 01:14 Nishanth Pradeep 
>> wrote:
>>
>> > We need one more binding vote.
>> >
>> > Binding Votes:
>> >
>> >- Matthias J. Sax
>> >- Guozhang Wong
>> >
>> > Community Votes:
>> >
>> >- Bill Bejeck
>> >- Ted Yu
>> >
>> > Best,
>> > Nishanth Pradeep
>> >
>> > On Fri, Jul 27, 2018 at 10:02 AM Bill Bejeck  wrote:
>> >
>> > > Thanks for the KIP!
>> > >
>> > > +1
>> > >
>> > > -Bill
>> > >
>> > > On Thu, Jul 26, 2018 at 2:39 AM Guozhang Wang 
>> > wrote:
>> > >
>> > > > +1
>> > > >
>> > > > On Wed, Jul 25, 2018 at 11:13 PM, Matthias J. Sax <
>> > matth...@confluent.io
>> > > >
>> > > > wrote:
>> > > >
>> > > > > +1 (binding)
>> > > > >
>> > > > > -Matthias
>> > > > >
>> > > > > On 7/25/18 7:47 PM, Ted Yu wrote:
>> > > > > > +1
>> > > > > >
>> > > > > > On Wed, Jul 25, 2018 at 7:24 PM Nishanth Pradeep <
>> > > > nishanth...@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > >> Hello,
>> > > > > >>
>> > > > > >> I'm calling a vote for KIP-321:
>> > > > > >>
>> > > > > >>
>> > > > > >>
>> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-321%3A+Update+
>> > > > > TopologyDescription+to+better+represent+Source+and+Sink+Nodes
>> > > > > >>
>> > > > > >> Best,
>> > > > > >> Nishanth Pradeep
>> > > > > >>
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > -- Guozhang
>> > > >
>> > >
>> >
>>
>

Re: [Vote] KIP-321: Update TopologyDescription to better represent Source and Sink Nodes

[Damian Guy]

You have 3 binding votes, so i'll defer to the others.

On Thu, 2 Aug 2018 at 04:41 Nishanth Pradeep  wrote:

> The only issue I see with this is that Sink#topic would also need to be
> Optional as was pointed out already. Since Sink#topic is a preexisting
> method, changing its return type would break backwards compatibility.
>
> On the other hand, it might be worth it to rip that bandaid now.
>
> Best,
> Nishanth Pradeep
>
> On Wed, Aug 1, 2018 at 11:56 AM Guozhang Wang  wrote:
>
>> For source node, only one of `Set topicsSet` and `TopicPattern
>> topicPattern()` will be specified by the user. Similarly for sink node,
>> only one of `String` and `TopicNameExtractor` will be specified by the
>> user. Although I've not seen Nishanth's updated PR, I think when it is not
>> specified today we will return null in that case.
>>
>> If we want to improve on this situation with Optional, we'd need to do it
>> on all of these functions. Also note that for `Source#toString()` and
>> `Sink#toString()` we should only include the specified field in the
>> resulted representation.
>>
>>
>> Guozhang
>>
>> On Wed, Aug 1, 2018 at 5:08 AM, Damian Guy  wrote:
>>
>> > Ewen - no as I don't believe they are never null. Whereas the
>> > topicNameExtractor method returns null if it is the default extractor or
>> > the extractor. So i think this would be better to be optional as it is
>> > optionally returning a TopicNameExtractor
>> >
>> > On Tue, 31 Jul 2018 at 23:01 Ewen Cheslack-Postava 
>> > wrote:
>> >
>> > > Generally +1 (binding)
>> > >
>> > > It would be helpful to just provide the full, updated interfaces in
>> the
>> > > KIP and mark things as new with comments if needed. I had to go back
>> and
>> > > read the discussion thread to make sure I was understanding the intent
>> > > correctly.
>> > >
>> > > Damian -- if we make that Optional, shouldn't the methods on Source
>> also
>> > > be Optional types?
>> > >
>> > > -Ewen
>> > >
>> > > On Mon, Jul 30, 2018 at 11:13 PM Damian Guy 
>> > wrote:
>> > >
>> > >> Hi Nishanth,
>> > >>
>> > >> I have one nit on the KIP. I think the topicNameExtractor method
>> should
>> > >> return Optional rather than null.
>> > >> Sorry I'm late here.
>> > >>
>> > >> Thanks,
>> > >> Damian
>> > >>
>> > >> On Tue, 31 Jul 2018 at 01:14 Nishanth Pradeep > >
>> > >> wrote:
>> > >>
>> > >> > We need one more binding vote.
>> > >> >
>> > >> > Binding Votes:
>> > >> >
>> > >> >- Matthias J. Sax
>> > >> >- Guozhang Wong
>> > >> >
>> > >> > Community Votes:
>> > >> >
>> > >> >- Bill Bejeck
>> > >> >- Ted Yu
>> > >> >
>> > >> > Best,
>> > >> > Nishanth Pradeep
>> > >> >
>> > >> > On Fri, Jul 27, 2018 at 10:02 AM Bill Bejeck 
>> > wrote:
>> > >> >
>> > >> > > Thanks for the KIP!
>> > >> > >
>> > >> > > +1
>> > >> > >
>> > >> > > -Bill
>> > >> > >
>> > >> > > On Thu, Jul 26, 2018 at 2:39 AM Guozhang Wang <
>> wangg...@gmail.com>
>> > >> > wrote:
>> > >> > >
>> > >> > > > +1
>> > >> > > >
>> > >> > > > On Wed, Jul 25, 2018 at 11:13 PM, Matthias J. Sax <
>> > >> > matth...@confluent.io
>> > >> > > >
>> > >> > > > wrote:
>> > >> > > >
>> > >> > > > > +1 (binding)
>> > >> > > > >
>> > >> > > > > -Matthias
>> > >> > > > >
>> > >> > > > > On 7/25/18 7:47 PM, Ted Yu wrote:
>> > >> > > > > > +1
>> > >> > > > > >
>> > >> > > > > > On Wed, Jul 25, 2018 at 7:24 PM Nishanth Pradeep <
>> > >> > > > nishanth...@gmail.com>
>> > >> > > > > > wrote:
>> > >> > > > > >
>> > >> > > > > >> Hello,
>> > >> > > > > >>
>> > >> > > > > >> I'm calling a vote for KIP-321:
>> > >> > > > > >>
>> > >> > > > > >>
>> > >> > > > > >>
>> > >> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > 321%3A+Update+
>> > >> > > > > TopologyDescription+to+better+represent+Source+and+Sink+Nodes
>> > >> > > > > >>
>> > >> > > > > >> Best,
>> > >> > > > > >> Nishanth Pradeep
>> > >> > > > > >>
>> > >> > > > > >
>> > >> > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > > >
>> > >> > > > --
>> > >> > > > -- Guozhang
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> > >
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>
>

Re: [VOTE] KIP-328: Ability to suppress updates for KTables

[Damian Guy]

Thanks John! +1

On Mon, 30 Jul 2018 at 23:58 Guozhang Wang  wrote:

> Yes, the addendum lgtm as well. Thanks!
>
> On Mon, Jul 30, 2018 at 3:34 PM, John Roesler  wrote:
>
> > Another thing that came up after I started working on an implementation
> is
> > that in addition to deprecating "retention" from the Windows interface,
> we
> > also need to deprecate "segmentInterval", for the same reasons. I simply
> > overlooked it previously. I've updated the KIP accordingly.
> >
> > Hopefully, this doesn't change anyone's vote.
> >
> > Thanks,
> > -John
> >
> > On Mon, Jul 30, 2018 at 5:31 PM John Roesler  wrote:
> >
> > > Thanks Guozhang,
> > >
> > > Thanks for that catch. to clarify, currently, events are "late" only
> when
> > > they are older than the retention period. Currently, we detect this in
> > the
> > > processor and record it as a "skipped-record". We then do not attempt
> to
> > > store the event in the window store. If a user provided a
> pre-configured
> > > window store with a retention period smaller than the one they specify
> > via
> > > Windows#until, the segmented store will drop the update with no metric
> > and
> > > record a debug-level log.
> > >
> > > With KIP-328, with the introduction of "grace period" and moving
> > retention
> > > fully into the state store, we need to have metrics for both "late
> > events"
> > > (new records older than the grace period) and "expired window events"
> > (new
> > > records for windows that are no longer retained in the state store). I
> > > already proposed metrics for the late events, and I've just updated the
> > KIP
> > > with metrics for the expired window events. I also updated the KIP to
> > make
> > > it clear that neither late nor expired events will count as
> > > "skipped-records" any more.
> > >
> > > -John
> > >
> > > On Mon, Jul 30, 2018 at 4:22 PM Guozhang Wang 
> > wrote:
> > >
> > >> Hi John,
> > >>
> > >> Thanks for the updated KIP, +1 from me, and one minor suggestion:
> > >>
> > >> Following your suggestion of the differentiation of `skipped-records`
> > v.s.
> > >> `late-event-drop`, we should probably consider moving the scenarios
> > where
> > >> records got ignored due the window not being available any more in
> > >> windowed
> > >> aggregation operators from the `skipped-records` metrics recording to
> > the
> > >> `late-event-drop` metrics recording.
> > >>
> > >>
> > >>
> > >> Guozhang
> > >>
> > >>
> > >> On Mon, Jul 30, 2018 at 1:36 PM, Bill Bejeck 
> wrote:
> > >>
> > >> > Thanks for the KIP!
> > >> >
> > >> > +1
> > >> >
> > >> > -Bill
> > >> >
> > >> > On Mon, Jul 30, 2018 at 3:42 PM Ted Yu  wrote:
> > >> >
> > >> > > +1
> > >> > >
> > >> > > On Mon, Jul 30, 2018 at 11:46 AM John Roesler 
> > >> wrote:
> > >> > >
> > >> > > > Hello devs,
> > >> > > >
> > >> > > > The discussion of KIP-328 has gone some time with no new
> comments,
> > >> so I
> > >> > > am
> > >> > > > calling for a vote!
> > >> > > >
> > >> > > > Here's the KIP: https://cwiki.apache.org/confluence/x/sQU0BQ
> > >> > > >
> > >> > > > The basic idea is to provide:
> > >> > > > * more usable control over update rate (vs the current state
> store
> > >> > > caches)
> > >> > > > * the final-result-for-windowed-computations feature which
> > several
> > >> > people
> > >> > > > have requested
> > >> > > >
> > >> > > > Thanks,
> > >> > > > -John
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> -- Guozhang
> > >>
> > >
> >
>
>
>
> --
> -- Guozhang
>

Re: [VOTE] KIP-233: Simplify StreamsBuilder#addGlobalStore

[Damian Guy]

+1

On Wed, 3 Jan 2018 at 03:49 Matthias J. Sax  wrote:

> It must be two different names, as we add two processor to the topology:
> a source processor that only read the data from a topic, and the actual
> processor that maintains the global table.
>
>
> -Matthias
>
> On 1/2/18 7:14 PM, Ewen Cheslack-Postava wrote:
> > +1 binding, seems like a nice simplification.
> >
> > Regarding the source and processor name, do they actually need to be
> unique
> > or could they use the same value? Since these use incrementing integers,
> it
> > could be nice for debuggability/understanding to have them use the same
> > name if possible instead of generating 2 separate names.
> >
> > -Ewen
> >
> > On Tue, Jan 2, 2018 at 9:12 AM, Guozhang Wang 
> wrote:
> >
> >> On a side note, could you update the "compatibility and upgrade"
> section,
> >> that when users start to make code changes to leverage the new API, what
> >> kind of upgrade executions they'd need to do? I feel they need to rename
> >> topics / etc.
> >>
> >> On Tue, Jan 2, 2018 at 9:10 AM, Guozhang Wang 
> wrote:
> >>
> >>> +1, thanks!
> >>>
> >>> On Wed, Dec 27, 2017 at 6:01 PM, Ted Yu  wrote:
> >>>
>  +1
> 
>  On Wed, Dec 27, 2017 at 12:15 PM, Bill Bejeck 
> >> wrote:
> 
> > +1
> >
> > On Wed, Dec 27, 2017 at 3:07 PM, Matthias J. Sax <
> >> matth...@confluent.io
> >
> > wrote:
> >
> >> +1
> >>
> >> On 12/26/17 9:00 PM, Panuwat Anawatmongkhon wrote:
> >>> Hi all,
> >>> I would like to start the vote thread.
> >>> This is link for the kip.
> >>>
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-233%
>  3A+Simplify+
> >> StreamsBuilder%23addGlobalStore
> >>>
> >>> Cheers
> >>>
> >>
> >>
> >
> 
> >>>
> >>>
> >>>
> >>> --
> >>> -- Guozhang
> >>>
> >>
> >>
> >>
> >> --
> >> -- Guozhang
> >>
> >
>
>

[DISCUSS] February Release Plan

[Damian Guy]

Hi all,

I would like to volunteer to be the release manager
for our next time-based feature release (v1.1.0). See
https://cwiki.apache.org/
confluence/display/KAFKA/Time+Based+Release+Plan if you missed
previous communication on time-based releases or need a reminder.

I put together a draft release plan
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546 with
February 2018 as the release month
(as previously agreed) and a list of KIPs that have already been voted:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546

Some important (and fast approaching) dates:
KIP Freeze: January 23, 2018
Feature Freeze: January 30, 2018

Thanks,
Damian

Re: [ANNOUNCE] New committer: Matthias J. Sax

[Damian Guy]

Can't think of anyone me deserving! Congratulations Matthias!
On Sat, 13 Jan 2018 at 00:17, Ismael Juma  wrote:

> Congratulations Matthias!
>
> On 12 Jan 2018 10:59 pm, "Guozhang Wang"  wrote:
>
> > Hello everyone,
> >
> > The PMC of Apache Kafka is pleased to announce Matthias J. Sax as our
> > newest Kafka committer.
> >
> > Matthias has made tremendous contributions to Kafka Streams API since
> early
> > 2016. His footprint has been all over the places in Streams: in the past
> > two years he has been the main driver on improving the join semantics
> > inside Streams DSL, summarizing all their shortcomings and bridging the
> > gaps; he has also been largely working on the exactly-once semantics of
> > Streams by leveraging on the transaction messaging feature in 0.11.0. In
> > addition, Matthias have been very active in community activity that goes
> > beyond mailing list: he's getting the close to 1000 up votes and 100
> > helpful flags on SO for answering almost all questions about Kafka
> Streams.
> >
> > Thank you for your contribution and welcome to Apache Kafka, Matthias!
> >
> >
> >
> > Guozhang, on behalf of the Apache Kafka PMC
> >
>

Re: [ANNOUNCE] New committer: Matthias J. Sax

[Damian Guy]

Congrats Matthias! Very well deserved
On Sat, 13 Jan 2018 at 00:57, Bill Bejeck  wrote:

> Congratulations Matthias!
>
> -Bill
>
> On Fri, Jan 12, 2018 at 7:43 PM, Damian Guy  wrote:
>
> > Can't think of anyone me deserving! Congratulations Matthias!
> > On Sat, 13 Jan 2018 at 00:17, Ismael Juma  wrote:
> >
> > > Congratulations Matthias!
> > >
> > > On 12 Jan 2018 10:59 pm, "Guozhang Wang"  wrote:
> > >
> > > > Hello everyone,
> > > >
> > > > The PMC of Apache Kafka is pleased to announce Matthias J. Sax as our
> > > > newest Kafka committer.
> > > >
> > > > Matthias has made tremendous contributions to Kafka Streams API since
> > > early
> > > > 2016. His footprint has been all over the places in Streams: in the
> > past
> > > > two years he has been the main driver on improving the join semantics
> > > > inside Streams DSL, summarizing all their shortcomings and bridging
> the
> > > > gaps; he has also been largely working on the exactly-once semantics
> of
> > > > Streams by leveraging on the transaction messaging feature in 0.11.0.
> > In
> > > > addition, Matthias have been very active in community activity that
> > goes
> > > > beyond mailing list: he's getting the close to 1000 up votes and 100
> > > > helpful flags on SO for answering almost all questions about Kafka
> > > Streams.
> > > >
> > > > Thank you for your contribution and welcome to Apache Kafka,
> Matthias!
> > > >
> > > >
> > > >
> > > > Guozhang, on behalf of the Apache Kafka PMC
> > > >
> > >
> >
>

1.1 KIPs

[Damian Guy]

Hi,

This is a reminder that we have one week left until the KIP deadline of Jan
23. There are still some KIPs that are under discussion and/or being voted
on. Please keep in mind that the voting needs to be complete before the
deadline for the KIP to be added to the release.

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546

Thanks,
Damian

Re: Vote for KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor

[Damian Guy]

Thanks for the KIP!  +1

On Sat, 13 Jan 2018 at 12:30 Boyang Chen  wrote:

> Hey Matt and Guozhang,
>
>
> I have already updated the pull request:
> https://github.com/apache/kafka/pull/4354
>
> and the KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-245%3A+Use+Properties+instead+of+StreamsConfig+in+KafkaStreams+constructor
>
>
> to reflect the change proposed by Guozhang(adding a 4th constructor)
>
> Let me know your thoughts!
>
>
> Best,
>
> Boyang
>
>
> 
> From: Boyang Chen 
> Sent: Saturday, January 13, 2018 9:37 AM
> To: Matthias J. Sax
> Subject: Re: Vote for KIP-245: Use Properties instead of StreamsConfig in
> KafkaStreams constructor
>
>
> Sounds good, will do. However I don't receive the +1 emails, interesting...
>
>
> 
> From: Matthias J. Sax 
> Sent: Saturday, January 13, 2018 9:32 AM
> To: Boyang Chen
> Subject: Re: Vote for KIP-245: Use Properties instead of StreamsConfig in
> KafkaStreams constructor
>
> Guozhang left a comment about having a 4th overload. I agree that we
> should add this 4th overload.
>
> Please update the KIP accordingly and follow up on the mailing list
> thread. Than we can vote it through.
>
> Thx.
>
> -Matthias
>
> On 1/12/18 4:48 PM, Boyang Chen wrote:
> > Hey Matt,
> >
> >
> > I haven't received any approval/veto on this KIP. Everything is ready
> > but only needs one approval. Any step I should take?
> >
> > Thanks for the help!
> >
> > Boyang
> >
> >
> >
> > 
> > *From:* Matthias J. Sax 
> > *Sent:* Saturday, January 13, 2018 3:47 AM
> > *To:* dev@kafka.apache.org
> > *Subject:* Re: Vote for KIP-245: Use Properties instead of StreamsConfig
> > in KafkaStreams constructor
> >
> > Boyang,
> >
> > what is the status of this KIP? The release plan for 1.1 was just
> > announced and we like to get this KIP into the release.
> >
> > Thx.
> >
> >
> > -Matthias
> >
> > On 1/2/18 11:18 AM, Guozhang Wang wrote:
> >> Boyang,
> >>
> >> Thanks for the proposed change, the wiki page lgtm. One minor comment
> >> otherwise I'm +1:
> >>
> >> For the new API, we now also have a constructor that accepts both a
> >> clientSupplier and a Time, so we should consider having four overloads
> in
> >> total:
> >>
> >>
> >> // New API (using Properties)
> >> public KafkaStreams(final Topology, final Properties props)
> >> public KafkaStreams(final Topology, final Properties props, final Time
> time)
> >> public KafkaStreams(final Topology, final Properties props, final
> >> KafkaClientSupplier
> >> clientSupplier)
> >> public KafkaStreams(final Topology, final Properties props, final
> >> KafkaClientSupplier
> >> clientSupplier, final Time time)
> >>
> >> Guozhang
> >>
> >> On Tue, Dec 26, 2017 at 7:26 PM, Satish Duggana <
> satish.dugg...@gmail.com>
> >> wrote:
> >>
> >>> Thanks for the KIP, +1 from me.
> >>>
> >>> On Wed, Dec 27, 2017 at 7:42 AM, Bill Bejeck 
> wrote:
> >>>
>  Thanks for the KIP.  +1 for me.
> 
>  On Tue, Dec 26, 2017 at 6:22 PM Ted Yu  wrote:
> 
> > +1 from me as well.
> >
> > On Tue, Dec 26, 2017 at 10:41 AM, Matthias J. Sax <
> >>> matth...@confluent.io
> >
> > wrote:
> >
> >> Thanks for the KIP Boyang!
> >>
> >> I don't have any further comments.
> >>
> >> +1 from me.
> >>
> >> @Ted: This is a rather simple KIP, thus, skipping the DISCUSS thread
> >> seems ok to me.
> >>
> >>
> >>
> >> -Matthias
> >>
> >>
> >> @Boyang: it's recommended to use this format for the subject
> >>
> >> "[VOTE] KIP-245: ..."
> >>
> >> Same for DISCUSS threads. People are used to those headlines and
> they
> >> pay more attention than. For this KIP, just leave it as it though.
> >>> For
> >> future reference only
> >> .
> >>
> >>
> >> On 12/26/17 4:55 AM, Ted Yu wrote:
> >>> Normally a DISCUSS thread precedes VOTE thread so that people have
> > ample
> >> time examining the proposal.
> >>>  Original message From: Boyang Chen <
> > bche...@outlook.com>
> >> Date: 12/26/17  1:22 AM  (GMT-07:00) To: dev@kafka.apache.org
> >>> Subject:
> >> Vote for KIP-245: Use Properties instead of StreamsConfig in
>  KafkaStreams
> >> constructor
> >>> Hi there,
> >>>
> >>> I'm Boyang who is a newbie contributor to Kafka. I would like to
>  start
> > a
> >> vote for the KIP-245:
> >>>
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>
> > 245%3A+Use+Properties+instead+of+StreamsConfig+in+
>  KafkaStreams+constructor
> >>>
> >>>
> >>> This is linked with JIRA: https://issues.apache.org/
> > issues.apache.org 
> issues.apache.org
> issues.apache.org
> Apache currently hosts two different issue tracking systems, Bugzilla and
> Jira. To find out how to rep

Re: 1.1 KIPs

[Damian Guy]

Hi Xavier,
I'll add it to the plan.

Thanks,
Damian

On Tue, 16 Jan 2018 at 19:04 Xavier Léauté  wrote:

> Hi Damian, I believe the list should also include KAFKA-5886 (KIP-91) which
> was voted for 1.0 but wasn't ready to be merged in time.
>
> On Tue, Jan 16, 2018 at 5:13 AM Damian Guy  wrote:
>
> > Hi,
> >
> > This is a reminder that we have one week left until the KIP deadline of
> Jan
> > 23. There are still some KIPs that are under discussion and/or being
> voted
> > on. Please keep in mind that the voting needs to be complete before the
> > deadline for the KIP to be added to the release.
> >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546
> >
> > Thanks,
> > Damian
> >
>

Re: [ANNOUNCE] New Kafka PMC Member: Rajini Sivaram

[Damian Guy]

Congratulations Rajini!

On Thu, 18 Jan 2018 at 00:57 Hu Xi  wrote:

> Congratulations, Rajini Sivaram.  Very well deserved!
>
>
> 
> 发件人: Konstantine Karantasis 
> 发送时间: 2018年1月18日 6:23
> 收件人: dev@kafka.apache.org
> 抄送: us...@kafka.apache.org
> 主题: Re: [ANNOUNCE] New Kafka PMC Member: Rajini Sivaram
>
> Congrats Rajini!
>
> -Konstantine
>
> On Wed, Jan 17, 2018 at 2:18 PM, Becket Qin  wrote:
>
> > Congratulations, Rajini!
> >
> > On Wed, Jan 17, 2018 at 1:52 PM, Ismael Juma  wrote:
> >
> > > Congratulations Rajini!
> > >
> > > On 17 Jan 2018 10:49 am, "Gwen Shapira"  wrote:
> > >
> > > Dear Kafka Developers, Users and Fans,
> > >
> > > Rajini Sivaram became a committer in April 2017.  Since then, she
> > remained
> > > active in the community and contributed major patches, reviews and KIP
> > > discussions. I am glad to announce that Rajini is now a member of the
> > > Apache Kafka PMC.
> > >
> > > Congratulations, Rajini and looking forward to your future
> contributions.
> > >
> > > Gwen, on behalf of Apache Kafka PMC
> > >
> >
>

Re: [VOTE] KIP-247: Add public test utils for Kafka Streams

[Damian Guy]

+1

On Thu, 18 Jan 2018 at 15:14 Bill Bejeck  wrote:

> Thanks for the KIP.
>
> +1
>
> -Bill
>
> On Wed, Jan 17, 2018 at 9:09 PM, Matthias J. Sax 
> wrote:
>
> > Hi,
> >
> > I would like to start the vote for KIP-247:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 247%3A+Add+public+test+utils+for+Kafka+Streams
> >
> >
> > -Matthias
> >
> >
>

Re: 1.1 KIPs

[Damian Guy]

Gentle reminder that KIP deadline is just 5 days away. If there is anything
that wants to be in 1.1 and hasn't been voted on yet, now is the time!

On Thu, 18 Jan 2018 at 08:49 Damian Guy  wrote:

> Hi Xavier,
> I'll add it to the plan.
>
> Thanks,
> Damian
>
> On Tue, 16 Jan 2018 at 19:04 Xavier Léauté  wrote:
>
>> Hi Damian, I believe the list should also include KAFKA-5886 (KIP-91)
>> which
>> was voted for 1.0 but wasn't ready to be merged in time.
>>
>> On Tue, Jan 16, 2018 at 5:13 AM Damian Guy  wrote:
>>
>> > Hi,
>> >
>> > This is a reminder that we have one week left until the KIP deadline of
>> Jan
>> > 23. There are still some KIPs that are under discussion and/or being
>> voted
>> > on. Please keep in mind that the voting needs to be complete before the
>> > deadline for the KIP to be added to the release.
>> >
>> >
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546
>> >
>> > Thanks,
>> > Damian
>> >
>>
>

Re: [VOTE] KIP-208: Add SSL support to Kafka Connect REST interface

[Damian Guy]

Thanks Jakub!

+1 (binding)

On Thu, 18 Jan 2018 at 23:49 Jakub Scholz  wrote:

> Hi all,
>
> We still need at least 2 more binding +1s. I think that the PR (
> https://github.com/apache/kafka/pull/4429) is shaping good. If we get the
> votes, we should be able to make the 1.1.0 release.
>
> Thanks & Regards
> Jakub
>
> On Fri, Jan 5, 2018 at 4:30 AM, Ewen Cheslack-Postava 
> wrote:
>
> > Jakub,
> >
> > I left a few comments in the discuss thread, but I'll also reply here
> just
> > to bump the VOTE thread's visibility. I would like to resolve the few
> > comments I left, but I am effectively +1 on this, the comments I left
> were
> > mainly details.
> >
> > Committers that could help with the necessary votes would probably be
> Gwen
> > and Jason (but others more than welcome to help out too :)
> >
> > -Ewen
> >
> > On Mon, Nov 6, 2017 at 1:52 AM, Jakub Scholz  wrote:
> >
> > > Hi all,
> > >
> > > Just a reminder that htis is still up for vote. I think this is
> important
> > > featrue which would deserve your votes.
> > >
> > > Regards
> > > Jakub
> > >
> > > On Mon, Oct 30, 2017 at 9:24 PM, Jakub Scholz  wrote:
> > >
> > > > Hi,
> > > >
> > > > It seems there are no more comments for this KIP, so I would like to
> > > start
> > > > the voting .
> > > >
> > > > For more details about the KIP-208 go to
> > > > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 208%3A+Add+SSL+support+to+Kafka+Connect+REST+interface
> > > >  > > 208%3A+Add+SSL+support+to+Kafka+Connect+REST+interface>*
> > > >
> > > > Thanks & Regards
> > > > Jakub
> > > >
> > >
> >
>

Re: 1.1 KIPs

[Damian Guy]

Hi,

Today is the KIP deadline. Tomorrow I'll update the release page with any
KIPS that have recently been voted on and accepted according to the process
that can be found here:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Process

Thanks,
Damian

On Thu, 18 Jan 2018 at 12:47 Damian Guy  wrote:

> Gentle reminder that KIP deadline is just 5 days away. If there is
> anything that wants to be in 1.1 and hasn't been voted on yet, now is the
> time!
>
> On Thu, 18 Jan 2018 at 08:49 Damian Guy  wrote:
>
>> Hi Xavier,
>> I'll add it to the plan.
>>
>> Thanks,
>> Damian
>>
>> On Tue, 16 Jan 2018 at 19:04 Xavier Léauté  wrote:
>>
>>> Hi Damian, I believe the list should also include KAFKA-5886 (KIP-91)
>>> which
>>> was voted for 1.0 but wasn't ready to be merged in time.
>>>
>>> On Tue, Jan 16, 2018 at 5:13 AM Damian Guy  wrote:
>>>
>>> > Hi,
>>> >
>>> > This is a reminder that we have one week left until the KIP deadline
>>> of Jan
>>> > 23. There are still some KIPs that are under discussion and/or being
>>> voted
>>> > on. Please keep in mind that the voting needs to be complete before the
>>> > deadline for the KIP to be added to the release.
>>> >
>>> >
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546
>>> >
>>> > Thanks,
>>> > Damian
>>> >
>>>
>>

Re: 1.1 KIPs

[Damian Guy]

Hi,

The KIP deadline has passed and i've updated the release plan:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546
If there is anything i've missed please let me know.

The feature freeze deadline is January 30th. At this point i'll cut the
branch for 1.1. So please make sure any major features have been committed
by then.

Thanks,
Damian

On Tue, 23 Jan 2018 at 15:29 Damian Guy  wrote:

> Hi,
>
> Today is the KIP deadline. Tomorrow I'll update the release page with any
> KIPS that have recently been voted on and accepted according to the process
> that can be found here:
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Process
>
> Thanks,
> Damian
>
> On Thu, 18 Jan 2018 at 12:47 Damian Guy  wrote:
>
>> Gentle reminder that KIP deadline is just 5 days away. If there is
>> anything that wants to be in 1.1 and hasn't been voted on yet, now is the
>> time!
>>
>> On Thu, 18 Jan 2018 at 08:49 Damian Guy  wrote:
>>
>>> Hi Xavier,
>>> I'll add it to the plan.
>>>
>>> Thanks,
>>> Damian
>>>
>>> On Tue, 16 Jan 2018 at 19:04 Xavier Léauté  wrote:
>>>
>>>> Hi Damian, I believe the list should also include KAFKA-5886 (KIP-91)
>>>> which
>>>> was voted for 1.0 but wasn't ready to be merged in time.
>>>>
>>>> On Tue, Jan 16, 2018 at 5:13 AM Damian Guy 
>>>> wrote:
>>>>
>>>> > Hi,
>>>> >
>>>> > This is a reminder that we have one week left until the KIP deadline
>>>> of Jan
>>>> > 23. There are still some KIPs that are under discussion and/or being
>>>> voted
>>>> > on. Please keep in mind that the voting needs to be complete before
>>>> the
>>>> > deadline for the KIP to be added to the release.
>>>> >
>>>> >
>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546
>>>> >
>>>> > Thanks,
>>>> > Damian
>>>> >
>>>>
>>>

Re: 1.1 KIPs

[Damian Guy]

Thanks Vahid, i've updated the release plan

On Wed, 24 Jan 2018 at 13:31 Vahid S Hashemian 
wrote:

> Hi Damian,
>
> Could you please add KIP-229 to the list? It was approved earlier this
> week
> https://www.mail-archive.com/dev@kafka.apache.org/msg84851.html
>
> Thanks for running the release.
> --Vahid
>
>
>
>
> From:   Damian Guy 
> To: dev@kafka.apache.org
> Date:   01/24/2018 01:20 PM
> Subject:Re: 1.1 KIPs
>
>
>
> Hi,
>
> The KIP deadline has passed and i've updated the release plan:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_pages_viewpage.action-3FpageId-3D75957546&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=TVUESwz92IFTGWgOw60U-bc5Fih9bzGtCRahKUZlSTc&s=ZsBKyGkffkjDH0yHsVpxNP-_qhSR681YSL48Bi5cTBs&e=
>
> If there is anything i've missed please let me know.
>
> The feature freeze deadline is January 30th. At this point i'll cut the
> branch for 1.1. So please make sure any major features have been committed
> by then.
>
> Thanks,
> Damian
>
> On Tue, 23 Jan 2018 at 15:29 Damian Guy  wrote:
>
> > Hi,
> >
> > Today is the KIP deadline. Tomorrow I'll update the release page with
> any
> > KIPS that have recently been voted on and accepted according to the
> process
> > that can be found here:
> >
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_Kafka-2BImprovement-2BProposals-23KafkaImprovementProposals-2DProcess&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=TVUESwz92IFTGWgOw60U-bc5Fih9bzGtCRahKUZlSTc&s=JEIfWpaEOFJ8NmyLHUs7NLlcgPgsNEdM1Iy9Y5Tr3Vs&e=
>
> >
> > Thanks,
> > Damian
> >
> > On Thu, 18 Jan 2018 at 12:47 Damian Guy  wrote:
> >
> >> Gentle reminder that KIP deadline is just 5 days away. If there is
> >> anything that wants to be in 1.1 and hasn't been voted on yet, now is
> the
> >> time!
> >>
> >> On Thu, 18 Jan 2018 at 08:49 Damian Guy  wrote:
> >>
> >>> Hi Xavier,
> >>> I'll add it to the plan.
> >>>
> >>> Thanks,
> >>> Damian
> >>>
> >>> On Tue, 16 Jan 2018 at 19:04 Xavier Léauté 
> wrote:
> >>>
> >>>> Hi Damian, I believe the list should also include KAFKA-5886 (KIP-91)
> >>>> which
> >>>> was voted for 1.0 but wasn't ready to be merged in time.
> >>>>
> >>>> On Tue, Jan 16, 2018 at 5:13 AM Damian Guy 
> >>>> wrote:
> >>>>
> >>>> > Hi,
> >>>> >
> >>>> > This is a reminder that we have one week left until the KIP
> deadline
> >>>> of Jan
> >>>> > 23. There are still some KIPs that are under discussion and/or
> being
> >>>> voted
> >>>> > on. Please keep in mind that the voting needs to be complete before
> >>>> the
> >>>> > deadline for the KIP to be added to the release.
> >>>> >
> >>>> >
> >>>>
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_pages_viewpage.action-3FpageId-3D75957546&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=TVUESwz92IFTGWgOw60U-bc5Fih9bzGtCRahKUZlSTc&s=ZsBKyGkffkjDH0yHsVpxNP-_qhSR681YSL48Bi5cTBs&e=
>
> >>>> >
> >>>> > Thanks,
> >>>> > Damian
> >>>> >
> >>>>
> >>>
>
>
>
>
>

1.1 release progress

[Damian Guy]

Hi,

Just a quick updated on the 1.1 release. Feature freeze is on the 30th of
January, so please ensure any major features have been merged and that any
minor features have PRs by this time.

We currently have 57 issues in progress and 117 todo:
https://issues.apache.org/jira/projects/KAFKA/versions/12339769
If people can start moving stuff out of 1.1 that would be great, otherwise
anything that is todo will be moved to the next release.

Thanks,
Damian

New release branch 1.1

[Damian Guy]

Hello Kafka developers and friends,

We now have a release branch for the 1.1 release (with 1.1.0 as the version).
Trunk has been bumped to 1.2.0-SNAPSHOT.

I'll be going over the JIRAs to move every non-blocker from this release to
the next release.

>From this point, most changes should go to trunk.
*Blockers (existing and new that we discover while testing the release)
will be double-committed. *Please discuss with your reviewer whether your
PR should go to trunk or to trunk+release so they can merge accordingly.

*Please help us test the release! *

Thanks!

Damian

1.1 release progress

[Damian Guy]

Hi all,

We are just over a week away from code freeze. We currently have 44 issues
in progress and 3 open blockers:
https://issues.apache.org/jira/projects/KAFKA/versions/12339769

If you have something that is in progress that is not going to make it into
1.1 can you please move it to a future release.

Thanks,
Damian

Re: [VOTE] KIP-251: Allow timestamp manipulation in Processor API

[Damian Guy]

Thanks Matthias, +1

On Fri, 9 Feb 2018 at 02:42 Ted Yu  wrote:

> +1
>  Original message From: "Matthias J. Sax" <
> matth...@confluent.io> Date: 2/8/18  6:05 PM  (GMT-08:00) To:
> dev@kafka.apache.org Subject: [VOTE] KIP-251: Allow timestamp
> manipulation in Processor API
> Hi,
>
> I want to start the vote for KIP-251:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-251%3A+Allow+timestamp+manipulation+in+Processor+API
>
>
> -Matthias
>
>

Re: 1.1 release progress

[Damian Guy]

Hi All,

The first 1.1 RC is due to be cut, however we currently have 2 blockers
outstanding:
https://issues.apache.org/jira/browse/KAFKA-6517 (which i suspect will be
merged shortly)
and
https://issues.apache.org/jira/browse/KAFKA-6549

Once we have finished these issues I can create the first RC. Hopefully
before the end of the week.

Regards,
Damian


On Mon, 5 Feb 2018 at 18:04 Damian Guy  wrote:

> Hi all,
>
> We are just over a week away from code freeze. We currently have 44 issues
> in progress and 3 open blockers:
> https://issues.apache.org/jira/projects/KAFKA/versions/12339769
>
> If you have something that is in progress that is not going to make it
> into 1.1 can you please move it to a future release.
>
> Thanks,
> Damian
>

Re: 1.1 release progress

[Damian Guy]

Hi Becket,

Thanks for letting me know.

Damian

On Fri, 16 Feb 2018 at 01:48 Ismael Juma  wrote:

> Sounds good. Thanks Becket.
>
> Ismael
>
> On 15 Feb 2018 5:30 pm, "Becket Qin"  wrote:
>
> > Hi Ismael,
> >
> > Yes, I am working on the fix. Will submit patch today.
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > On Thu, Feb 15, 2018 at 2:53 PM, Ismael Juma  wrote:
> >
> > > Hi Becket,
> > >
> > > Thanks for filing that. Are you working on a fix?
> > >
> > > Ismael
> > >
> > > On Thu, Feb 15, 2018 at 2:51 PM, Becket Qin 
> > wrote:
> > >
> > > > Hi Damian,
> > > >
> > > > I just created another ticket KAFKA-6568, which I believe should also
> > be
> > > a
> > > > blocker unless people disagree.
> > > >
> > > > Thanks,
> > > >
> > > > Jiangjie (Becket) Qin
> > > >
> > > > On Wed, Feb 14, 2018 at 8:52 AM, Damian Guy 
> > > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > The first 1.1 RC is due to be cut, however we currently have 2
> > blockers
> > > > > outstanding:
> > > > > https://issues.apache.org/jira/browse/KAFKA-6517 (which i suspect
> > will
> > > > be
> > > > > merged shortly)
> > > > > and
> > > > > https://issues.apache.org/jira/browse/KAFKA-6549
> > > > >
> > > > > Once we have finished these issues I can create the first RC.
> > Hopefully
> > > > > before the end of the week.
> > > > >
> > > > > Regards,
> > > > > Damian
> > > > >
> > > > >
> > > > > On Mon, 5 Feb 2018 at 18:04 Damian Guy 
> wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > We are just over a week away from code freeze. We currently have
> 44
> > > > > issues
> > > > > > in progress and 3 open blockers:
> > > > > > https://issues.apache.org/jira/projects/KAFKA/versions/12339769
> > > > > >
> > > > > > If you have something that is in progress that is not going to
> make
> > > it
> > > > > > into 1.1 can you please move it to a future release.
> > > > > >
> > > > > > Thanks,
> > > > > > Damian
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: 1.1 release progress

[Damian Guy]

Hi,

A quick update on the 1.1 release. I'm in the process of generating RC0,
but I'm waiting to get a passing system test build before pushing. Stay
tuned!

Thanks,
Damian

On Fri, 16 Feb 2018 at 09:49 Damian Guy  wrote:

> Hi Becket,
>
> Thanks for letting me know.
>
> Damian
>
> On Fri, 16 Feb 2018 at 01:48 Ismael Juma  wrote:
>
>> Sounds good. Thanks Becket.
>>
>> Ismael
>>
>> On 15 Feb 2018 5:30 pm, "Becket Qin"  wrote:
>>
>> > Hi Ismael,
>> >
>> > Yes, I am working on the fix. Will submit patch today.
>> >
>> > Thanks,
>> >
>> > Jiangjie (Becket) Qin
>> >
>> > On Thu, Feb 15, 2018 at 2:53 PM, Ismael Juma  wrote:
>> >
>> > > Hi Becket,
>> > >
>> > > Thanks for filing that. Are you working on a fix?
>> > >
>> > > Ismael
>> > >
>> > > On Thu, Feb 15, 2018 at 2:51 PM, Becket Qin 
>> > wrote:
>> > >
>> > > > Hi Damian,
>> > > >
>> > > > I just created another ticket KAFKA-6568, which I believe should
>> also
>> > be
>> > > a
>> > > > blocker unless people disagree.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Jiangjie (Becket) Qin
>> > > >
>> > > > On Wed, Feb 14, 2018 at 8:52 AM, Damian Guy 
>> > > wrote:
>> > > >
>> > > > > Hi All,
>> > > > >
>> > > > > The first 1.1 RC is due to be cut, however we currently have 2
>> > blockers
>> > > > > outstanding:
>> > > > > https://issues.apache.org/jira/browse/KAFKA-6517 (which i suspect
>> > will
>> > > > be
>> > > > > merged shortly)
>> > > > > and
>> > > > > https://issues.apache.org/jira/browse/KAFKA-6549
>> > > > >
>> > > > > Once we have finished these issues I can create the first RC.
>> > Hopefully
>> > > > > before the end of the week.
>> > > > >
>> > > > > Regards,
>> > > > > Damian
>> > > > >
>> > > > >
>> > > > > On Mon, 5 Feb 2018 at 18:04 Damian Guy 
>> wrote:
>> > > > >
>> > > > > > Hi all,
>> > > > > >
>> > > > > > We are just over a week away from code freeze. We currently
>> have 44
>> > > > > issues
>> > > > > > in progress and 3 open blockers:
>> > > > > > https://issues.apache.org/jira/projects/KAFKA/versions/12339769
>> > > > > >
>> > > > > > If you have something that is in progress that is not going to
>> make
>> > > it
>> > > > > > into 1.1 can you please move it to a future release.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Damian
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [VOTE] 1.0.1 RC2

[Damian Guy]

+1

Built src and ran tests
Ran streams quickstart

On Thu, 22 Feb 2018 at 21:32 Ted Yu  wrote:

> +1
>
> MetricsTest#testMetricsLeak failed but it is flaky test
>
> On Wed, Feb 21, 2018 at 4:06 PM, Ewen Cheslack-Postava 
> wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the third candidate for release of Apache Kafka 1.0.1.
> >
> > This is a bugfix release for the 1.0 branch that was first released with
> > 1.0.0 about 3 months ago. We've fixed 49 issues since that release. Most
> of
> > these are non-critical, but in aggregate these fixes will have
> significant
> > impact. A few of the more significant fixes include:
> >
> > * KAFKA-6277: Make loadClass thread-safe for class loaders of Connect
> > plugins
> > * KAFKA-6185: Selector memory leak with high likelihood of OOM in case of
> > down conversion
> > * KAFKA-6269: KTable state restore fails after rebalance
> > * KAFKA-6190: GlobalKTable never finishes restoring when consuming
> > transactional messages
> > * KAFKA-6529: Stop file descriptor leak when client disconnects with
> staged
> > receives
> > * KAFKA-6238: Issues with protocol version when applying a rolling
> upgrade
> > to 1.0.0
> >
> > Release notes for the 1.0.1 release:
> > http://home.apache.org/~ewencp/kafka-1.0.1-rc2/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by Saturday Feb 24, 9pm PT ***
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > http://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > http://home.apache.org/~ewencp/kafka-1.0.1-rc2/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/
> >
> > * Javadoc:
> > http://home.apache.org/~ewencp/kafka-1.0.1-rc2/javadoc/
> >
> > * Tag to be voted upon (off 1.0 branch) is the 1.0.1 tag:
> > https://github.com/apache/kafka/tree/1.0.1-rc2
> >
> > * Documentation:
> > http://kafka.apache.org/10/documentation.html
> >
> > * Protocol:
> > http://kafka.apache.org/10/protocol.html
> >
> > /**
> >
> > Thanks,
> > Ewen Cheslack-Postava
> >
>

Re: [DISCUSS] KIP-261: Add Single Value Fetch in Window Stores

[Damian Guy]

Thanks Guozhang. +1

On Thu, 22 Feb 2018 at 23:44 Guozhang Wang  wrote:

> Thanks Ted, have updated the wiki page.
>
> On Thu, Feb 22, 2018 at 1:48 PM, Ted Yu  wrote:
>
> > +1
> >
> > There were some typos:
> > CachingWindowedStore -> CachingWindowStore
> > RocksDBWindowedStore -> RocksDBWindowStore
> > KStreamWindowedAggregate -> KStreamWindowAggregate
> > KStreamWindowedReduce -> KStreamWindowReduce
> >
> > Cheers
> >
> > On Thu, Feb 22, 2018 at 1:34 PM, Guozhang Wang 
> wrote:
> >
> > > Hi all,
> > >
> > > I have submitted KIP-261 to add a new API for window stores in order to
> > > optimize our current windowed aggregation implementations inside
> Streams
> > > DSL
> > > :
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 261%3A+Add+Single+Value+Fetch+in+Window+Stores
> > >
> > > This change would require people who have customized window store
> > > implementations to make code changes as part of their upgrade path.
> But I
> > > think it is worth while given that the fraction of customized window
> > store
> > > should be very small.
> > >
> > >
> > > Feedback and suggestions are welcome.
> > >
> > > Thanks,
> > > -- Guozhang
> > >
> >
>
>
>
> --
> -- Guozhang
>

[VOTE] 1.1.0 RC0

[Damian Guy]

Hello Kafka users, developers and client-developers,

This is the first candidate for release of Apache Kafka 1.1.0.

This is minor version release of Apache Kakfa. It Includes 29 new KIPs.
Please see the release plan for more details:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71764913

A few highlights:

* Significant Controller improvements (much faster and session expiration
edge cases fixed)
* Data balancing across log directories (JBOD)
* More efficient replication when the number of partitions is large
* Dynamic Broker Configs
* Delegation tokens (KIP-48)
* Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)

Release notes for the 1.1.0 release:
http://home.apache.org/~damianguy/kafka-1.1.0-rc0/RELEASE_NOTES.html

*** Please download, test and vote by Wednesday, February 28th, 5pm PT

Kafka's KEYS file containing PGP keys we use to sign the release:
http://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
http://home.apache.org/~damianguy/kafka-1.1.0-rc0/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/

* Javadoc:
http://home.apache.org/~damianguy/kafka-1.1.0-rc0/javadoc/

* Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
https://github.com/apache/kafka/tree/1.1.0-rc0


* Documentation:
http://kafka.apache.org/11/documentation.html

* Protocol:
http://kafka.apache.org/11/protocol.html

* Successful Jenkins builds for the 1.1 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-1.1-jdk7/63/
System tests: https://jenkins.confluent.io/job/system-test-kafka/job/1.1/21/


Finally, just want to say thanks to the people that have helped me get the
System Tests green this week. In particular:

Konstantine Karantasis
Randall Hauch
Colin Mcabe


/**

Thanks,
Damian

Re: [VOTE] KIP-261: Add Single Value Fetch in Window Stores

[Damian Guy]

Thanks Guozhang!
+1

On Sat, 24 Feb 2018 at 19:12 Ted Yu  wrote:

> +1
>
> On Sat, Feb 24, 2018 at 11:10 AM, Guozhang Wang 
> wrote:
>
> > Hi all,
> >
> > I want to start voting on KIP-261 to add a new API for window stores in
> > order to optimize our current windowed aggregation implementations inside
> > Streams DSL:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 261%3A+Add+Single+Value+Fetch+in+Window+Stores
> >
> >
> > Please cast your vote before next Wednesday, EOD.
> >
> >
> > Thanks,
> > -- Guozhang
> >
>

Re: [VOTE] 1.1.0 RC0

[Damian Guy]

Hi Jason,

Ok - thanks. Let me know how you get on.

Cheers,
Damian

On Wed, 28 Feb 2018 at 19:23 Jason Gustafson  wrote:

> Hey Damian,
>
> I think we should consider
> https://issues.apache.org/jira/browse/KAFKA-6593
> for the release. I have a patch available, but still working on validating
> both the bug and the fix.
>
> -Jason
>
> On Wed, Feb 28, 2018 at 9:34 AM, Matthias J. Sax 
> wrote:
>
> > No. Both will be released.
> >
> > -Matthias
> >
> > On 2/28/18 6:32 AM, Marina Popova wrote:
> > > Sorry, maybe a stupid question, but:
> > >  I see that Kafka 1.0.1 RC2 is still not released, but now 1.1.0 RC0 is
> > coming up...
> > > Does it mean 1.0.1 will be abandoned and we should be looking forward
> to
> > 1.1.0 instead?
> > >
> > > thanks!
> > >
> > > ​Sent with ProtonMail Secure Email.​
> > >
> > > ‐‐‐ Original Message ‐‐‐
> > >
> > > On February 26, 2018 6:28 PM, Vahid S Hashemian <
> > vahidhashem...@us.ibm.com> wrote:
> > >
> > >> +1 (non-binding)
> > >>
> > >> Built the source and ran quickstart (including streams) successfully
> on
> > >>
> > >> Ubuntu (with both Java 8 and Java 9).
> > >>
> > >> I understand the Windows platform is not officially supported, but I
> ran
> > >>
> > >> the same on Windows 10, and except for Step 7 (Connect) everything
> else
> > >>
> > >> worked fine.
> > >>
> > >> There are a number of warning and errors (including
> > >>
> > >> java.lang.ClassNotFoundException). Here's the final error message:
> > >>
> > >>> bin\\windows\\connect-standalone.bat config\\connect-standalone.
> > properties
> > >>
> > >> config\\connect-file-source.properties config\\connect-file-sink.
> > properties
> > >>
> > >> ...
> > >>
> > >> \[2018-02-26 14:55:56,529\] ERROR Stopping after connector error
> > >>
> > >> (org.apache.kafka.connect.cli.ConnectStandalone)
> > >>
> > >> java.lang.NoClassDefFoundError:
> > >>
> > >> org/apache/kafka/connect/transforms/util/RegexValidator
> > >>
> > >> at
> > >>
> > >> org.apache.kafka.connect.runtime.SinkConnectorConfig.<
> > clinit>(SinkConnectorConfig.java:46)
> > >>
> > >> at
> > >>
> > >>
> > >> org.apache.kafka.connect.runtime.AbstractHerder.
> > validateConnectorConfig(AbstractHerder.java:263)
> > >>
> > >> at
> > >>
> > >> org.apache.kafka.connect.runtime.standalone.StandaloneHerder.
> > putConnectorConfig(StandaloneHerder.java:164)
> > >>
> > >> at
> > >>
> > >> org.apache.kafka.connect.cli.ConnectStandalone.main(
> > ConnectStandalone.java:107)
> > >>
> > >> Caused by: java.lang.ClassNotFoundException:
> > >>
> > >> org.apache.kafka.connect.transforms.util.RegexValidator
> > >>
> > >> at
> > >>
> > >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(
> > BuiltinClassLoader.java:582)
> > >>
> > >> at
> > >>
> > >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.
> > loadClass(ClassLoaders.java:185)
> > >>
> > >> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
> > >>
> > >> ... 4 more
> > >>
> > >> Thanks for running the release.
> > >>
> > >> --Vahid
> > >>
> > >> From: Damian Guy damian@gmail.com
> > >>
> > >> To: dev@kafka.apache.org, us...@kafka.apache.org,
> > >>
> > >> kafka-clie...@googlegroups.com
> > >>
> > >> Date: 02/24/2018 08:16 AM
> > >>
> > >> Subject: \[VOTE\] 1.1.0 RC0
> > >>
> > >> Hello Kafka users, developers and client-developers,
> > >>
> > >> This is the first candidate for release of Apache Kafka 1.1.0.
> > >>
> > >> This is minor version release of Apache Kakfa. It Includes 29 new
> KIPs.
> > >>
> > >> Please see the release plan for more details:
> > >>
> > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.
> > apache.org_confluence_pages_viewpage.a

Re: [VOTE] 1.1.0 RC0

[Damian Guy]

Thanks Jason. Assuming the system tests pass i'll cut RC1 tomorrow.

Thanks,
Damian

On Thu, 1 Mar 2018 at 19:10 Jason Gustafson  wrote:

> The fix has been merged to 1.1.
>
> Thanks,
> Jason
>
> On Wed, Feb 28, 2018 at 11:35 AM, Damian Guy  wrote:
>
> > Hi Jason,
> >
> > Ok - thanks. Let me know how you get on.
> >
> > Cheers,
> > Damian
> >
> > On Wed, 28 Feb 2018 at 19:23 Jason Gustafson  wrote:
> >
> > > Hey Damian,
> > >
> > > I think we should consider
> > > https://issues.apache.org/jira/browse/KAFKA-6593
> > > for the release. I have a patch available, but still working on
> > validating
> > > both the bug and the fix.
> > >
> > > -Jason
> > >
> > > On Wed, Feb 28, 2018 at 9:34 AM, Matthias J. Sax <
> matth...@confluent.io>
> > > wrote:
> > >
> > > > No. Both will be released.
> > > >
> > > > -Matthias
> > > >
> > > > On 2/28/18 6:32 AM, Marina Popova wrote:
> > > > > Sorry, maybe a stupid question, but:
> > > > >  I see that Kafka 1.0.1 RC2 is still not released, but now 1.1.0
> RC0
> > is
> > > > coming up...
> > > > > Does it mean 1.0.1 will be abandoned and we should be looking
> forward
> > > to
> > > > 1.1.0 instead?
> > > > >
> > > > > thanks!
> > > > >
> > > > > ​Sent with ProtonMail Secure Email.​
> > > > >
> > > > > ‐‐‐ Original Message ‐‐‐
> > > > >
> > > > > On February 26, 2018 6:28 PM, Vahid S Hashemian <
> > > > vahidhashem...@us.ibm.com> wrote:
> > > > >
> > > > >> +1 (non-binding)
> > > > >>
> > > > >> Built the source and ran quickstart (including streams)
> successfully
> > > on
> > > > >>
> > > > >> Ubuntu (with both Java 8 and Java 9).
> > > > >>
> > > > >> I understand the Windows platform is not officially supported,
> but I
> > > ran
> > > > >>
> > > > >> the same on Windows 10, and except for Step 7 (Connect) everything
> > > else
> > > > >>
> > > > >> worked fine.
> > > > >>
> > > > >> There are a number of warning and errors (including
> > > > >>
> > > > >> java.lang.ClassNotFoundException). Here's the final error message:
> > > > >>
> > > > >>> bin\\windows\\connect-standalone.bat config\\connect-standalone.
> > > > properties
> > > > >>
> > > > >> config\\connect-file-source.properties config\\connect-file-sink.
> > > > properties
> > > > >>
> > > > >> ...
> > > > >>
> > > > >> \[2018-02-26 14:55:56,529\] ERROR Stopping after connector error
> > > > >>
> > > > >> (org.apache.kafka.connect.cli.ConnectStandalone)
> > > > >>
> > > > >> java.lang.NoClassDefFoundError:
> > > > >>
> > > > >> org/apache/kafka/connect/transforms/util/RegexValidator
> > > > >>
> > > > >> at
> > > > >>
> > > > >> org.apache.kafka.connect.runtime.SinkConnectorConfig.<
> > > > clinit>(SinkConnectorConfig.java:46)
> > > > >>
> > > > >> at
> > > > >>
> > > > >>
> > > > >> org.apache.kafka.connect.runtime.AbstractHerder.
> > > > validateConnectorConfig(AbstractHerder.java:263)
> > > > >>
> > > > >> at
> > > > >>
> > > > >> org.apache.kafka.connect.runtime.standalone.StandaloneHerder.
> > > > putConnectorConfig(StandaloneHerder.java:164)
> > > > >>
> > > > >> at
> > > > >>
> > > > >> org.apache.kafka.connect.cli.ConnectStandalone.main(
> > > > ConnectStandalone.java:107)
> > > > >>
> > > > >> Caused by: java.lang.ClassNotFoundException:
> > > > >>
> > > > >> org.apache.kafka.connect.transforms.util.RegexValidator
> > > > >>
> > > > >> at
> > > > >>
> > > > >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(
&

Re: [DISCUSS] KIP-265: Make Windowed Serde to public APIs

[Damian Guy]

Thanks Guozhang, LGTM

On Fri, 2 Mar 2018 at 03:00 Hu Xi  wrote:

> Guozhang,
>
>
> Thanks for this KIP. Please help confirm questions below:
>
>   1.   Do we also need to retrofit `SimpleConsumerShell` to have it
> support these newly-added serdes?
>   2.   Does this KIP cover the changes for ConsoleConsumer as well?
>
>
> 
> 发件人: Guozhang Wang 
> 发送时间: 2018年3月2日 8:20
> 收件人: dev@kafka.apache.org
> 主题: [DISCUSS] KIP-265: Make Windowed Serde to public APIs
>
> Hello all,
>
> I'd like to have a discussion on making windowed serde to public APIs of
> Kafka Streams. It involves a couple of new configs, plus a few new public
> classes for windowed serializer and deserializer, and also adding the
> corresponding console consumer options in order to fetch from a topic
> written by a windowed store.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-265%3A+Make+Windowed+Serde+to+public+APIs
>
> I'd love to hear from your opinions on the proposed APIs, and if you have
> already encountered this and have to implement your own serdes, does the
> current public API fit your needs.
>
>
> Thanks,
>
> -- Guozhang
>

Re: [kafka-clients] Re: [VOTE] 1.1.0 RC0

[Damian Guy]

Thanks Jun

On Fri, 2 Mar 2018 at 02:25 Jun Rao  wrote:

> KAFKA-6111 is now merged to 1.1 branch.
>
> Thanks,
>
> Jun
>
> On Thu, Mar 1, 2018 at 2:50 PM, Jun Rao  wrote:
>
>> Hi, Damian,
>>
>> It would also be useful to include KAFKA-6111, which prevents 
>> deleteLogDirEventNotifications
>> path to be deleted correctly from Zookeeper. The patch should be committed
>> later today.
>>
>> Thanks,
>>
>> Jun
>>
>> On Thu, Mar 1, 2018 at 1:47 PM, Damian Guy  wrote:
>>
>>> Thanks Jason. Assuming the system tests pass i'll cut RC1 tomorrow.
>>>
>>> Thanks,
>>> Damian
>>>
>>> On Thu, 1 Mar 2018 at 19:10 Jason Gustafson  wrote:
>>>
>>>> The fix has been merged to 1.1.
>>>>
>>>> Thanks,
>>>> Jason
>>>>
>>>> On Wed, Feb 28, 2018 at 11:35 AM, Damian Guy 
>>>> wrote:
>>>>
>>>> > Hi Jason,
>>>> >
>>>> > Ok - thanks. Let me know how you get on.
>>>> >
>>>> > Cheers,
>>>> > Damian
>>>> >
>>>> > On Wed, 28 Feb 2018 at 19:23 Jason Gustafson 
>>>> wrote:
>>>> >
>>>> > > Hey Damian,
>>>> > >
>>>> > > I think we should consider
>>>> > > https://issues.apache.org/jira/browse/KAFKA-6593
>>>> > > for the release. I have a patch available, but still working on
>>>> > validating
>>>> > > both the bug and the fix.
>>>> > >
>>>> > > -Jason
>>>> > >
>>>> > > On Wed, Feb 28, 2018 at 9:34 AM, Matthias J. Sax <
>>>> matth...@confluent.io>
>>>> > > wrote:
>>>> > >
>>>> > > > No. Both will be released.
>>>> > > >
>>>> > > > -Matthias
>>>> > > >
>>>> > > > On 2/28/18 6:32 AM, Marina Popova wrote:
>>>> > > > > Sorry, maybe a stupid question, but:
>>>> > > > >  I see that Kafka 1.0.1 RC2 is still not released, but now
>>>> 1.1.0 RC0
>>>> > is
>>>> > > > coming up...
>>>> > > > > Does it mean 1.0.1 will be abandoned and we should be looking
>>>> forward
>>>> > > to
>>>> > > > 1.1.0 instead?
>>>> > > > >
>>>> > > > > thanks!
>>>> > > > >
>>>> > > > > ​Sent with ProtonMail Secure Email.​
>>>> > > > >
>>>> > > > > ‐‐‐ Original Message ‐‐‐
>>>> > > > >
>>>> > > > > On February 26, 2018 6:28 PM, Vahid S Hashemian <
>>>> > > > vahidhashem...@us.ibm.com> wrote:
>>>> > > > >
>>>> > > > >> +1 (non-binding)
>>>> > > > >>
>>>> > > > >> Built the source and ran quickstart (including streams)
>>>> successfully
>>>> > > on
>>>> > > > >>
>>>> > > > >> Ubuntu (with both Java 8 and Java 9).
>>>> > > > >>
>>>> > > > >> I understand the Windows platform is not officially supported,
>>>> but I
>>>> > > ran
>>>> > > > >>
>>>> > > > >> the same on Windows 10, and except for Step 7 (Connect)
>>>> everything
>>>> > > else
>>>> > > > >>
>>>> > > > >> worked fine.
>>>> > > > >>
>>>> > > > >> There are a number of warning and errors (including
>>>> > > > >>
>>>> > > > >> java.lang.ClassNotFoundException). Here's the final error
>>>> message:
>>>> > > > >>
>>>> > > > >>> bin\\windows\\connect-standalone.bat
>>>> config\\connect-standalone.
>>>> > > > properties
>>>> > > > >>
>>>> > > > >> config\\connect-file-source.properties
>>>> config\\connect-file-sink.
>>>> > > > properties
>>>> > > > >>
>>>> > > > >> ...
>>>> > > > >>
>

[VOTE] 1.1.0 RC1

[Damian Guy]

Hello Kafka users, developers and client-developers,

This is the second candidate for release of Apache Kafka 1.1.0.

This is minor version release of Apache Kakfa. It Includes 29 new KIPs.
Please see the release plan for more details:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71764913

A few highlights:

* Significant Controller improvements (much faster and session expiration
edge cases fixed)
* Data balancing across log directories (JBOD)
* More efficient replication when the number of partitions is large
* Dynamic Broker Configs
* Delegation tokens (KIP-48)
* Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)

Release notes for the 1.1.0 release:
http://home.apache.org/~damianguy/kafka-1.1.0-rc1/RELEASE_NOTES.html

*** Please download, test and vote by Friday, March 9th, 5pm PT

Kafka's KEYS file containing PGP keys we use to sign the release:
http://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
http://home.apache.org/~damianguy/kafka-1.1.0-rc1/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/

* Javadoc:
http://home.apache.org/~damianguy/kafka-1.1.0-rc1/javadoc/

* Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
https://github.com/apache/kafka/tree/1.1.0-rc1


* Documentation:
http://kafka.apache.org/11/documentation.html

* Protocol:
http://kafka.apache.org/11/protocol.html

* Successful Jenkins builds for the 1.1 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-1.1-jdk7/68
System tests: https://jenkins.confluent.io/job/system-test-kafka/job/1.1/30/

/**

Thanks,
Damian Guy

Re: [VOTE] KIP-265: Make Windowed Serde to public APIs

[Damian Guy]

Thanks Guozhang +1

On Tue, 6 Mar 2018 at 00:26 Hu Xi  wrote:

> +1 (non-binding)
>
> 
> 发件人: Matthias J. Sax 
> 发送时间: 2018年3月6日 8:19
> 收件人: dev@kafka.apache.org
> 主题: Re: [VOTE] KIP-265: Make Windowed Serde to public APIs
>
> +1 (binding)
>
> Thanks for the KIP Guozhang!
>
> -Matthias
>
> On 3/5/18 2:41 PM, Bill Bejeck wrote:
> > Thanks for the KIP Guozhang.
> >
> > +1
> >
> > -Bill
> >
> > On Mon, Mar 5, 2018 at 5:11 PM, Ted Yu  wrote:
> >
> >> +1
> >>
> >> On Mon, Mar 5, 2018 at 1:46 PM, Guozhang Wang 
> wrote:
> >>
> >>> Hello all,
> >>>
> >>> I'd like to start voting on KIP-265, on making windowed serde to public
> >>> APIs of Kafka Streams. It involves a couple of new configs, plus a few
> >> new
> >>> public classes for windowed serializer and deserializer, and also
> adding
> >>> the corresponding console consumer options in order to fetch from a
> topic
> >>> written by a windowed store.
> >>>
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>> 265%3A+Make+Windowed+Serde+to+public+APIs
> >>>
> >>>
> >>>
> >>> Thanks,
> >>> -- Guozhang
> >>>
> >>
> >
>
>

Re: [VOTE] 1.1.0 RC1

[Damian Guy]

Hi Jeff,

Thanks, we will look into this.

Regards,
Damian

On Thu, 8 Mar 2018 at 18:27 Jeff Chao  wrote:

> Hello,
>
> We at Heroku have run 1.1.0 RC1 through our normal performance and
> regression test suite and have found performance to be comparable to 1.0.0.
>
> That said, we're however -1 (non-binding) since this release includes
> Zookeeper 3.4.11 <https://issues.apache.org/jira/browse/KAFKA-6390> which
> is affected by the critical regression ZOOKEEPER-2960
> <https://issues.apache.org/jira/browse/ZOOKEEPER-2960>. As 3.4.12 isn't
> released yet, it might be better to have 3.4.10 included instead.
>
> Jeff
> Heroku
>
>
> On Tue, Mar 6, 2018 at 1:19 PM, Ted Yu  wrote:
>
> > +1
> >
> > Checked signature
> > Ran test suite - apart from flaky testMetricsLeak, other tests passed.
> >
> > On Tue, Mar 6, 2018 at 2:45 AM, Damian Guy  wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the second candidate for release of Apache Kafka 1.1.0.
> > >
> > > This is minor version release of Apache Kakfa. It Includes 29 new KIPs.
> > > Please see the release plan for more details:
> > >
> > > https://cwiki.apache.org/confluence/pages/viewpage.
> > action?pageId=71764913
> > >
> > > A few highlights:
> > >
> > > * Significant Controller improvements (much faster and session
> expiration
> > > edge cases fixed)
> > > * Data balancing across log directories (JBOD)
> > > * More efficient replication when the number of partitions is large
> > > * Dynamic Broker Configs
> > > * Delegation tokens (KIP-48)
> > > * Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)
> > >
> > > Release notes for the 1.1.0 release:
> > > http://home.apache.org/~damianguy/kafka-1.1.0-rc1/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote by Friday, March 9th, 5pm PT
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > http://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > http://home.apache.org/~damianguy/kafka-1.1.0-rc1/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/
> > >
> > > * Javadoc:
> > > http://home.apache.org/~damianguy/kafka-1.1.0-rc1/javadoc/
> > >
> > > * Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
> > > https://github.com/apache/kafka/tree/1.1.0-rc1
> > >
> > >
> > > * Documentation:
> > > http://kafka.apache.org/11/documentation.html
> > >
> > > * Protocol:
> > > http://kafka.apache.org/11/protocol.html
> > >
> > > * Successful Jenkins builds for the 1.1 branch:
> > > Unit/integration tests:
> https://builds.apache.org/job/kafka-1.1-jdk7/68
> > > System tests: https://jenkins.confluent.io/
> > job/system-test-kafka/job/1.1/
> > > 30/
> > >
> > > /**
> > >
> > > Thanks,
> > > Damian Guy
> > >
> >
>

Re: Want to contribute

[Damian Guy]

Hi Eduardo,

There are usually JIRAs that have the newbie label. This would be a good
place to start.

Cheers,
Damian

On Fri, 9 Mar 2018 at 13:57 Eduardo Mello  wrote:

> Hello,
> I want to begin to code for open sources projects in my free time and I
> want to help the Apache Kafka!
> I already have read the How to Contribute and Code Guidelines.
> Do u guys have a label for "easy" Jira's issue or something like this?
>
> Att,
> Eduardo Mello
>

Re: Want to contribute

[Damian Guy]

Hi Sanket,

Done.
Thanks,
Damian

On Fri, 9 Mar 2018 at 14:56 Sanket Band  wrote:

> Hi Damian, Matthias
>
> Could you add me to the KAFKA project on jira, my jira id is sband
>
> Thanks
> Sanket
>
> On Fri, 9 Mar 2018, 20:23 Damian Guy,  wrote:
>
> > Hi Eduardo,
> >
> > There are usually JIRAs that have the newbie label. This would be a good
> > place to start.
> >
> > Cheers,
> > Damian
> >
> > On Fri, 9 Mar 2018 at 13:57 Eduardo Mello  wrote:
> >
> > > Hello,
> > > I want to begin to code for open sources projects in my free time and I
> > > want to help the Apache Kafka!
> > > I already have read the How to Contribute and Code Guidelines.
> > > Do u guys have a label for "easy" Jira's issue or something like this?
> > >
> > > Att,
> > > Eduardo Mello
> > >
> >
>

Re: [VOTE] KIP-267: Add Processor Unit Test Support to Kafka Streams Test Utils

[Damian Guy]

+1
On Fri, 9 Mar 2018 at 21:30, Matthias J. Sax  wrote:

> Guozhang, renaming the class is an internal change and I think it's not
> required to cover it in the KIP.
>
>
> +1 (binding)
>
>
> -Matthias
>
> On 3/9/18 1:26 PM, Guozhang Wang wrote:
> > Hi John,
> >
> > You mentioned you are going to do the renaming for the existing class
> > to InternalMockProcessorContext
> > but that seems not reflected on the KIP wiki, could you add that in the
> > proposed changes section?
> >
> > Other than that, I'm +1 on the KIP.
> >
> > Thanks!
> >
> > Guozhang
> >
> > On Fri, Mar 9, 2018 at 12:39 PM, Ted Yu  wrote:
> >
> >> +1
> >>  Original message From: Bill Bejeck 
> >> Date: 3/9/18  12:29 PM  (GMT-08:00) To: dev@kafka.apache.org Subject:
> Re:
> >> [VOTE] KIP-267: Add Processor Unit Test Support to Kafka Streams Test
> Utils
> >> Thanks for the KIP.
> >>
> >> +1
> >>
> >> -Bill
> >>
> >> On Fri, Mar 9, 2018 at 3:16 PM, John Roesler  wrote:
> >>
> >>> Hello all,
> >>>
> >>> I'd like to start voting on KIP-267, to introduce a
> MockProcessorContent
> >>> enabling Processor, Transformer, and ValueTransformer authors to
> >> unit-test
> >>> their components.
> >>>
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>> 267%3A+Add+Processor+Unit+Test+Support+to+Kafka+Streams+
> >>> Test+Utils?src=jira
> >>>
> >>> Thanks,
> >>> -John
> >>>
> >>
> >
> >
> >
>
>

[VOTE] 1.1.0 RC2

[Damian Guy]

Hello Kafka users, developers and client-developers,

This is the third candidate for release of Apache Kafka 1.1.0.

This is minor version release of Apache Kakfa. It Includes 29 new KIPs.
Please see the release plan for more details:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71764913

A few highlights:

* Significant Controller improvements (much faster and session expiration
edge cases fixed)
* Data balancing across log directories (JBOD)
* More efficient replication when the number of partitions is large
* Dynamic Broker Configs
* Delegation tokens (KIP-48)
* Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)

Release notes for the 1.1.0 release:
http://home.apache.org/~damianguy/kafka-1.1.0-rc2/RELEASE_NOTES.html

*** Please download, test and vote by Friday, March 16, 1pm PDT>

Kafka's KEYS file containing PGP keys we use to sign the release:
http://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
http://home.apache.org/~damianguy/kafka-1.1.0-rc2/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/

* Javadoc:
http://home.apache.org/~damianguy/kafka-1.1.0-rc2/javadoc/

* Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
https://github.com/apache/kafka/tree/1.1.0-rc
2


* Documentation:
http://kafka.apache.org/11/documentation.html


* Protocol:
http://kafka.apache.org/11/protocol.html


* Successful Jenkins builds for the 1.1 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-1.1-jdk7/78
System tests: https://jenkins.confluent.io/job/system-test-kafka/job/1.1/38/

/**

Thanks,
Damian


*

Re: [VOTE] 1.1.0 RC2

[Damian Guy]

Thanks for pointing out Satish. Links updated:



Hello Kafka users, developers and client-developers,

This is the third candidate for release of Apache Kafka 1.1.0.

This is minor version release of Apache Kakfa. It Includes 29 new KIPs.
Please see the release plan for more details:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546

A few highlights:

* Significant Controller improvements (much faster and session expiration
edge cases fixed)
* Data balancing across log directories (JBOD)
* More efficient replication when the number of partitions is large
* Dynamic Broker Configs
* Delegation tokens (KIP-48)
* Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)

Release notes for the 1.1.0 release:
http://home.apache.org/~damianguy/kafka-1.1.0-rc2/RELEASE_NOTES.html

*** Please download, test and vote by Friday, March 16, 1pm PDT>

Kafka's KEYS file containing PGP keys we use to sign the release:
http://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
http://home.apache.org/~damianguy/kafka-1.1.0-rc2/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/

* Javadoc:
http://home.apache.org/~damianguy/kafka-1.1.0-rc2/javadoc/

* Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
https://github.com/apache/kafka/tree/1.1.0-rc2


* Documentation:
http://kafka.apache.org/11/documentation.html
<http://kafka.apache.org/1/documentation.html>

* Protocol:
http://kafka.apache.org/11/protocol.html
<http://kafka.apache.org/1/protocol.html>

* Successful Jenkins builds for the 1.1 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-1.1-jdk7/78
System tests: https://jenkins.confluent.io/job/system-test-kafka/job/1.1/38/

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546

On Wed, 14 Mar 2018 at 04:41 Satish Duggana 
wrote:

> Hi Damian,
> Thanks for starting vote thread for 1.1.0 release.
>
> There may be a typo on the tag to be voted upon for this release candidate.
> I guess it should be https://github.com/apache/kafka/tree/1.1.0-rc2
> instead
> of https://github.com/apache/kafka/tree/1.1.0-rc.
>
> On Wed, Mar 14, 2018 at 8:27 AM, Satish Duggana 
> wrote:
>
> > Hi Damian,
> > Given release plan link in earlier mail is about 1.0 release. You may
> want
> > to replace that with 1.1.0 release plan link[1].
> >
> > 1 - https://cwiki.apache.org/confluence/pages/viewpage.
> > action?pageId=75957546
> >
> > Thanks,
> > Satish.
> >
> > On Wed, Mar 14, 2018 at 12:47 AM, Damian Guy 
> wrote:
> >
> >> Hello Kafka users, developers and client-developers,
> >>
> >> This is the third candidate for release of Apache Kafka 1.1.0.
> >>
> >> This is minor version release of Apache Kakfa. It Includes 29 new KIPs.
> >> Please see the release plan for more details:
> >>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71764913
> >>
> >> A few highlights:
> >>
> >> * Significant Controller improvements (much faster and session
> expiration
> >> edge cases fixed)
> >> * Data balancing across log directories (JBOD)
> >> * More efficient replication when the number of partitions is large
> >> * Dynamic Broker Configs
> >> * Delegation tokens (KIP-48)
> >> * Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)
> >>
> >> Release notes for the 1.1.0 release:
> >> http://home.apache.org/~damianguy/kafka-1.1.0-rc2/RELEASE_NOTES.html
> >>
> >> *** Please download, test and vote by Friday, March 16, 1pm PDT>
> >>
> >> Kafka's KEYS file containing PGP keys we use to sign the release:
> >> http://kafka.apache.org/KEYS
> >>
> >> * Release artifacts to be voted upon (source and binary):
> >> http://home.apache.org/~damianguy/kafka-1.1.0-rc2/
> >>
> >> * Maven artifacts to be voted upon:
> >> https://repository.apache.org/content/groups/staging/
> >>
> >> * Javadoc:
> >> http://home.apache.org/~damianguy/kafka-1.1.0-rc2/javadoc/
> >>
> >> * Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
> >> https://github.com/apache/kafka/tree/1.1.0-rc
> >> <https://github.com/apache/kafka/tree/1.1.0-rc1>2
> >>
> >>
> >> * Documentation:
> >> http://kafka.apache.org/11/documentation.html
> >> <http://kafka.apache.org/1/documentation.html>
> >>
> >> * Protocol:
> >> http://kafka.apache.org/11/protocol.html
> >> <http://kafka.apache.org/1/protocol.html>
> >>
> >> * Successful Jenkins builds for the 1.1 branch:
> >> Unit/integration tests: https://builds.apache.org/job/kafka-1.1-jdk7/78
> >> System tests: https://jenkins.confluent.io/j
> >> ob/system-test-kafka/job/1.1/38/
> >>
> >> /**
> >>
> >> Thanks,
> >> Damian
> >>
> >>
> >> *
> >>
> >
> >
>

Re: [VOTE] 1.1.0 RC2

[Damian Guy]

Hi All,

I'm going to close this vote thread as there has been another blocker
merged to 1.1. I'll create another RC later today.

Thanks,
Damian

On Thu, 15 Mar 2018 at 03:31 Satish Duggana 
wrote:

> +1 (non-binding)
>
> - Ran testAll/releaseTarGzAll on 1.1.0-RC2 tag
> - Ran through quickstart of core/streams
> - Ran a cluster from binaries built form sources
>   - with multiple operations.
>
> Thanks,
> Satish.
>
>
> On Thu, Mar 15, 2018 at 3:25 AM, Harsha  wrote:
>
> > +1
> > Ran tests
> > Ran a 3 node cluster to test basic operations.
> >
> > Thanks,
> > Harsha
> >
> > On Wed, Mar 14, 2018, at 9:04 AM, Ted Yu wrote:
> > > +1
> > >
> > > Ran test suite - passed (apart from testMetricsLeak which is flaky).
> > >
> > > On Wed, Mar 14, 2018 at 3:30 AM, Damian Guy 
> > wrote:
> > >
> > > > Thanks for pointing out Satish. Links updated:
> > > >
> > > > 
> > > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the third candidate for release of Apache Kafka 1.1.0.
> > > >
> > > > This is minor version release of Apache Kakfa. It Includes 29 new
> KIPs.
> > > > Please see the release plan for more details:
> > > >
> > > > https://cwiki.apache.org/confluence/pages/viewpage.
> > action?pageId=75957546
> > > >
> > > > A few highlights:
> > > >
> > > > * Significant Controller improvements (much faster and session
> > expiration
> > > > edge cases fixed)
> > > > * Data balancing across log directories (JBOD)
> > > > * More efficient replication when the number of partitions is large
> > > > * Dynamic Broker Configs
> > > > * Delegation tokens (KIP-48)
> > > > * Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)
> > > >
> > > > Release notes for the 1.1.0 release:
> > > > http://home.apache.org/~damianguy/kafka-1.1.0-rc2/RELEASE_NOTES.html
> > > >
> > > > *** Please download, test and vote by Friday, March 16, 1pm PDT>
> > > >
> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > http://kafka.apache.org/KEYS
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > > http://home.apache.org/~damianguy/kafka-1.1.0-rc2/
> > > >
> > > > * Maven artifacts to be voted upon:
> > > > https://repository.apache.org/content/groups/staging/
> > > >
> > > > * Javadoc:
> > > > http://home.apache.org/~damianguy/kafka-1.1.0-rc2/javadoc/
> > > >
> > > > * Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
> > > > https://github.com/apache/kafka/tree/1.1.0-rc2
> > > >
> > > >
> > > > * Documentation:
> > > > http://kafka.apache.org/11/documentation.html
> > > > <http://kafka.apache.org/1/documentation.html>
> > > >
> > > > * Protocol:
> > > > http://kafka.apache.org/11/protocol.html
> > > > <http://kafka.apache.org/1/protocol.html>
> > > >
> > > > * Successful Jenkins builds for the 1.1 branch:
> > > > Unit/integration tests: https://builds.apache.org/job/
> > kafka-1.1-jdk7/78
> > > > System tests: https://jenkins.confluent.io/
> > job/system-test-kafka/job/1.1/
> > > > 38/
> > > >
> > > > https://cwiki.apache.org/confluence/pages/viewpage.
> > action?pageId=75957546
> > > >
> > > > On Wed, 14 Mar 2018 at 04:41 Satish Duggana <
> satish.dugg...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Damian,
> > > > > Thanks for starting vote thread for 1.1.0 release.
> > > > >
> > > > > There may be a typo on the tag to be voted upon for this release
> > > > candidate.
> > > > > I guess it should be
> https://github.com/apache/kafka/tree/1.1.0-rc2
> > > > > instead
> > > > > of https://github.com/apache/kafka/tree/1.1.0-rc.
> > > > >
> > > > > On Wed, Mar 14, 2018 at 8:27 AM, Satish Duggana <
> > > > satish.dugg...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Damian,
> > > > > > Given release plan 

Subject: [VOTE] 1.1.0 RC3

[Damian Guy]

Hello Kafka users, developers and client-developers,

This is the fourth candidate for release of Apache Kafka 1.1.0.

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546

A few highlights:

* Significant Controller improvements (much faster and session expiration
edge cases fixed)
* Data balancing across log directories (JBOD)
* More efficient replication when the number of partitions is large
* Dynamic Broker Configs
* Delegation tokens (KIP-48)
* Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)


Release notes for the 1.1.0 release:
http://home.apache.org/~damianguy/kafka-1.1.0-rc3/RELEASE_NOTES.html

*** Please download, test and vote by Monday, March 19, 9am PDT

Kafka's KEYS file containing PGP keys we use to sign the release:
http://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
http://home.apache.org/~damianguy/kafka-1.1.0-rc3/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/

* Javadoc:
http://home.apache.org/~damianguy/kafka-1.1.0-rc3/javadoc/

* Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
https://github.com/apache/kafka/tree/1.1.0-rc3


* Documentation:
http://kafka.apache.org/11/documentation.html


* Protocol:
http://kafka.apache.org/11/protocol.html



* Successful Jenkins builds for the 1.1 branch:
Unit/integration tests: https://builds.apache.org/job/kafka-1.1-jdk7/82/

Note: I'll update with passing system tests link once successful. If not
then we can do another RC

/**

Thanks,
Damian

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

[Damian Guy]

Hi Debasish,

Thanks for the KIP - will be a great addition to streams. I've only had a
quick scan, but seeing as the Scala classes are going to be in their own
package could we drop the S at the end of the class names?

Thanks,
Damian


On Fri, 16 Mar 2018 at 15:25 Debasish Ghosh 
wrote:

> Hi -
>
> A new KIP, KIP-270 is up for discussion:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-270+-+A+Scala+Wrapper+Library+for+Kafka+Streams
>
> The relevant JIRA issue: https://issues.apache.org/jira/browse/KAFKA-6670
>
> The library as proposed in the KIP has been implemented at
> https://github.com/lightbend/kafka-streams-scala and the current release
> is
> 0.2.0 (
> https://github.com/lightbend/kafka-streams-scala/releases/tag/v0.2.0).
> We at Lightbend has been using it since quite some time now.
>
> regards.
>
> --
> Debasish Ghosh
> Principal Engineer
>
> Twitter: @debasishg
> Blog: http://debasishg.blogspot.com
> Code: https://github.com/debasishg
>

Re: Subject: [VOTE] 1.1.0 RC3

[Damian Guy]

We have a green system test build -
https://jenkins.confluent.io/job/system-test-kafka/job/1.1/43/
<https://jenkins.confluent.io/job/system-test-kafka/job/1.1/>

Thanks,
Damian

On Fri, 16 Mar 2018 at 22:10 Vahid S Hashemian 
wrote:

> Hi Damian,
>
> Thanks for running the release.
>
> I tried building from source and running the quick start on Linux &
> Windows with both Java 8 & 9.
> Here's the result:
>
> +-+-+-+
> | |  Linux  | Windows |
> + +-+-+
> | | J8 | J9 | J8 | J9 |
> +-+++++
> |  Build  |  + |  + |  + |  + |
> +-+++++
> |  Single broker  |  + |  + |  + |  + |
> | produce/consume |||||
> +-+++++
> | Connect |  + |  ? |  - |  - |
> +-+++++
> | Streams |  + |  + |  + |  + |
> +-+++++
>
> ?: Connect quickstart on Linux with Java 9 runs but the connect tool
> throws a bunch of exceptions (https://www.codepile.net/pile/yVg8XJB8)
> -: Connect quickstart on Windows fails (Java 8:
> https://www.codepile.net/pile/xJGra6BP, Java 9:
> https://www.codepile.net/pile/oREYeORK)
>
> Given that Windows is not an officially supported platform, and the
> exceptions with Linux/Java 9 are not breaking the functionality, my vote
> is a +1 (non-binding).
>
> Thanks.
> --Vahid
>
>
>
>
> From:   Damian Guy 
> To: dev@kafka.apache.org, us...@kafka.apache.org,
> kafka-clie...@googlegroups.com
> Date:   03/15/2018 07:55 AM
> Subject:Subject: [VOTE] 1.1.0 RC3
>
>
>
> Hello Kafka users, developers and client-developers,
>
> This is the fourth candidate for release of Apache Kafka 1.1.0.
>
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_pages_viewpage.action-3FpageId-3D75957546&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2sgX46mjZPws01U&s=cKgJtQXXRauZ3HSAoSbsC9SLVTAkO-pbLdPrOCBuJzE&e=
>
>
> A few highlights:
>
> * Significant Controller improvements (much faster and session expiration
> edge cases fixed)
> * Data balancing across log directories (JBOD)
> * More efficient replication when the number of partitions is large
> * Dynamic Broker Configs
> * Delegation tokens (KIP-48)
> * Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)
>
>
> Release notes for the 1.1.0 release:
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__home.apache.org_-7Edamianguy_kafka-2D1.1.0-2Drc3_RELEASE-5FNOTES.html&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2sgX46mjZPws01U&s=26FgbzRhKImhxyEkB4KzDPG-l8W_Y99xU6LykOAgpFI&e=
>
>
> *** Please download, test and vote by Monday, March 19, 9am PDT
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__kafka.apache.org_KEYS&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2sgX46mjZPws01U&s=xlnrfgxVFMRCKk8pTOhujyC-Um4ogtsxK6Xwks6mc3U&e=
>
>
> * Release artifacts to be voted upon (source and binary):
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__home.apache.org_-7Edamianguy_kafka-2D1.1.0-2Drc3_&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2sgX46mjZPws01U&s=ulHUeYnWIp28Gsn4VV1NK3FrGV4Jn5rUpuU6tvgekME&e=
>
>
> * Maven artifacts to be voted upon:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__repository.apache.org_content_groups_staging_&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2sgX46mjZPws01U&s=G9o4hXVXF0bjL_a3Wocod9GUEfy9WBBgoGa2u6yFKQw&e=
>
>
> * Javadoc:
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__home.apache.org_-7Edamianguy_kafka-2D1.1.0-2Drc3_javadoc_&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2sgX46mjZPws01U&s=2auaI4IIJhEORGYm1Kdpxt5TDHh0PzSvtK77lC3SJVY&e=
>
>
> * Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_kafka_tree_1.1.0-2Drc3&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2sgX46mjZPws01U&s=h7G8XPD8vAWl_gqySi2Iocag5NnP32IT_PyirPC3Lss&

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

[Damian Guy]

+1

On Wed, 21 Mar 2018 at 01:44 abel-sun  wrote:

>
>Thanks you of your offer ,agree with you!
>
> On 2018/03/21 00:56:11, Richard Yu  wrote:
> > Hi Matthias,
> > Thanks for setting up the upgrade path.
> >
> > +1 (non-binding)
> >
> > On Tue, Mar 20, 2018 at 3:42 PM, Matthias J. Sax 
> > wrote:
> >
> > > Hi,
> > >
> > > I would like to start the vote for KIP-268:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 268%3A+Simplify+Kafka+Streams+Rebalance+Metadata+Upgrade
> > >
> > > PR https://github.com/apache/kafka/pull/4636 contains the fixes to
> > > upgrade from metadata version 1 to 2. Some tests are still missing but
> > > I'll add them asap.
> > >
> > > For "version probing" including new metadata version 3 I plan to do a
> > > follow-up PR after PR-4636 is merged.
> > >
> > >
> > > -Matthias
> > >
> > >
> >
>

Re: Subject: [VOTE] 1.1.0 RC3

[Damian Guy]

Hi,

Closing of this vote thread as there will be another RC.

Thanks,
Damian

On Mon, 19 Mar 2018 at 23:47 Ismael Juma  wrote:

> Vahid,
>
> The Java 9 Connect issue is similar to the one being fixed for Trogdor in
> the following PR:
>
> https://github.com/apache/kafka/pull/4725
>
> We need to do something similar for Connect.
>
> Ismael
>
> On Fri, Mar 16, 2018 at 3:10 PM, Vahid S Hashemian <
> vahidhashem...@us.ibm.com> wrote:
>
> > Hi Damian,
> >
> > Thanks for running the release.
> >
> > I tried building from source and running the quick start on Linux &
> > Windows with both Java 8 & 9.
> > Here's the result:
> >
> > +-+-+-+
> > | |  Linux  | Windows |
> > + +-+-+
> > | | J8 | J9 | J8 | J9 |
> > +-+++++
> > |  Build  |  + |  + |  + |  + |
> > +-+++++
> > |  Single broker  |  + |  + |  + |  + |
> > | produce/consume |||||
> > +-+++++
> > | Connect |  + |  ? |  - |  - |
> > +-+++++
> > | Streams |  + |  + |  + |  + |
> > +-+++++
> >
> > ?: Connect quickstart on Linux with Java 9 runs but the connect tool
> > throws a bunch of exceptions (https://www.codepile.net/pile/yVg8XJB8)
> > -: Connect quickstart on Windows fails (Java 8:
> > https://www.codepile.net/pile/xJGra6BP, Java 9:
> > https://www.codepile.net/pile/oREYeORK)
> >
> > Given that Windows is not an officially supported platform, and the
> > exceptions with Linux/Java 9 are not breaking the functionality, my vote
> > is a +1 (non-binding).
> >
> > Thanks.
> > --Vahid
> >
> >
> >
> >
> > From:   Damian Guy 
> > To: dev@kafka.apache.org, us...@kafka.apache.org,
> > kafka-clie...@googlegroups.com
> > Date:   03/15/2018 07:55 AM
> > Subject:Subject: [VOTE] 1.1.0 RC3
> >
> >
> >
> > Hello Kafka users, developers and client-developers,
> >
> > This is the fourth candidate for release of Apache Kafka 1.1.0.
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.
> > apache.org_confluence_pages_viewpage.action-3FpageId-
> > 3D75957546&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_
> >
> itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2s
> > gX46mjZPws01U&s=cKgJtQXXRauZ3HSAoSbsC9SLVTAkO-pbLdPrOCBuJzE&e=
> >
> >
> > A few highlights:
> >
> > * Significant Controller improvements (much faster and session expiration
> > edge cases fixed)
> > * Data balancing across log directories (JBOD)
> > * More efficient replication when the number of partitions is large
> > * Dynamic Broker Configs
> > * Delegation tokens (KIP-48)
> > * Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)
> >
> >
> > Release notes for the 1.1.0 release:
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__home.
> > apache.org_-7Edamianguy_kafka-2D1.1.0-2Drc3_RELEASE-5FNOTES.
> > html&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-
> > kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2sgX46mjZPws01U&s=
> > 26FgbzRhKImhxyEkB4KzDPG-l8W_Y99xU6LykOAgpFI&e=
> >
> >
> > *** Please download, test and vote by Monday, March 19, 9am PDT
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__kafka.
> > apache.org_KEYS&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_
> >
> itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=Qn2GySTKcOV5MFr3WDl63BDv7pTd2s
> > gX46mjZPws01U&s=xlnrfgxVFMRCKk8pTOhujyC-Um4ogtsxK6Xwks6mc3U&e=
> >
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__home.
> > apache.org_-7Edamianguy_kafka-2D1.1.0-2Drc3_&d=DwIBaQ&c=jf_
> > iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=
> > Qn2GySTKcOV5MFr3WDl63BDv7pTd2sgX46mjZPws01U&s=
> > ulHUeYnWIp28Gsn4VV1NK3FrGV4Jn5rUpuU6tvgekME&e=
> >
> >
> > * Maven artifacts to be voted upon:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__
> > repository.apache.org_content_groups_staging_&d=DwIBaQ&c=jf_
> > iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE

Re: [VOTE] KIP-274: Kafka Streams Skipped Records Metrics

[Damian Guy]

+1

On Thu, 5 Apr 2018 at 01:56 Matthias J. Sax  wrote:

> +1 (binding) on the latest proposal.
>
>
> -Matthias
>
>
> On 4/2/18 12:10 PM, Richard Yu wrote:
> > +1
> >
> > On Mon, Apr 2, 2018 at 8:42 AM, Guozhang Wang 
> wrote:
> >
> >> +1 (binding).
> >>
> >> On Mon, Apr 2, 2018 at 7:22 AM, Ted Yu  wrote:
> >>
> >>> +1
> >>>
> >>> On Mon, Apr 2, 2018 at 7:11 AM, Bill Bejeck  wrote:
> >>>
>  Thanks for the KIP.
> 
>  +1
> 
>  -Bill
> 
>  On Mon, Apr 2, 2018 at 10:09 AM, John Roesler 
> >> wrote:
> 
> > Dear Kafka community,
> >
> > I am proposing KIP-274 to improve visibility when Streams skips
> >> invalid
> > records.
> >
> > The proposal is to simplify the metrics and add warning logs.
> >
> > Please find details in the wiki:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 274%3A+Kafka+Streams+Skipped+Records+Metrics
> >
> > Thanks,
> >
> > -John
> >
> 
> >>>
> >>
> >>
> >>
> >> --
> >> -- Guozhang
> >>
> >
>
>

Re: [VOTE] KIP-284 Set default retention ms for Streams repartition topics to Long.MAX_VALUE

[Damian Guy]

+1 binding

On Mon, 9 Apr 2018 at 21:20 Guozhang Wang  wrote:

> +1 (binding)
>
> On Mon, Apr 9, 2018 at 11:21 AM, Matthias J. Sax 
> wrote:
>
> > +1 (binding)
> >
> > -Matthias
> >
> > On 4/9/18 8:57 AM, Bill Bejeck wrote:
> > > Thanks for the KIP.
> > >
> > > +1
> > >
> > > -Bill
> > >
> > > On Mon, Apr 9, 2018 at 7:57 AM, zhenya Sun  wrote:
> > >
> > >> +1
> > >>
> > >>
> > >> from my iphone！
> > >> On 04/09/2018 17:27, khaireddine Rezgui wrote:
> > >> Hi guys,
> > >>
> > >> I created this thread to get your agreement about the new default
> value
> > of
> > >> the retention_ms in kafka stream repartion topics.
> > >>
> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >> 284%3A+Set+default+retention+ms+for+Streams+repartition+
> > >> topics+to+Long.MAX_VALUE
> > >>
> > >> Thanks,
> > >>
> > >
> >
> >
>
>
> --
> -- Guozhang
>

Re: [VOTE] KIP-270 A Scala wrapper library for Kafka Streams

[Damian Guy]

Thanks for the KIP Debasish - +1 binding

On Wed, 11 Apr 2018 at 12:16 Ted Yu  wrote:

> +1
>  Original message From: Debasish Ghosh <
> debasish.gh...@lightbend.com> Date: 4/11/18  3:09 AM  (GMT-08:00) To:
> dev@kafka.apache.org Subject: [VOTE] KIP-270 A Scala wrapper library for
> Kafka Streams
> Hello everyone -
>
> This is in continuation to the discussion regarding
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-270+-+A+Scala+Wrapper+Library+for+Kafka+Streams
> ,
> which is a KIP for implementing a Scala wrapper library for Kafka Streams.
>
> We have had a PR (https://github.com/apache/kafka/pull/4756) for quite
> some
> time now and we have had lots of discussions and suggested improvements.
> Thanks to all who participated in the discussion and came up with all the
> suggestions for improvements.
>
> The purpose of this thread is to get an agreement on the implementation and
> have it included as part of Kafka.
>
> Looking forward ..
>
> regards.
>
> --
> Debasish Ghosh
> Principal Engineer
>
> Twitter: @debasishg
> Blog: http://debasishg.blogspot.com
> Code: https://github.com/debasishg
>

Re: [VOTE] KIP-279: Fix log divergence between leader and follower after fast leader fail over

[Damian Guy]

Thanks Anna +1
On Sun, 15 Apr 2018 at 15:40, Guozhang Wang  wrote:

> Anna, thanks for the KIP! +1 from me.
>
> Just one minor comment: `Broker A will respond with offset 11`, seems it
> should be `21`.
>
> On Sun, Apr 15, 2018 at 3:48 AM, Dong Lin  wrote:
>
> > Thanks for the KIP! LGTM. +1
> >
> > On Sat, Apr 14, 2018 at 5:31 PM, Ted Yu  wrote:
> >
> > > +1
> > >
> > > On Sat, Apr 14, 2018 at 3:54 PM, Anna Povzner 
> wrote:
> > >
> > > > Hi All,
> > > >
> > > >
> > > > I would like to start the vote on KIP-279: Fix log divergence between
> > > > leader and follower after fast leader fail over.
> > > >
> > > >
> > > > For reference, here's the KIP wiki:
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 279%3A+Fix+log+divergence+between+leader+and+follower+
> > > > after+fast+leader+fail+over
> > > >
> > > >
> > > >
> > > > and discussion thread:
> > > > https://www.mail-archive.com/dev@kafka.apache.org/msg86753.html
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Anna
> > > >
> > >
> >
>
>
>
> --
> -- Guozhang
>

Re: [VOTE] KIP-276 Add StreamsConfig prefix for different consumers

[Damian Guy]

Thanks +1

On Fri, 20 Apr 2018 at 23:41 Bill Bejeck  wrote:

> +1
>
> Thanks,
> Bill
>
> On Thu, Apr 19, 2018 at 9:50 AM, Boyang Chen  wrote:
>
> > Thanks guys！
> >
> > Sent from my iPhone
> >
> > > On Apr 19, 2018, at 8:28 PM, Matthias J. Sax 
> > wrote:
> > >
> > > +1 (binding)
> > >
> > > -Matthias
> > >
> > >> On 4/19/18 9:08 AM, Ted Yu wrote:
> > >> +1
> > >>  Original message From: Guozhang Wang <
> > wangg...@gmail.com> Date: 4/18/18  10:18 PM  (GMT-08:00) To:
> > dev@kafka.apache.org Subject: Re: [VOTE] KIP-276 Add StreamsConfig
> prefix
> > for different consumers
> > >> Thanks Boyang, +1 from me.
> > >>
> > >>> On Wed, Apr 18, 2018 at 4:43 PM, Boyang Chen 
> > wrote:
> > >>>
> > >>> Hey guys,
> > >>>
> > >>>
> > >>> sorry I forgot to include a summary of this KIP.  Basically the idea
> > is to
> > >>> separate the stream config for different internal consumers of Kafka
> > >>> Streams by supplying prefixes . We currently have
> > >>>
> > >>> (1) "main.consumer." for the main consumer
> > >>>
> > >>> (2) "restore.consumer." for the restore consumer
> > >>>
> > >>> (3) "global.consumer." for the global consumer
> > >>>
> > >>>
> > >>> Best,
> > >>>
> > >>> Boyang
> > >>>
> > >>> 
> > >>> From: Boyang Chen 
> > >>> Sent: Tuesday, April 17, 2018 12:18 PM
> > >>> To: dev@kafka.apache.org
> > >>> Subject: [VOTE] KIP-276 Add StreamsConfig prefix for different
> > consumers
> > >>>
> > >>>
> > >>> Hey friends.
> > >>>
> > >>>
> > >>> I would like to start a vote on KIP 276: add StreamsConfig prefix for
> > >>> different consumers.
> > >>>
> > >>> KIP: here > >>> 276+Add+StreamsConfig+prefix+for+different+consumers>
> > >>>
> > >>> Pull request: here
> > >>>
> > >>> Jira: here
> > >>>
> > >>>
> > >>> Let me know if you have questions.
> > >>>
> > >>>
> > >>> Thank you!
> > >>>
> > >>>
> > >>>  > >>> 276+Add+StreamsConfig+prefix+for+different+consumers>
> > >>>
> > >>>
> > >>
> > >>
> > >
> >
>

Re: [VOTE] KIP-353: Allow Users to Configure Multi-Streams Timestamp Synchronization Behavior

[Damian Guy]

+1

On Tue, 14 Aug 2018 at 19:35 Ted Yu  wrote:

> +1
>  Original message From: Bill Bejeck 
> Date: 8/14/18  11:09 AM  (GMT-08:00) To: dev@kafka.apache.org Subject:
> Re: [VOTE] KIP-353: Allow Users to Configure Multi-Streams Timestamp
> Synchronization Behavior
> +1
>
> Thanks,
> Bill
>
> On Thu, Aug 9, 2018 at 4:20 PM John Roesler  wrote:
>
> > +1 non-binding
> >
> > On Thu, Aug 9, 2018 at 3:14 PM Matthias J. Sax 
> > wrote:
> >
> > > +1 (binding)
> > >
> > > On 8/9/18 11:57 AM, Guozhang Wang wrote:
> > > > Hello all,
> > > >
> > > > I would like to start the voting processing on the following KIP, to
> > > allow
> > > > users control when a task can be processed based on its buffered
> > records,
> > > > and how the stream time of a task be advanced.
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 353%3A+Improve+Kafka+Streams+Timestamp+Synchronization
> > > >
> > > >
> > > >
> > > > Thanks,
> > > > -- Guozhang
> > > >
> > >
> > >
> >
>

Re: [VOTE] KIP-356: Add withCachingDisabled() to StoreBuilder

[Damian Guy]

+1

On Tue, 14 Aug 2018 at 22:58 Matthias J. Sax  wrote:

> +1 (binding)
>
> On 8/14/18 11:16 AM, Eno Thereska wrote:
> > +1 (non binding)
> >
> > Thanks
> > Eno
> >
> > On Tue, Aug 14, 2018 at 10:53 AM, Bill Bejeck  wrote:
> >
> >> Thanks for the KIP.
> >>
> >> +1
> >>
> >> -Bill
> >>
> >> On Tue, Aug 14, 2018 at 1:42 PM Guozhang Wang 
> wrote:
> >>
> >>> Hello folks,
> >>>
> >>> I'd like to start a voting thread on the following KIP:
> >>>
> >>>
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-356%3A+Add+
> >> withCachingDisabled%28%29+to+StoreBuilder
> >>>
> >>> It is a pretty straightforward one, adding a missing API to
> StoreBuilder
> >>> which should actually be added at the very beginning but somehow was
> >> lost.
> >>> Hence I skipped the DISCUSS process of it. But if you have any
> feedbacks
> >>> please feel free to share as well.
> >>>
> >>>
> >>>
> >>> -- Guozhang
> >>>
> >>
> >
>
>

Re: [VOTE] KIP-365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde

[Damian Guy]

+1

On Sun, 2 Sep 2018 at 15:20 Matthias J. Sax  wrote:

> +1 (binding)
>
> On 9/1/18 2:40 PM, Guozhang Wang wrote:
> > +1 (binding).
> >
> > On Mon, Aug 27, 2018 at 5:20 PM, Dongjin Lee  wrote:
> >
> >> +1 (non-binding)
> >>
> >> On Tue, Aug 28, 2018 at 8:53 AM Bill Bejeck  wrote:
> >>
> >>> +1
> >>>
> >>> -Bill
> >>>
> >>> On Mon, Aug 27, 2018 at 3:24 PM Ted Yu  wrote:
> >>>
>  +1
> 
>  On Mon, Aug 27, 2018 at 12:18 PM John Roesler 
> >> wrote:
> 
> > +1 (non-binding)
> >
> > On Sat, Aug 25, 2018 at 1:16 PM Joan Goyeau  wrote:
> >
> >> Hi,
> >>
> >> We want to make sure that we always have a serde for all
> >>> Materialized,
> >> Serialized, Joined, Consumed and Produced.
> >> For that we can make use of the implicit parameters in Scala.
> >>
> >> KIP:
> >>
> >>
> >
> 
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>
> 365%3A+Materialized%2C+Serialized%2C+Joined%2C+Consumed+and+Produced+with+
> >> implicit+Serde
> >>
> >> Github PR: https://github.com/apache/kafka/pull/5551
> >>
> >> Please make your votes.
> >> Thanks
> >>
> >
> 
> >>>
> >>> --
> >>> *Dongjin Lee*
> >>>
> >>> *A hitchhiker in the mathematical world.*
> >>>
> >>> *github:  github.com/dongjinleekr
> >>> linkedin: kr.linkedin.com/in/
> >> dongjinleekr
> >>> slideshare:
> www.slideshare.net/
> >> dongjinleekr
> >>> *
> >>>
> >>
> >
> >
> >
>
>

Re: [VOTE] KIP-366 - Make FunctionConversations private

[Damian Guy]

+1 (binding)

Thanks

On Fri, 14 Sep 2018 at 10:00 Joan Goyeau  wrote:

> Ok so we have only one binding vote up to now. I guess we need at least 3
> binding votes.
>
> Thanks
>
> On Fri, 14 Sep 2018 at 09:59 Joan Goyeau  wrote:
>
> > Matt, This is now all updated.
> >
> > On Fri, 7 Sep 2018 at 03:34 Matthias J. Sax 
> wrote:
> >
> >> Can you please update the KIP accordingly?
> >>
> >> It still says "make private" instead of "deprecating"
> >>
> >> -Matthias
> >>
> >> On 9/6/18 10:07 AM, Attila Sasvári wrote:
> >> > +1 (non-binding)
> >> >
> >> > On Thu, Sep 6, 2018 at 6:38 PM Guozhang Wang 
> >> wrote:
> >> >
> >> >> +1 for deprecating and copying the class over to internals.
> >> >>
> >> >> On Thu, Sep 6, 2018 at 6:56 AM, Bill Bejeck 
> wrote:
> >> >>
> >> >>> +1
> >> >>>
> >> >>> -Bill
> >> >>>
> >> >>> On Thu, Sep 6, 2018 at 4:29 AM Joan Goyeau  wrote:
> >> >>>
> >>  Sournds good, I'll make the deprecation and copy the class over.
> >> 
> >>  Thanks
> >> 
> >>  On Wed, 5 Sep 2018 at 22:48 John Roesler 
> wrote:
> >> 
> >> > I'm a +1 (non-binding) because we doubt the class is in use.
> >> >
> >> > If you decide to copy it to a private version and deprecate the
> >> >>> original
> >> > instead, as Matthias suggested, I would still be a +1.
> >> >
> >> > Thanks,
> >> > -John
> >> >
> >> > On Sat, Sep 1, 2018 at 6:47 AM Joan Goyeau 
> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> As pointed out in this comment
> >> >> https://github.com/apache/kafka/pull/5539#discussion_r212380648
> >> >>> "This
> >> >> class
> >> >> was already defaulted to public visibility, and we can't retract
> it
> >>  now,
> >> >> without a KIP.", the object FunctionConversions is only of
> internal
> >> >>> use
> >> > and
> >> >> therefore should be private to the lib only so that we can do
> >> >> changes
> >> >> without going through KIP like this one.
> >> >>
> >> >> KIP:
> >> >>
> >> >>
> >> >
> >>  https://cwiki.apache.org/confluence/display/KAFKA/KIP-366%3A+Make+
> >> >>> FunctionConversions+private
> >> >>
> >> >> Please make your votes.
> >> >> Thanks
> >> >>
> >> >
> >> 
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> -- Guozhang
> >> >>
> >> >
> >>
> >>
>

Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times

[Damian Guy]

Thanks +1 (binding)

On Sun, 16 Sep 2018 at 19:37 Nikolay Izhikov  wrote:

> Dear commiters,
>
> I got two binding +1 in [VOTE] thread for this KIP [1].
> I still need one more.
>
> Please, take a look at KIP.
>
> [1]
> https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
>
> В Чт, 13/09/2018 в 19:33 +0300, Nikolay Izhikov пишет:
> > Fixed.
> >
> > Thanks, for help!
> >
> > Please, take a look and vote.
> >
> > В Чт, 13/09/2018 в 08:40 -0700, Matthias J. Sax пишет:
> > > No need to start a new voting thread :)
> > >
> > > For the KIP update, I think it should be:
> > >
> > > > ReadOnlyWindowStore {
> > > > //Deprecated methods.
> > > > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetch(K from, K to, long
> timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetchAll(long timeFrom, long
> timeTo);
> > > >
> > > > //New methods.
> > > > WindowStoreIterator fetch(K key, Instant from, Duration
> duration) throws IllegalArgumentException;
> > > > KeyValueIterator, V> fetch(K from, K to, Instant
> from, Duration duration) throws IllegalArgumentException;
> > > > KeyValueIterator, V> fetchAll(Instant from, Duration
> duration) throws IllegalArgumentException;
> > > > }
> > > >
> > > >
> > > > WindowStore {
> > > > //New methods.
> > > > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetch(K from, K to, long
> timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetchAll(long timeFrom, long
> timeTo);
> > > > }
> > >
> > > Ie, long-versions are replaced with Instant/Duration in
> > > `ReadOnlyWindowStore`, and `long` method are added in `WindowStore` --
> > > this way, we effectively "move" the long-versions from
> > > `ReadOnlyWindowStore` to `WindowStore`.
> > >
> > > -Matthias
> > >
> > > On 9/13/18 8:08 AM, Nikolay Izhikov wrote:
> > > > Hello, Matthias.
> > > >
> > > > > I like the KIP as-is. Feel free to start a VOTE thread.
> > > >
> > > >
> > > > I'm already started one [1].
> > > > Can you vote in it or I should create a new one?
> > > >
> > > >
> > > > I've updated KIP.
> > > > This has been changed:
> > > >
> > > > ReadOnlyWindowStore {
> > > > //Deprecated methods.
> > > > WindowStoreIterator fetch(K key, long timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetch(K from, K to, long
> timeFrom, long timeTo);
> > > > KeyValueIterator, V> fetchAll(long timeFrom, long
> timeTo);
> > > > }
> > > >
> > > > WindowStore {
> > > >   //New methods.
> > > > WindowStoreIterator fetch(K key, Instant from, Duration
> duration) throws IllegalArgumentException;
> > > > KeyValueIterator, V> fetch(K from, K to, Instant
> from, Duration duration) throws IllegalArgumentException;
> > > > KeyValueIterator, V> fetchAll(Instant from, Duration
> duration) throws IllegalArgumentException;
> > > > }
> > > >
> > > > [1]
> https://lists.apache.org/thread.html/e976352e7e42d459091ee66ac790b6a0de7064eac0c57760d50c983b@%3Cdev.kafka.apache.org%3E
> > > >
> > > > В Ср, 12/09/2018 в 15:46 -0700, Matthias J. Sax пишет:
> > > > > Great!
> > > > >
> > > > > I did not double check the ReadOnlySessionStore interface before,
> and
> > > > > just assumed it would take a timestamp, too. My bad.
> > > > >
> > > > > Please update the KIP for ReadOnlyWindowStore and WindowStore.
> > > > >
> > > > > I like the KIP as-is. Feel free to start a VOTE thread. Even if
> there
> > > > > might be minor follow up comments, we can vote in parallel.
> > > > >
> > > > >
> > > > > -Matthias
> > > > >
> > > > > On 9/12/18 1:06 PM, John Roesler wrote:
> > > > > > Hi Nikolay,
> > > > > >
> > > > > > Yes, the changes we discussed for ReadOnlyXxxStore and XxxStore
> should be
> > > > > > in this KIP.
> > > > > >
> > > > > > And you're right, it seems like ReadOnlySessionStore is not
> necessary to
> > > > > > touch, since it doesn't reference any `long` timestamps.
> > > > > >
> > > > > > Thanks,
> > > > > > -John
> > > > > >
> > > > > > On Wed, Sep 12, 2018 at 4:36 AM Nikolay Izhikov <
> nizhi...@apache.org> wrote:
> > > > > >
> > > > > > > Hello, Matthias.
> > > > > > >
> > > > > > > > His proposal is, to deprecate existing methods on
> `ReadOnlyWindowStore`>
> > > > > > >
> > > > > > > and `ReadOnlySessionStore` and add them to `WindowStore` and>
> `SessionStore`
> > > > > > > > Does this make sense?
> > > > > > >
> > > > > > > You both are experienced Kafka developers, so yes, it does
> make a sense to
> > > > > > > me :).
> > > > > > > Do we want to make this change in KIP-358 or it required
> another KIP?
> > > > > > >
> > > > > > > > Btw: the KIP misses `ReadOnlySessionStore` atm.
> > > > > > >
> > > > > > > Sorry, but I don't understand you.
> > > > > > > As far as I can see, there is only 2 methods in
> `ReadOnlySessionStore`.
> > > > > > > Which method should be migrated to Duration?
> > > > > > >
> > > > > > >
> > > > >

Re: [VOTE] KIP-372: Naming Repartition Topics for Joins and Grouping

[Damian Guy]

+1 (binding)

On Tue, 18 Sep 2018 at 16:33 Bill Bejeck  wrote:

> All,
>
> In starting work on the PR for KIP-372, the Grouped interface needed some
> method renaming to be more consistent with the other configuration classes
> (Joined, Produced, etc.).  As such I've updated the Grouped code section of
> the KIP.
>
> As these changes address a comment from Matthias on the initial draft of
> the KIP and don't change any of the existing behavior already outlined,  I
> don't think a re-vote is required.
>
> Thanks,
> Bill
>
> On Tue, Sep 18, 2018 at 10:09 AM John Roesler  wrote:
>
> > +1 (non-binding)
> >
> > Thanks!
> >
> > On Mon, Sep 17, 2018 at 7:29 PM Dongjin Lee  wrote:
> >
> > > Great improvements. +1. (Non-binding)
> > >
> > > On Tue, Sep 18, 2018 at 5:14 AM Matthias J. Sax  >
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > -Matthias
> > > >
> > > > On 9/17/18 1:12 PM, Guozhang Wang wrote:
> > > > > +1 from me, thanks Bill !
> > > > >
> > > > > On Mon, Sep 17, 2018 at 12:43 PM, Bill Bejeck 
> > > wrote:
> > > > >
> > > > >> All,
> > > > >>
> > > > >> I'd like to start the voting process for KIP-372.  Here's the link
> > to
> > > > the
> > > > >> updated proposal
> > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > >> 372%3A+Naming+Repartition+Topics+for+Joins+and+Grouping
> > > > >>
> > > > >> I'll start with my own +1.
> > > > >>
> > > > >> Thanks,
> > > > >> Bill
> > > > >>
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > *Dongjin Lee*
> > >
> > > *A hitchhiker in the mathematical world.*
> > >
> > > *github:  github.com/dongjinleekr
> > > linkedin:
> > kr.linkedin.com/in/dongjinleekr
> > > slideshare:
> > > www.slideshare.net/dongjinleekr
> > > *
> > >
> >
>

Re: [VOTE] KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses

[Damian Guy]

Thanks  +1 (binding)

On Sun, 23 Sep 2018 at 23:43 Edoardo Comar  wrote:

> bumping the thread as the KIP needs 2 more binding votes ... pretty please
> ...
> --
>
> Edoardo Comar
>
> IBM Event Streams
> IBM UK Ltd, Hursley Park, SO21 2JN
>
>
>
>
> From:   "Skrzypek, Jonathan" 
> To: "dev@kafka.apache.org" 
> Date:   20/09/2018 17:08
> Subject:RE: [VOTE] KIP-302 - Enable Kafka clients to use all DNS
> resolved IP addresses
>
>
>
> Ok thanks.
>
> +1 (non-binding)
>
> The only thing I'm not too sure about is the naming around configuration
> entries for this, both for KIP-235 and KIP-302.
>
> KIP-235 expands DNS A records for bootstrap :
> resolve.canonical.bootstrap.servers.only
> KIP-302 expands DNS A records for advertised.listeners : use.all.dns.ips
>
> I'm a bit concerned that those don't easily explain what this does.
> Documentation helps obviously, but would we have suggestions for better
> naming ?
> I'm fine if we go for those but worth thinking about I think.
>
> Also, we probably want a third option to have both ? That's why we
> initially put in ".only" for KIP-235's parameter.
>
> Jonathan Skrzypek
>
>
> -Original Message-
> From: Edoardo Comar [mailto:eco...@uk.ibm.com]
> Sent: 20 September 2018 09:55
> To: dev@kafka.apache.org
> Subject: RE: [VOTE] KIP-302 - Enable Kafka clients to use all DNS resolved
> IP addresses
>
> Hi Jonathan
> we'll update the PR for KIP-302 soon. We do not need KIP-235 actually,
> they only share the name of the configuration entry.
>
> thanks
> Edo
>
> PS - we need votes :-)
>
> --
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
>
>
>
> From:   "Skrzypek, Jonathan" 
> To: "dev@kafka.apache.org" 
> Date:   19/09/2018 16:12
> Subject:***UNCHECKED*** RE: [VOTE] KIP-302 - Enable Kafka clients
> to use all  DNS resolved IP addresses
>
>
>
> I'm assuming this needs KIP-235 to be merged.
> Unfortunately I've tripped over some merge issues with git and struggled
> to fix.
> Hopefully this is fixed but any help appreciated :
> https://github.com/apache/kafka/pull/4485
>
>
>
> Jonathan Skrzypek
>
>
>
> -Original Message-
> From: Eno Thereska [mailto:eno.there...@gmail.com]
> Sent: 19 September 2018 11:01
> To: dev@kafka.apache.org
> Subject: Re: [VOTE] KIP-302 - Enable Kafka clients to use all DNS resolved
> IP addresses
>
> +1 (non-binding).
>
> Thanks
> Eno
>
> On Wed, Sep 19, 2018 at 10:09 AM, Rajini Sivaram 
> wrote:
>
> > Hi Edo,
> >
> > Thanks for the KIP!
> >
> > +1 (binding)
> >
> > On Tue, Sep 18, 2018 at 3:51 PM, Edoardo Comar 
> wrote:
> >
> > > Hi All,
> > >
> > > I'd like to start the vote on KIP-302:
> > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>
>
> > > 302+-+Enable+Kafka+clients+to+use+all+DNS+resolved+IP+addresses
> > >
> > > We'd love to get this in 2.1.0
> > > Kip freeze is just a few days away ... please cast your votes  :-):-)
> > >
> > > Thanks!!
> > > Edo
> > >
> > > --
> > >
> > > Edoardo Comar
> > >
> > > IBM Message Hub
> > >
> > > IBM UK Ltd, Hursley Park, SO21 2JN
> > > Unless stated otherwise above:
> > > IBM United Kingdom Limited - Registered in England and Wales with
> number
> > > 741598.
> > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> > 3AU
> > >
> >
>
> 
>
> Your Personal Data: We may collect and process information about you that
> may be subject to data protection laws. For more information about how we
> use and disclose your personal data, how we protect your information, our
> legal basis to use your information, your rights and who you can contact,
> please refer to:
> http://www.gs.com/privacy-notices
> <
> http://www.gs.com/privacy-notices
>
> >
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
> 
>
> Your Personal Data: We may collect and process information about you that
> may be subject to data protection laws. For more information about how we
> use and disclose your personal data, how we protect your information, our
> legal basis to use your information, your rights and who you can contact,
> please refer to: www.gs.com/privacy-notices<
> http://www.gs.com/privacy-notices
> >
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

[Damian Guy]

+1

On Thu, 21 Sep 2017 at 13:46 Guozhang Wang  wrote:

> +1 for me as well for collapsing.
>
> Jeyhun, could you update the wiki accordingly to show what's the final
> updates post KIP-182 that needs to be done in KIP-159 including KIP-149?
> The child page I made is just a suggestion, but you would still need to
> update your proposal for people to comment and vote on.
>
>
> Guozhang
>
>
> On Thu, Sep 14, 2017 at 10:37 PM, Ted Yu  wrote:
>
> > +1
> >
> > One interface is cleaner.
> >
> > On Thu, Sep 14, 2017 at 7:26 AM, Bill Bejeck  wrote:
> >
> > > +1 for me on collapsing the Rich and ValueWithKey interfaces
> > into 1
> > > interface.
> > >
> > > Thanks,
> > > Bill
> > >
> > > On Wed, Sep 13, 2017 at 11:31 AM, Jeyhun Karimov  >
> > > wrote:
> > >
> > > > Hi Damian,
> > > >
> > > > Thanks for your feedback. Actually, this (what you propose) was the
> > first
> > > > idea of KIP-149. Then we decided to divide it into two KIPs. I also
> > > > expressed my opinion that keeping the two interfaces (Rich and
> withKey)
> > > > separate would add more overloads. So, email discussion resulted that
> > > this
> > > > would not be a problem.
> > > >
> > > > Our initial idea was similar to :
> > > >
> > > > public abstract class RichValueMapper  implements
> > > > ValueMapperWithKey, RichFunction {
> > > > ..
> > > > }
> > > >
> > > >
> > > > So, we check the type of object, whether it is RichXXX or XXXWithKey
> > > inside
> > > > the called method and continue accordingly.
> > > >
> > > > If this is ok with the community, I would like to revert the current
> > > design
> > > > to this again.
> > > >
> > > > Cheers,
> > > > Jeyhun
> > > >
> > > > On Wed, Sep 13, 2017 at 3:02 PM Damian Guy 
> > wrote:
> > > >
> > > > > Hi Jeyhun,
> > > > >
> > > > > Thanks for sending out the update. I guess i was thinking more
> along
> > > the
> > > > > lines of option 2 where we collapse the Rich and
> ValueWithKey
> > > etc
> > > > > interfaces into 1 interface that has all of the arguments. I think
> we
> > > > then
> > > > > only need to add one additional overload for each operator?
> > > > >
> > > > > Thanks,
> > > > > Damian
> > > > >
> > > > > On Wed, 13 Sep 2017 at 10:59 Jeyhun Karimov 
> > > > wrote:
> > > > >
> > > > > > Dear all,
> > > > > >
> > > > > > I would like to resume the discussion on KIP-159. I (and
> Guozhang)
> > > > think
> > > > > > that releasing KIP-149 and KIP-159 in the same release would make
> > > sense
> > > > > to
> > > > > > avoid a release with "partial" public APIs. There is a KIP [1]
> > > proposed
> > > > > by
> > > > > > Guozhang (and approved by me) to unify both KIPs.
> > > > > > Please feel free to comment on this.
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > https://cwiki.apache.org/confluence/pages/viewpage.
> > > > action?pageId=73637757
> > > > > >
> > > > > > Cheers,
> > > > > > Jeyhun
> > > > > >
> > > > > > On Fri, Jul 21, 2017 at 2:00 AM Jeyhun Karimov <
> > je.kari...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Matthias, Damian, all,
> > > > > > >
> > > > > > > Thanks for your comments and sorry for super-late update.
> > > > > > >
> > > > > > > Sure, the DSL refactoring is not blocking for this KIP.
> > > > > > > I made some changes to KIP document based on my prototype.
> > > > > > >
> > > > > > > Please feel free to comment.
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Jeyhun
> > > > > > >
> > > > > > > On Fri, Jul 7, 2017 at 9:35 PM Matthias J. Sax <
> > > > matth...@c

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

[Damian Guy]

Hi All,

There has been one further update to the KIP. WindowedKStream has been
renamed to TimeWindowedKStream.

Thanks,
Damian

On Tue, 19 Sep 2017 at 12:21 Damian Guy  wrote:

> All, a small addition to the KIP. During implementation we realized it
> would be good to add the below two methods to StreamBuilder
>
> public synchronized  GlobalKTable globalTable(final String
> topic, final Materialized> materialized)
> public synchronized  KTable table(final String topic, final
> Materialized> materialized)
>
> Thanks,
> Damian
>
> On Fri, 15 Sep 2017 at 12:37 Damian Guy  wrote:
>
>> Sounds good to me.
>>
>> On Thu, 14 Sep 2017 at 19:55 Guozhang Wang  wrote:
>>
>>> I'd suggest we remove both to and through together in KIP-182, since for
>>> operator "KTable#to" is as confusing as to "KTable#through" which
>>> overwhelms its benefit as a syntax sugar. I think the extra step
>>> "toStream"
>>> is actually better to remind the caller that it is sending its changelog
>>> stream to topic, plus it is not that much characters.
>>>
>>>
>>> Guozhang
>>>
>>> On Wed, Sep 13, 2017 at 12:40 AM, Damian Guy 
>>> wrote:
>>>
>>> > Hi Guozhang,
>>> >
>>> > I had an offline discussion with Matthias and Bill about it. It is
>>> thought
>>> > that `to` offers some benefit, i.e., syntactic sugar, so perhaps no
>>> harm in
>>> > keeping it. However, `through` less so, seeing as we can materialize
>>> stores
>>> > via `filter`, `map` etc, so one of the main benefits of `through` no
>>> longer
>>> > exists. WDYT?
>>> >
>>> > Thanks,
>>> > Damian
>>> >
>>> > On Tue, 12 Sep 2017 at 18:17 Guozhang Wang  wrote:
>>> >
>>> > > Hi Damian,
>>> > >
>>> > > Why we are deprecating KTable.through while keeping KTable.to?
>>> Should we
>>> > > either keep both of them or deprecate both of them in favor or
>>> > > KTable.toStream if people agree that it is confusing to users?
>>> > >
>>> > >
>>> > > Guozhang
>>> > >
>>> > >
>>> > > On Tue, Sep 12, 2017 at 1:18 AM, Damian Guy 
>>> > wrote:
>>> > >
>>> > > > Hi All,
>>> > > >
>>> > > > A minor update to the KIP, i needed to add KTable.to(Produced) for
>>> > > > consistency. KTable.through will be deprecated in favour of using
>>> > > > KTable.toStream().through()
>>> > > >
>>> > > > Thanks,
>>> > > > Damian
>>> > > >
>>> > > > On Thu, 7 Sep 2017 at 08:52 Damian Guy 
>>> wrote:
>>> > > >
>>> > > > > Thanks all. The vote is now closed and the KIP has been accepted
>>> > with:
>>> > > > > 2 non binding votes - bill and matthias
>>> > > > > 3 binding  - Damian, Guozhang, Sriram
>>> > > > >
>>> > > > > Regards,
>>> > > > > Damian
>>> > > > >
>>> > > > > On Tue, 5 Sep 2017 at 22:24 Sriram Subramanian >> >
>>> > > wrote:
>>> > > > >
>>> > > > >> +1
>>> > > > >>
>>> > > > >> On Tue, Sep 5, 2017 at 1:33 PM, Guozhang Wang <
>>> wangg...@gmail.com>
>>> > > > wrote:
>>> > > > >>
>>> > > > >> > +1
>>> > > > >> >
>>> > > > >> > On Fri, Sep 1, 2017 at 3:45 PM, Matthias J. Sax <
>>> > > > matth...@confluent.io>
>>> > > > >> > wrote:
>>> > > > >> >
>>> > > > >> > > +1
>>> > > > >> > >
>>> > > > >> > > On 9/1/17 2:53 PM, Bill Bejeck wrote:
>>> > > > >> > > > +1
>>> > > > >> > > >
>>> > > > >> > > > On Thu, Aug 31, 2017 at 10:20 AM, Damian Guy <
>>> > > > damian@gmail.com>
>>> > > > >> > > wrote:
>>> > > > >> > > >
>>> > > > >> > > >> Thanks everyone for voting! Unfortunately i've had

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

[Damian Guy]

Hi Jeyhun,

All KIP-182 API PRs have now been merged. So you can consider it as stable.
Thanks,
Damian

On Thu, 21 Sep 2017 at 15:23 Jeyhun Karimov  wrote:

> Hi all,
>
> Thanks a lot for your comments. For the single interface (RichXXX and
> XXXWithKey) solution, I have already submitted a PR but probably it is
> outdated (when the KIP first proposed), I need to revisit that one.
>
> @Guozhang, from our (offline) discussion, I understood that we may not make
> it merge this KIP into the upcoming release, as KIP-159 is not voted yet
> (because we want both KIP-149 and KIP-159 to be as an "atomic" merge).  So
> I decided to wait until KIP-182 gets stable (there are some minor updates
> AFAIK) and update the KIP accordingly. Please correct me if I am wrong or I
> misunderstood.
>
> Cheers,
> Jeyhun
>
>
> On Thu, Sep 21, 2017 at 4:11 PM Damian Guy  wrote:
>
> > +1
> >
> > On Thu, 21 Sep 2017 at 13:46 Guozhang Wang  wrote:
> >
> > > +1 for me as well for collapsing.
> > >
> > > Jeyhun, could you update the wiki accordingly to show what's the final
> > > updates post KIP-182 that needs to be done in KIP-159 including
> KIP-149?
> > > The child page I made is just a suggestion, but you would still need to
> > > update your proposal for people to comment and vote on.
> > >
> > >
> > > Guozhang
> > >
> > >
> > > On Thu, Sep 14, 2017 at 10:37 PM, Ted Yu  wrote:
> > >
> > > > +1
> > > >
> > > > One interface is cleaner.
> > > >
> > > > On Thu, Sep 14, 2017 at 7:26 AM, Bill Bejeck 
> > wrote:
> > > >
> > > > > +1 for me on collapsing the Rich and ValueWithKey
> interfaces
> > > > into 1
> > > > > interface.
> > > > >
> > > > > Thanks,
> > > > > Bill
> > > > >
> > > > > On Wed, Sep 13, 2017 at 11:31 AM, Jeyhun Karimov <
> > je.kari...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Damian,
> > > > > >
> > > > > > Thanks for your feedback. Actually, this (what you propose) was
> the
> > > > first
> > > > > > idea of KIP-149. Then we decided to divide it into two KIPs. I
> also
> > > > > > expressed my opinion that keeping the two interfaces (Rich and
> > > withKey)
> > > > > > separate would add more overloads. So, email discussion resulted
> > that
> > > > > this
> > > > > > would not be a problem.
> > > > > >
> > > > > > Our initial idea was similar to :
> > > > > >
> > > > > > public abstract class RichValueMapper  implements
> > > > > > ValueMapperWithKey, RichFunction {
> > > > > > ..
> > > > > > }
> > > > > >
> > > > > >
> > > > > > So, we check the type of object, whether it is RichXXX or
> > XXXWithKey
> > > > > inside
> > > > > > the called method and continue accordingly.
> > > > > >
> > > > > > If this is ok with the community, I would like to revert the
> > current
> > > > > design
> > > > > > to this again.
> > > > > >
> > > > > > Cheers,
> > > > > > Jeyhun
> > > > > >
> > > > > > On Wed, Sep 13, 2017 at 3:02 PM Damian Guy  >
> > > > wrote:
> > > > > >
> > > > > > > Hi Jeyhun,
> > > > > > >
> > > > > > > Thanks for sending out the update. I guess i was thinking more
> > > along
> > > > > the
> > > > > > > lines of option 2 where we collapse the Rich and
> > > ValueWithKey
> > > > > etc
> > > > > > > interfaces into 1 interface that has all of the arguments. I
> > think
> > > we
> > > > > > then
> > > > > > > only need to add one additional overload for each operator?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Damian
> > > > > > >
> > > > > > > On Wed, 13 Sep 2017 at 10:59 Jeyhun Karimov <
> > je.kari...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > &

Re: KIP Access

[Damian Guy]

Hi Matt,

You should have permission now.
Thanks,
Damian

On Wed, 18 Oct 2017 at 15:54 Matt Farmer  wrote:

> Hey everyone,
>
> I'm a software engineer at MailChimp working on our Data Systems team. I'm
> looking to file a KIP to improve the error handling hooks that Kafka
> Streams exposes when producing messages. We've got a use case internally
> that requires us to be able to ignore certain classes of errors (in this
> case RecordTooLargeException), log some details about the message, and
> carry on processing other messages. We developed some changes to allow this
> internally, and would like to kick off the process of contributing back a
> similar change upstream. (I kind of expect what we contribute back to have
> a bit of a different shape than what we built internally.)
>
> It looks like filing a KIP is the correct next step here, but it looks like
> I need some additional permissions on Confluence. What's the process for
> getting those permissions? My Confluence username is farmdawgnation, if
> that helps.
>
> Thanks,
> Matt
>

Re: Authorize to create a KIP page

[Damian Guy]

Hi Jan,

You should have permissions now.

Thanks,
Damian


On Wed, 25 Oct 2017 at 09:41 Jan Filipiak  wrote:

> Hello,
>
> to get KAFKA-3705 moving a little bit more I am thinking of starting a
> KIP to discuss the finaly API design.
>
> My confluence id is: "jfilipiak"
>
> Best Jan
>

Re: [VOTE] KIP-205: Add all() and range() API to ReadOnlyWindowStore

[Damian Guy]

+1

On Tue, 24 Oct 2017 at 16:46 Guozhang Wang  wrote:

> +1. Thanks.
>
> On Mon, Oct 23, 2017 at 8:11 PM, Richard Yu 
> wrote:
>
> > Hi all,
> >
> > I want to propose KIP-205 for the addition of new API. It is about adding
> > methods similar to those found in ReadOnlyKeyValueStore to the
> > ReadOnlyWindowStore class. As it appears the discussion has reached a
> > conclusion, I would like to start the voting process.
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 205%3A+Add+all%28%29+and+range%28%29+API+to+ReadOnlyWindowStore
> >
> > Thanks for your patience!
> >
>
>
>
> --
> -- Guozhang
>

Re: [VOTE] KIP-171 - Extend Consumer Group Reset Offset for Stream Application

[Damian Guy]

Thanks for the KIP - +1 (binding)

On Mon, 23 Oct 2017 at 18:39 Guozhang Wang  wrote:

> Thanks Jorge for driving this KIP! +1 (binding).
>
>
> Guozhang
>
> On Mon, Oct 16, 2017 at 2:11 PM, Bill Bejeck  wrote:
>
> > +1
> >
> > Thanks,
> > Bill
> >
> > On Fri, Oct 13, 2017 at 6:36 PM, Ted Yu  wrote:
> >
> > > +1
> > >
> > > On Fri, Oct 13, 2017 at 3:32 PM, Matthias J. Sax <
> matth...@confluent.io>
> > > wrote:
> > >
> > > > +1
> > > >
> > > >
> > > >
> > > > On 9/11/17 3:04 PM, Jorge Esteban Quilcate Otoya wrote:
> > > > > Hi All,
> > > > >
> > > > > It seems that there is no further concern with the KIP-171.
> > > > > At this point we would like to start the voting process.
> > > > >
> > > > > The KIP can be found here:
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 171+-+Extend+Consumer+Group+Reset+Offset+for+Stream+Application
> > > > >
> > > > >
> > > > > Thanks!
> > > > >
> > > >
> > > >
> > >
> >
>
>
>
> --
> -- Guozhang
>

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

[Damian Guy]

> > >>>>>>>>>>>
> > >>>>>>>>>>> 6. I'm not clear why we want to move `commit()` from
> > >>>>> ProcessorContext
> > >>>>>>>>> to
> > >>>>>>>>>>> RecordContext?
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> -
> > >>>>>>>>>> Because it makes sense logically and  to reduce code
> maintenance
> > >>>>> (both
> > >>>>>>>>>> interfaces have offset() timestamp() topic() partition()
> > >>>> methods),  I
> > >>>>>>>>>> inherit ProcessorContext from RecordContext.
> > >>>>>>>>>> Since we need commit() method both in ProcessorContext and in
> > >>>>>>>>> RecordContext
> > >>>>>>>>>> I move commit() method to parent class (RecordContext).
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Cheers,
> > >>>>>>>>>> Jeyhun
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Wed, Oct 11, 2017 at 12:59 AM, Guozhang Wang <
> > >>>> wangg...@gmail.com>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Jeyhun,
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks for the updated KIP, here are my comments.
> > >>>>>>>>>>>
> > >>>>>>>>>>> 0. RichInitializer definition seems missing.
> > >>>>>>>>>>>
> > >>>>>>>>>>> 1. I'd suggest moving the key parameter in the RichValueXX
> and
> > >>>>>>>>>> RichReducer
> > >>>>>>>>>>> after the value parameters, as well as in the templates; e.g.
> > >>>>>>>>>>>
> > >>>>>>>>>>> public interface RichValueJoiner {
> > >>>>>>>>>>> VR apply(final V1 value1, final V2 value2, final K key,
> > final
> > >>>>>>>>>>> RecordContext
> > >>>>>>>>>>> recordContext);
> > >>>>>>>>>>> }
> > >>>>>>>>>>>
> > >>>>>>>>>>> My motivation is that for lambda expression in J8, users that
> > >>>> would
> > >>>>>>> not
> > >>>>>>>>>>> care about the key but only the context, or vice versa, is
> > likely
> > >>>> to
> > >>>>>>>>>> write
> > >>>>>>>>>>> it as (value1, value2, dummy, context) -> ... than putting
> the
> > >>>> dummy
> > >>>>>>> at
> > >>>>>>>>>> the
> > >>>>>>>>>>> beginning of the parameter list. Generally speaking we'd like
> > to
> > >>>>> make
> > >>>>>>>>> all
> > >>>>>>>>>>> the "necessary" parameters prior to optional ones.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> 2. Some of the listed functions are not necessary since their
> > >>>>> pairing
> > >>>>>>>>>> APIs
> > >>>>>>>>>>> are being deprecated in 1.0 already:
> > >>>>>>>>>>>
> > >>>>>>>>>>>  KGroupedStream groupBy(final RichKeyValueMapper > >>>> super
> > >>>>> K,
> > >>>>>>>>> ?
> > >>>>>>>>>>> super V, KR> selector,
> > >>>>>>>>>>>final Serde keySerde,
> > >>>>>>>>

Re: Re: [DISCUSS] KIP-213 Support non-key joining in KTable

[Damian Guy]

Hi Jan, Thanks for the KIP!

In both alternatives the API will need to use the `Joined` class rather
than than passing in `Serde`s. Also, as with all other joins etc, there
probably should be an overload that doesn't require any `Serdes`.

It isn't clear to me what `joinPrefixFaker` is doing? In the comment it
says "returning an outputKey that when serialized only produces a prefix of
the output key which is the same serializing K" So why not just use "K" ?

Thanks,
Damian


On Fri, 27 Oct 2017 at 10:27 Ted Yu  wrote:

> I think if you explain what A and B are in the beginning, it makes sense to
> use them since readers would know who they reference.
>
> Cheers
>
> On Thu, Oct 26, 2017 at 11:04 PM, Jan Filipiak 
> wrote:
>
> >
> >
> > Thanks for the remarks. hope I didn't miss any.
> > Not even sure if it makes sense to introduce A and B or just stick with
> > "this ktable", "other ktable"
> >
> > Thank you
> > Jan
> >
> >
> > On 27.10.2017 06:58, Ted Yu wrote:
> >
> >> Do you mind addressing my previous comments ?
> >>
> >> http://search-hadoop.com/m/Kafka/uyzND1hzF8SRzUqb?subj=Re+
> >> DISCUSS+KIP+213+Support+non+key+joining+in+KTable
> >>
> >> On Thu, Oct 26, 2017 at 9:38 PM, Jan Filipiak  >
> >> wrote:
> >>
> >> Hello everyone,
> >>>
> >>> this is the new discussion thread after the ID-clash.
> >>>
> >>> Best
> >>> Jan
> >>>
> >>> __
> >>>
> >>>
> >>> Hello Kafka-users,
> >>>
> >>> I want to continue with the development of KAFKA-3705, which allows the
> >>> Streams DSL to perform KTableKTable-Joins when the KTables have a
> >>> one-to-many relationship.
> >>> To make sure we cover the requirements of as many users as possible and
> >>> have a good solution afterwards I invite everyone to read through the
> >>> KIP I
> >>> put together and discuss it here in this Thread.
> >>>
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-213+
> >>> Support+non-key+joining+in+KTable
> >>> https://issues.apache.org/jira/browse/KAFKA-3705
> >>> https://github.com/apache/kafka/pull/3720
> >>>
> >>> I think a public discussion and vote on a solution is exactly what is
> >>> needed to bring this feauture into kafka-streams. I am looking forward
> to
> >>> everyones opinion!
> >>>
> >>> Please keep the discussion on the mailing list rather than commenting
> on
> >>> the wiki (wiki discussions get unwieldy fast).
> >>>
> >>> Best
> >>> Jan
> >>>
> >>>
> >>>
> >>>
> >
> >
> >
>

Re: wiki access for KIP

[Damian Guy]

Hi Steven,

You should have permission now.

Thanks,
Damian

On Wed, 1 Nov 2017 at 06:43 Steven Aerts  wrote:

> I hereby would like to request write access to the wiki to create a
> KIP for KAFKA-6018.
>
> My wiki id is steven.aerts.
>
> Thanks,
>
>
>Steven
>

Re: [ANNOUNCE] Apache Kafka 1.0.0 Released

[Damian Guy]

ing and running reusable producers or
> consumers
> that connect Kafka topics to existing applications or data systems. For
> example, a connector to a relational database might capture every change to
> a table.three key capabilities:
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
> ** Building real-time streaming data pipelines that reliably get data
> between
> systems or applications.
>
> ** Building real-time streaming applications that transform or react
> to the streams
> of data.
>
>
> Apache Kafka is in use at large and small companies worldwide, including
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
> A big thank you for the following 108 contributors to this release!
>
> Abhishek Mendhekar, Xi Hu, Andras Beni, Andrey Dyachkov, Andy Chambers,
> Apurva Mehta, Armin Braun, Attila Kreiner, Balint Molnar, Bart De Vylder,
> Ben Stopford, Bharat Viswanadham, Bill Bejeck, Boyang Chen, Bryan Baugher,
> Colin P. Mccabe, Koen De Groote, Dale Peakall, Damian Guy, Dana Powers,
> Dejan Stojadinović, Derrick Or, Dong Lin, Zhendong Liu, Dustin Cote,
> Edoardo Comar, Eno Thereska, Erik Kringen, Erkan Unal, Evgeny Veretennikov,
> Ewen Cheslack-Postava, Florian Hussonnois, Janek P, Gregor Uhlenheuer,
> Guozhang Wang, Gwen Shapira, Hamidreza Afzali, Hao Chen, Jiefang He, Holden
> Karau, Hooman Broujerdi, Hugo Louro, Ismael Juma, Jacek Laskowski, Jakub
> Scholz, James Cheng, James Chien, Jan Burkhardt, Jason Gustafson, Jeff
> Chao, Jeff Klukas, Jeff Widman, Jeremy Custenborder, Jeyhun Karimov,
> Jiangjie Qin, Joel Dice, Joel Hamill, Jorge Quilcate Otoya, Kamal C, Kelvin
> Rutt, Kevin Lu, Kevin Sweeney, Konstantine Karantasis, Perry Lee, Magnus
> Edenhill, Manikumar Reddy, Manikumar Reddy O, Manjula Kumar, Mariam John,
> Mario Molina, Matthias J. Sax, Max Zheng, Michael Andre Pearce, Michael
> André Pearce, Michael G. Noll, Michal Borowiecki, Mickael Maison, Nick
> Pillitteri, Oleg Prozorov, Onur Karaman, Paolo Patierno, Pranav Maniar,
> Qihuang Zheng, Radai Rosenblatt, Alex Radzish, Rajini Sivaram, Randall
> Hauch, Richard Yu, Robin Moffatt, Sean McCauliff, Sebastian Gavril, Siva
> Santhalingam, Soenke Liebau, Stephane Maarek, Stephane Roset, Ted Yu,
> Thibaud Chardonnens, Tom Bentley, Tommy Becker, Umesh Chaudhary, Vahid
> Hashemian, Vladimír Kleštinec, Xavier Léauté, Xianyang Liu, Xin Li, Linhua
> Xin
>
>
> We welcome your help and feedback. For more information on how to report
> problems, and to get involved, visit the project website at
> http://kafka.apache.org/
>
>
>
>
> Thanks,
> Guozhang Wang
>