Re: [VOTE] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

2017-11-08 Thread Damian Guy
+1 (binding) On Sat, 4 Nov 2017 at 16:50 Matthias J. Sax wrote: > Yes. A KIP needs 3 binding "+1" to be accepted. > > You can still work on the PR and get it ready to get merged -- I am > quite confident that this KIP will be accepted :) > > > -Matthias > > On 11/4/17 3:56 PM, Matt Farmer wrote:

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

2017-11-09 Thread Damian Guy
Thanks Matthias, LGTM On Thu, 9 Nov 2017 at 11:13 Matthias J. Sax wrote: > Hi, > > I want to propose a new KIP to make Streams API more resilient to broker > disconnections. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-224%3A+Add+configuration+parameters+%60retries%60+and+%60retry

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

2017-11-09 Thread Damian Guy
ll > > > > On Thu, Nov 9, 2017 at 8:40 AM, Ted Yu wrote: > > > > > lgtm > > > > > > bq. pass both parameter > > > > > > parameter should be in plural. > > > Same with 'two new configuration parameter' > > > > >

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

2017-11-10 Thread Damian Guy
+1 (binding) - thanks On Tue, 7 Nov 2017 at 15:14 Paolo Patierno wrote: > Because Guozhang and Colin are doing a great job on reviewing the related > PR and we are really close to have it in a decent/final shape, what do the > other committers think about this KIP ? > > We are stuck at 2 binding

Re: [VOTE] KIP-224: Add configuration parameters `retries` to Streams API

2017-11-14 Thread Damian Guy
+1 On Tue, 14 Nov 2017 at 02:40 Bill Bejeck wrote: > Thanks for the KIP, +1 > > -Bill > > On Mon, Nov 13, 2017 at 7:25 PM, Ted Yu wrote: > > > +1 > > > > On Mon, Nov 13, 2017 at 4:20 PM, Matthias J. Sax > > wrote: > > > > > Hi @all, > > > > > > I would like to start the vote for KIP-224: > > >

Re: SessionKeySchema#segmentsToSearch()

2017-11-20 Thread Damian Guy
w.r.t `SessionKeySchema` MAX_VALUE is correct. This is because we use the end time (rather than the start time) of the session to define which segment the session is in. So it could be in any segment starting from `from` time. On Sun, 19 Nov 2017 at 20:27 Ted Yu wrote: > For `getMinSegmentGreate

Re: [VOTE] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

2017-11-20 Thread Damian Guy
+1 On Mon, 20 Nov 2017 at 17:52 Gwen Shapira wrote: > +1 > > Make sense. We have a supplier for every other client type :) > > On Fri, Nov 17, 2017 at 1:33 PM Matthias J. Sax > wrote: > > > +1 > > > > On 11/17/17 9:35 AM, Ted Yu wrote: > > > +1 > > > > > > On Fri, Nov 17, 2017 at 9:34 AM, Bill

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

2017-11-22 Thread Damian Guy
. > >>> > >>> > >>> Guozhang > >>> > >> > >> > >> On 11/6/17 1:41 PM, Jeyhun Karimov wrote: > >> > >>> Hi Matthias, > >>> > >>> Thanks a lot for correcting. It is a leftover from th

Re: [DISCUSS] Allowing write access to GitHub repositories

2017-12-12 Thread Damian Guy
+1 On Tue, 12 Dec 2017 at 10:20 Rajini Sivaram wrote: > Hi Ismael, > > Thank you for looking into this. Sounds like a good improvement. > > Regards, > > Rajini > > On Mon, Dec 11, 2017 at 6:35 AM, Guozhang Wang wrote: > > > Sounds good to me. > > > > On Sun, Dec 10, 2017 at 10:14 PM, Ismael Jum

Re: [VOTE] Allowing write access to GitHub repositories (aka GitBox)

2017-12-12 Thread Damian Guy
+1 On Tue, 12 Dec 2017 at 15:47 Ismael Juma wrote: > Hi all, > > The Apache Infra team has started a new project earlier this year called > GitBox that supports two-way synchronization between GitHub and > git-wip-us.apache.org and, most importantly, provides GitHub write access > to committers.

Re: [VOTE] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

2017-12-19 Thread Damian Guy
ang > > > > > > > > > > > > > > > On Mon, Nov 20, 2017 at 11:12 AM, Guozhang Wang > > wrote: > > > > > >> +1 from myself as well. > > >> > > >> I'm closing this KIP as accepted with 3 binding votes

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

2017-12-21 Thread Damian Guy
+1 On Wed, 20 Dec 2017 at 21:09 Ted Yu wrote: > Ping for more (binding) votes. > > The pull request is ready. > > On Fri, Dec 15, 2017 at 12:57 PM, Guozhang Wang > wrote: > > > +1 (binding), thanks! > > > > On Fri, Dec 15, 2017 at 11:56 AM, Ted Yu wrote: > > > > > Hi, > > > Here is the discuss

Re: [VOTE] KIP-420: Add Single Value Fetch in Session Stores

2019-01-24 Thread Damian Guy
+1 On Wed, 23 Jan 2019 at 23:23, Guozhang Wang wrote: > Hello Matthias, > > Cool. I'd add it to the wiki page as well. > > > Guozhang > > On Sat, Jan 19, 2019 at 10:59 AM Matthias J. Sax > wrote: > > > Thanks for the KIP Guozhang! > > > > Would it make sense to add a default implementation for

Re: [ANNOUNCE] New Committer: Bill Bejeck

2019-02-13 Thread Damian Guy
Congratulations Bill! On Wed, 13 Feb 2019 at 16:51, Satish Duggana wrote: > Congratulations Bill! > > On Thu, Feb 14, 2019 at 6:41 AM Marcelo Barbosa > wrote: > > > > Wow! Congrats Bill! > > Cheers, > > Barbosa > > Em quarta-feira, 13 de fevereiro de 2019 23:03:54 BRST, Guozhang > Wang esc

Re: [VOTE] KIP-422: Use the default value of max.poll.interval in Streams

2019-03-29 Thread Damian Guy
+1 On Wed, 27 Mar 2019 at 21:38, John Roesler wrote: > Ah, good point, Guozhang. I'll remove that mention from the KIP. > > On Wed, Mar 27, 2019 at 3:30 PM Bill Bejeck wrote: > > > +1 for me, > > > > Thanks, > > Bill > > > > On Wed, Mar 27, 2019 at 4:13 PM Guozhang Wang > wrote: > > > > > +1 f

Re: [VOTE] KIP-443: Return to default segment.ms and segment.index.bytes in Streams repartition topics

2019-03-29 Thread Damian Guy
+1 On Fri, 29 Mar 2019 at 01:59, John Roesler wrote: > +1 (nonbinding) from me. > > On Thu, Mar 28, 2019 at 7:08 PM Guozhang Wang wrote: > > > Hello folks, > > > > I'd like to directly start a voting thread on this simple KIP to change > the > > default override values for repartition topics: >

Re: Speeding up integration tests

2019-04-01 Thread Damian Guy
Just to throw my 2c in here. It would also be good if we could run the tests that failed "last time" first. That way at least if they are broken then we'd fail fast rather than waiting for ever. Not much sucks more than waiting for 2 hours to find out a test failed when it fails regularly and you c

Re: [ANNOUNCE] New committer: Damian Guy

2017-06-10 Thread Damian Guy
On Fri, Jun 9, 2017 at 1:34 PM Guozhang Wang wrote: > > > > > Hello all, > > > > > > > > > The PMC of Apache Kafka is pleased to announce that we have invited > > Damian > > > Guy as a committer to the project. > > > > > > Da

Re: Contributor

2017-06-21 Thread Damian Guy
Hi Andras, You should have access now. Thanks, Damian On Wed, 21 Jun 2017 at 10:45 Andras Beni wrote: > Hi All, > > I'd like to contribute to Apache Kafka. > Can you please add me (username: andrasbeni) to the contributors list for > this project at issues.apache.org? > > Thank you, > Andras >

[DISCUSS] Streams DSL/StateStore Refactoring

2017-06-21 Thread Damian Guy
Hi, I'd like to get a discussion going around some of the API choices we've made in the DLS. In particular those that relate to stateful operations (though this could expand). As it stands we lean heavily on overloaded methods in the API, i.e, there are 9 overloads for KGroupedStream.count(..)! It

Re: Contributor

2017-06-21 Thread Damian Guy
Done - thanks On Wed, 21 Jun 2017 at 12:19 Tom Bentley wrote: > Please can I also be added? My username is tombentley. > > Thanks > > Tom > > On 21 June 2017 at 12:03, Damian Guy wrote: > > > Hi Andras, > > > > You should have access now. > > &g

Re: confluence permission request

2017-06-21 Thread Damian Guy
Hi, That should be done. Thanks, Damian On Wed, 21 Jun 2017 at 05:42 Kenji Hayashida wrote: > To Kafka Dev Team, > > Sorry, forgot sending my ID. > My ID is kenjih. > > Thanks. > > - Kenji Hayashida > > 2017-06-21 13:29 GMT+09:00 Kenji Hayashida : > > > To Kafka Dev Team, > > > > Hi, could you

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-21 Thread Damian Guy
by quite a bit. > > Eno > > > On Jun 21, 2017, at 3:37 PM, Damian Guy wrote: > > > > Hi, > > > > I'd like to get a discussion going around some of the API choices we've > > made in the DLS. In particular those that relate to stateful operati

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-22 Thread Damian Guy
sn't necessarily knows what > it returns. Also, one needs to figure out the special method (`table()` in > this case) that gives you what you actually care about (`KTable` in this > case). Not major issues, but worth mentioning while doing the comparison. > > The builder approach

Re: Kafka streams KStream and ktable join issue

2017-06-23 Thread Damian Guy
My guess is it is because the record doesn't have a key, i.e., the key is null. We have a fix for this in 0.11, in that we will skip records with a null key during restore. On Fri, 23 Jun 2017 at 03:57 Matthias J. Sax wrote: > Hi, > > can you reproduce the error reliably? Are use using 0.10.2.0

Re: [VOTE] KIP-161: streams deserialization exception handlers

2017-06-23 Thread Damian Guy
Thanks for the KIP Eno. +1 (binding) On Fri, 23 Jun 2017 at 11:00 Eno Thereska wrote: > Starting voting thread for: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-161%3A+streams+deserialization+exception+handlers > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-161:+streams

Re: Mirroring documentation improvement

2017-06-23 Thread Damian Guy
Evgeniy, you should now have access. Tom & Paolo what are your wiki usernames? Thanks, Damian On Fri, 23 Jun 2017 at 14:52 Tom Bentley wrote: > If any of the committers see this: Could I also have edit access please? > > On 23 June 2017 at 14:47, Evgeniy Veretennikov > > wrote: > > > Hi Tom,

Re: Mirroring documentation improvement

2017-06-23 Thread Damian Guy
Sent: Friday, June 23, 2017 2:10 PM > To: dev@kafka.apache.org > Subject: Re: Mirroring documentation improvement > > Hi Damian, my username is tombentley > > Thanks > > Tom > > On 23 June 2017 at 14:57, Damian Guy wrote: > > > Evgeniy, you should now have access. &g

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-28 Thread Damian Guy
.withStoreName("name").mapValues(...) > > Each .withXXX(...) return the current KTable and all the .withXXX() are > just added to the KTable interface. Or do I miss anything why this wont' > work or any obvious disadvantage? > > > See above. > > -Matthias

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-29 Thread Damian Guy
all the withXX() in KTable interface > > -- but this might be subjective. > > > > > > However, I don't understand your argument about putting aggregate() > > after the withXX() -- all the calls to withXX() set optional parameters > > for aggregate() and not f

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-29 Thread Damian Guy
uot;: maybe better "withQueryableStateName"? > > 7. "withLogConfig": maybe better "withLoggingTopicConfig()"? > > > > Guozhang > > > > On Wed, Jun 28, 2017 at 3:59 PM, Matthias J. Sax > wrote: > > > I see your point about "wh

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-29 Thread Damian Guy
ped stream. Then the final count()/reduce()/aggregate() call has any of the params that are required/specific to that function. > > -Matthias > > On 6/28/17 2:55 AM, Damian Guy wrote: > >> I also think that mixing optional parameters with configs is a bad idea. > >> Have

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-29 Thread Damian Guy
of(10)).withJoinType(JoinType.LEFT).build()); } I'm not going to say which way i'm leaning, yet! Thanks, Damian On Thu, 29 Jun 2017 at 11:47 Damian Guy wrote: > >> However, I don't understand your argument about putting aggregate() >> after the withXX() -- all

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-29 Thread Damian Guy
I feel that I could > confidently say the user couldn’t really mess it up. > > // Count > > KTable count = > > kGroupedStream.count(Count.count().withQueryableStoreName("my-store")); > The kGroupedStream is reusable and if they attempted to reuse the Count > for some reason it

Re: [DISCUSS] KIP-171: Extend Consumer Group Reset Offset for Stream Application

2017-06-29 Thread Damian Guy
Hi, Thanks for the KIP. What is not clear is how is this going to handle state stores? Right now the streams reset tool, resets everything and clears up the state stores. What are we going to do if we reset to a particular offset? If we clear the state then we've lost any previously aggregated val

Re: [VOTE] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-06-30 Thread Damian Guy
> >> > >> Thanks. > >> > >> 2017-04-24 18:56 GMT+02:00 Guozhang Wang : > >> > >>> Florian, could you also add the part of deprecating > >> `KafkaStreams.toString` > >>> in your KIP as well? > >>> > >>&g

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-30 Thread Damian Guy
eam.groupBy().count() As i said above, everything that happens before the final aggregate call can be applied to any of them. So it makes sense to me to do those things ahead of the final aggregate call. > Last about builder pattern. I am convinced that we need some "terminal" > operato

[DISCUSS] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

2017-06-30 Thread Damian Guy
Hi, I've put together what will hopefully be a not too contentious KIP to enable the setting of default configs for streams internal topics via StreamsConfig. You can find the KIP here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-173%3A+Add+prefix+to+StreamsConfig+to+enable+setting+defa

Re: [DISCUSS] KIP-167: Add interface for the state store restoration process

2017-06-30 Thread Damian Guy
Thanks for the updated KIP Bill. In the PR you have AbstractBatchingRestoreCallback and AbstractNotifyingRestoreCallback which are both in public packages, so are part of the API. I think you need to add those to the KIP to round it off. Otherwise LGTM. Thanks, Damian On Fri, 30 Jun 2017 at 12:

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-06-30 Thread Damian Guy
; input -- not necessarily a operator/window question. > > Sure, within a sub-topolgy it makes sense for the window retention to be the same. However, different sub-toplogies may have different constraints. > > -Matthias > > On 6/30/17 12:31 AM, Damian Guy wrote: > > Than

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-07-04 Thread Damian Guy
I would argue to go a bit slower and more carefull on > this one. At some point we need to get it right. Peeking over to the hadoop > guys with their hughe userbase. Config files really work well for them. > > Best Jan > > > > > > On 30.06.2017 09:31, Damian Guy wrote: >

Re: RocksDB flushing issue on 0.10.2 streams

2017-07-04 Thread Damian Guy
Hi Greg, Obviously a bit difficult to read the RocksDBException, but my guess is it is because the state directory gets deleted right before the flush happens: 2017-07-04 10:54:46,829 [myid:] - INFO [StreamThread-21:StateDirectory@213] - Deleting obsolete state directory 0_10 for task 0_10 Yes i

Re: RocksDB flushing issue on 0.10.2 streams

2017-07-05 Thread Damian Guy
; >>> rebalance was happening? It looks to me like the standby task was just > >>> committing as part of normal operations. > >>> > >>> On Tue, Jul 4, 2017 at 7:40 AM, Damian Guy > wrote: > >>> > >>>> Hi Greg, > >&g

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

2017-07-05 Thread Damian Guy
HI Jeyhun, Is the intention that these methods are new overloads on the KStream, KTable, etc? It is worth noting that a ProcessorContext is not a RecordContext. A RecordContext, as it stands, only exists during the processing of a single record. Whereas the ProcessorContext exists for the lifetim

Re: RocksDB flushing issue on 0.10.2 streams

2017-07-05 Thread Damian Guy
BTW - i'm trying to reproduce it, but not having much luck so far... On Wed, 5 Jul 2017 at 09:27 Damian Guy wrote: > Thans for the updates Greg. There were some minor changes around this in > 0.11.0 to make it less likely to happen, but we've only ever seen the > locking fai

Re: RocksDB flushing issue on 0.10.2 streams

2017-07-05 Thread Damian Guy
aps during > a > > rebalance there is a race condition?) I'm guessing there's probably a > more > > benign explanation, but that is what it looks like right now. > > > > On Wed, Jul 5, 2017 at 7:00 AM, Damian Guy wrote: > > > >> BTW - i'm trying

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

2017-07-05 Thread Damian Guy
Congratulations Ismael! Very well deserved. Cheers, Damian On Wed, 5 Jul 2017 at 22:54, Dong Lin wrote: > Congratulations Ismael! > > On Wed, Jul 5, 2017 at 1:55 PM, Jun Rao wrote: > > > Hi, Everyone, > > > > Ismael Juma has been active in the Kafka community since he became > > a Kafka committe

Re: RocksDB flushing issue on 0.10.2 streams

2017-07-06 Thread Damian Guy
n Wed, 5 Jul 2017 at 23:23 Damian Guy wrote: > Thanks Greg. I'll look into it more tomorrow. Just finding it difficult to > reproduce in a test. > Thanks for providing the sequence, gives me something to try and repo. > Appreciated. > > Thanks, > Damian > On Wed, 5 Ju

Re: RocksDB flushing issue on 0.10.2 streams

2017-07-06 Thread Damian Guy
hu, 6 Jul 2017 at 12:43 Damian Guy wrote: > Greg, what OS are you running on? > Are you able to reproduce this in a test at all? > For instance, based on what you described it would seem that i should be > able to start a streams app, wait for it to be up and running, run the > state

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

2017-07-07 Thread Damian Guy
replacing the overrides. Thanks, Damian > About ProcessorContext vs RecordContext, you are right. I think I need to > implement a prototype to understand the full picture as some parts of the > KIP might not be as straightforward as I thought. > > > Cheers, > Jeyhun > > O

Re: Possible deadlock during shutdown in setState in kafka streams 10.2

2017-07-07 Thread Damian Guy
Hi Greg, Would you mind creating a JIRA for this with the thread dump ( i don't see it attached to your message). Thanks, Damian On Fri, 7 Jul 2017 at 10:36 Greg Fodor wrote: > I'm running a 10.2 job across 5 nodes with 32 stream threads on each node > and find that when gracefully shutdown al

Re: Possible deadlock during shutdown in setState in kafka streams 10.2

2017-07-08 Thread Damian Guy
Thanks On Fri, 7 Jul 2017 at 18:20, Greg Fodor wrote: > Sure thing: https://issues.apache.org/jira/browse/KAFKA-5571 > > On Fri, Jul 7, 2017 at 2:59 AM, Damian Guy wrote: > > > Hi Greg, > > > > Would you mind creating a JIRA for this with the thread dump ( i don&#

Re: [VOTE]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-07-08 Thread Damian Guy
+1 On Fri, 7 Jul 2017 at 16:08, Eno Thereska wrote: > +1 (non-binding) Thanks. > > Eno > > On 6 Jul 2017, at 21:49, Gwen Shapira wrote: > > > > +1 > > > > On Wed, Jul 5, 2017 at 9:25 AM Matthias J. Sax > > wrote: > > > >> +1 > >> > >> On 6/27/17 1:41 PM, Jeyhun Karimov wrote: > >>> Dear all, >

Re: [DISCUSS] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

2017-07-10 Thread Damian Guy
docs for > educating users. > > > Guozhang > > > On Fri, Jun 30, 2017 at 4:18 AM, Damian Guy wrote: > > > Hi, > > > > I've put together what will hopefully be a not too contentious KIP to > > enable the setting of default configs for streams int

Re: [ANNOUNCE] New Kafka PMC member Jason Gustafson

2017-07-12 Thread Damian Guy
Congratulations Jason On Wed, 12 Jul 2017 at 08:57 Sriram Subramanian wrote: > Congratulations Jason! > > On Wed, Jul 12, 2017 at 8:02 AM, Rajini Sivaram > wrote: > > > Congratulations, Jason! > > > > On Wed, Jul 12, 2017 at 3:53 PM, Ismael Juma wrote: > > > > > Congratulations Jason! > > > >

Re: [VOTE] KIP-167: Add interface for the state store restoration process

2017-07-15 Thread Damian Guy
+1 On Thu, 13 Jul 2017 at 07:13 Eno Thereska wrote: > +1 (non-binding). > > Thanks Bill. > > Eno > > On 12 Jul 2017, at 09:12, Bill Bejeck wrote: > > > > All, > > > > Now that we've concluded a second round of discussion on KIP-167, I'd > like > > to start a vote. > > > > > > Thanks, > > Bill >

Re: [DISCUSS] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

2017-07-15 Thread Damian Guy
For configs > for StreamsKafkaClient and later AdminClient. > > Can so picky back this on this KIP or should we do a new KIP? > > > -Matthias > > On 7/10/17 6:45 PM, Guozhang Wang wrote: > > Thanks Damian. LGTM. > > > > Guozhang > > > > On Mon

Re: [DISCUSS] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

2017-07-17 Thread Damian Guy
nClient does not require a KIP, we can save one > KIP. > > But I am fine either way if you want to exclude it. > > -Matthias > > On 7/15/17 1:59 PM, Damian Guy wrote: > > Thanks Matthias. I'll have a look into it. Though i guess we are planning > > on migrating t

[VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

2017-07-17 Thread Damian Guy
Hi, I'd like to kick off the vote for KIP-173: https://cwiki.apache.org/confluence/display/KAFKA/KIP-173%3A+Add+prefix+to+StreamsConfig+to+enable+setting+default+internal+topic+configs A PR for this can be found here: https://github.com/apache/kafka/pull/3459 Thanks, Damian

Re: [VOTE] KIP-162: Enable topic deletion by default

2017-07-19 Thread Damian Guy
+1 On Tue, 6 Jun 2017 at 18:19 BigData dev wrote: > +1 (non-binding) > > Thanks, > Bharat > > On Tue, Jun 6, 2017 at 9:21 AM, Ashwin Sinha > wrote: > > > +1 > > > > On Tue, Jun 6, 2017 at 11:20 PM, Mickael Maison < > mickael.mai...@gmail.com> > > wrote: > > > > > +1 (non binding), thanks > > >

Re: KIP-162: Enable topic deletion by default

2017-07-19 Thread Damian Guy
+1 On Wed, 31 May 2017 at 13:36 Jim Jagielski wrote: > +1 > > On May 27, 2017, at 9:27 PM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > > > > Sure, that sounds good. > > > > I suggested that to keep command line behavior consistent. > > Plus, removal of ACL access is something that

Re: [VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

2017-07-20 Thread Damian Guy
expose the TOPIC_PREFIX constant? > > 2. The vote thread ended up inside the discuss thread in Gmail. It may be > > worth sending another email to make it clear that the vote is ongoing. > You > > can link back to this thread so that the existing votes are still > counted. >

Re: [DISCUSS] 2017 October release planning and release version

2017-07-20 Thread Damian Guy
+1 on 1.0! Are we also going to move to java 8? I also think we should drop the Unstable annotations completely. Cheers, Damian On Wed, 19 Jul 2017 at 21:36 Guozhang Wang wrote: > Hi Stevo, > > Just trying to add to what Ismael has already replied you: > > > > Practice/"features" like protocol

Re: [VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

2017-07-21 Thread Damian Guy
s > algorithm decided to place it under the same thread. This is a recurring > problem and it affects discoverability of vote threads for Gmail users (I > assume there are many such users). > > Ismael > > On Thu, Jul 20, 2017 at 1:32 AM, Damian Guy wrote: > > > Thanks Ism

Re: [VOTE] KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs

2017-07-21 Thread Damian Guy
Sorry, i mentioned Gouzhang twice in the vote. Actual votes: 3 binding (Guozhang, Damian, Ismael) 2 non-binding (Eno, Matthias) Thanks, Damian On Fri, 21 Jul 2017 at 10:00 Damian Guy wrote: > Hi, > The Vote for this KIP is now closed. > KIP-173 has been accepted with > 3 bindi

Re: [VOTE]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

2017-07-21 Thread Damian Guy
Hi Jeyhun, Feel free to close the vote. It has been accepted. Thanks, Damian On Mon, 17 Jul 2017 at 06:18 Guozhang Wang wrote: > +1. Thanks! > > On Sat, Jul 8, 2017 at 1:35 AM, Damian Guy wrote: > > > +1 > > On Fri, 7 Jul 2017 at 16:08, Eno Thereska > wrote

Re: [VOTE] KIP-167 (Addendum): Add interface for the state store restoration process

2017-07-26 Thread Damian Guy
+1 On Tue, 25 Jul 2017 at 18:17 Sriram Subramanian wrote: > +1 > > On Fri, Jul 21, 2017 at 12:08 PM, Guozhang Wang > wrote: > > > +1 > > > > On Thu, Jul 20, 2017 at 11:00 PM, Matthias J. Sax > > > wrote: > > > > > +1 > > > > > > On 7/20/17 4:22 AM, Bill Bejeck wrote: > > > > Hi, > > > > > > >

[DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-07-27 Thread Damian Guy
Hi, I've put together a KIP to make some changes to the KafkaStreams DSL that will hopefully allow us to: 1) reduce the explosion of overloads 2) add new features without having to continue adding more overloads 3) provide simpler ways for people to use custom storage engines and wrap them with lo

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-07-27 Thread Damian Guy
Updated link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-182%3A+Reduce+Streams+DSL+overloads+and+allow+easier+use+of+custom+storage+engines Thanks, Damian On Thu, 27 Jul 2017 at 13:09 Damian Guy wrote: > Hi, > > I've put together a KIP to make some changes to the Ka

Re: Kafka Streams debugging with "no fluent" API choice

2017-08-01 Thread Damian Guy
I don't know specifically why this is removed, however if you want to get the same functionality you can use peek, i.e: stream.map(...).peek(...).filter(..) You can log the key values out in the peek call. On Tue, 1 Aug 2017 at 11:48 Paolo Patierno wrote: > Hi guys, > > > I was thinking about

Re: Kafka Streams debugging with "no fluent" API choice

2017-08-01 Thread Damian Guy
Patierno > Senior Software Engineer (IoT) @ Red Hat > Microsoft MVP on Windows Embedded & IoT > Microsoft Azure Advisor > > Twitter : @ppatierno<http://twitter.com/ppatierno> > Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno> > Blog : DevExperience

Re: Kafka Streams debugging with "no fluent" API choice

2017-08-01 Thread Damian Guy
e Advisor > > Twitter : @ppatierno<http://twitter.com/ppatierno> > Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno> > Blog : DevExperience<http://paolopatierno.wordpress.com/> > > > > From: Damian Guy > Sent: Tuesd

Re: KStreamPrintTest : no differences in the unit tests

2017-08-02 Thread Damian Guy
Yes - they are basically the same. Feel free to submit a patch to remove one of them On Wed, 2 Aug 2017 at 15:28 Paolo Patierno wrote: > Hi devs, > > taking a look at KStreamPrintTest I can't find any substantial difference > between the two tests : > > > testPrintStreamWithProvidedKeyValueMappe

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-09 Thread Damian Guy
eep its static > constructors with only two `as` as they are today.K > > 4. Is there any modifications on StateStoreSupplier? Is it replaced by > BytesStoreSupplier? Seems some more descriptions are lacking here. Also in > > No modifications to StateStoreSupplier. It is superseceded by Bytes

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-09 Thread Damian Guy
onstructors for each spec, > > with one constructor having only one parameter to make it more usable, > i.e. > > as a user I do not need to give all parameters if I only want to override > > one of them? Maybe we can just name the constructors as `with` but I'm > not &g

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-10 Thread Damian Guy
erde(final Serde valueSerde) public static Produced streamPartitioner(final StreamPartitioner partitioner) So if you only want to use one you can just use the function that takes one argument. > > Guozhang > > > On Wed, Aug 9, 2017 at 6:20 AM, Damian Guy wrote: >

[DISCUSS] 0.11.0.1 bug fix release

2017-08-16 Thread Damian Guy
Hi, It seems like it must be time for 0.11.0.1 bug fix release! Since the 0.11.0.0 release we've fixed 30 JIRAs that are targeted for 0.11.0.1: https://issues.apache.org/jira/browse/KAFKA-5700?jql=project%20%3D%20KAFKA%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%200.11.0.1%20ORDER%

Re: [DISCUSS] 0.11.0.1 bug fix release

2017-08-17 Thread Damian Guy
1.0.1 will be available? > -Srikanth > > On Wed, Aug 16, 2017 at 5:59 PM, Damian Guy wrote: > > > Hi, > > > > It seems like it must be time for 0.11.0.1 bug fix release! > > > > Since the 0.11.0.0 release we've fixed 30 JIRAs that > > are targeted

Re: [DISCUSS] 0.11.0.1 bug fix release

2017-08-17 Thread Damian Guy
Just a quick update. The list has reduced to 6 remaining issues: https://issues.apache.org/jira/projects/KAFKA/versions/12340632 Thanks to everyone for completing and/or moving tickets to future releases. Damian On Thu, 17 Aug 2017 at 09:50 Damian Guy wrote: > Hi Srikanth, > Optimist

Re: [DISCUSS] 0.11.0.1 bug fix release

2017-08-22 Thread Damian Guy
can get this cleaned up in the next day or two and then i can go about building an RC. Thanks, Damian On Thu, 17 Aug 2017 at 17:45 Damian Guy wrote: > Just a quick update. > > The list has reduced to 6 remaining issues: > https://issues.apache.org/jira/projects/KAFKA/versi

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-22 Thread Damian Guy
I've just updated the KIP with some additional changes targeted at StreamsBuilder Thanks, Damian On Thu, 10 Aug 2017 at 12:59 Damian Guy wrote: > >> Got it, thanks. >> >> Does it still make sense to have one static constructors for each spec, >> with one const

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-23 Thread Damian Guy
roupedTable and should mention which > one? There is just one sentence "Deprecate the existing overloads.", but > we don't deprecate all existing once. I personally don't care to much if > we spell deprecated method out explicitly, but right now it's not > cons

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-23 Thread Damian Guy
KIP has been updated. thanks On Wed, 23 Aug 2017 at 09:10 Damian Guy wrote: > Hi Matthias, > > >> KStream: >> leftJoin and outerJoin for KStream/KTable join should not have >> `JoinWindows` parameter >> >> Thanks! > > >> >> Nit: To

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-23 Thread Damian Guy
update it > Thanks, > Bill > > On Wed, Aug 23, 2017 at 4:15 AM, Damian Guy wrote: > > > KIP has been updated. thanks > > > > On Wed, 23 Aug 2017 at 09:10 Damian Guy wrote: > > > > > Hi Matthias, > > > > > > > > >> KSt

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-23 Thread Damian Guy
we add one for consistency? > > > > Thanks, > > Bill > > > > On Wed, Aug 23, 2017 at 4:15 AM, Damian Guy > wrote: > > > >> KIP has been updated. thanks > >> > >> On Wed, 23 Aug 2017 at 09:10 Damian Guy wrote: > >> > >

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-24 Thread Damian Guy
dd new `groupBy` and `groupByKey` methods that return `GroupedKStream`, we can't change the existing ones as that would break compatibility. So what would we name these methods? > > Guozhang > > > > On Wed, Aug 23, 2017 at 1:01 PM, Damian Guy wrote: > > > We alre

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-24 Thread Damian Guy
I've updated the kip to reflect Bill's comment and also to make StreamBuilder methods have topic as the first param, i.e., StreamBuilder#stream no longer accepts varargs. On Thu, 24 Aug 2017 at 09:12 Damian Guy wrote: > On Thu, 24 Aug 2017 at 02:49 Guozhang Wang wrote: > >&

Re: [DISCUSS] 0.11.0.1 bug fix release

2017-08-24 Thread Damian Guy
A quick update. There are 2 remaining issues, that both have patches available. Hopefully they will be merged soon and we can begin: https://issues.apache.org/jira/projects/KAFKA/versions/12340632 Thanks, Damian On Tue, 22 Aug 2017 at 10:29 Damian Guy wrote: > An update on the 0.11.0.1 rele

[VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-24 Thread Damian Guy
Hi, I'd like to kick off the voting thread for KIP-182: https://cwiki.apache.org/confluence/display/KAFKA/KIP-182%3A+Reduce+Streams+DSL+overloads+and+allow+easier+use+of+custom+storage+engines Thanks, Damian

Re: 答复: [ANNOUNCE] New Kafka PMC member: Jiangjie (Becket) Qin

2017-08-24 Thread Damian Guy
Congratulations. On Thu, 24 Aug 2017 at 17:44 Matthias J. Sax wrote: > Congrats! > > On 8/23/17 10:40 PM, Hu Xi wrote: > > Congrats Becket! > > > > > > > > 发件人: Guozhang Wang > > 发送时间: 2017年8月24日 13:32 > > 收件人: dev@kafka.apache.org > > 主题: Re: [ANNOUNCE] New Kaf

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-25 Thread Damian Guy
t; > > > > > > > On Thu, Aug 24, 2017 at 11:32 AM, Matthias J. Sax > > > wrote: > > > >> We now have > >> > >>> public synchronized KStream stream(final > Collection > >> topic, final Consumed options) > >> > &

Re: [DISCUSS] KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-25 Thread Damian Guy
, i.e, ChangeLogging, Caching, Metered. Presently they can't do this without jumping through various hoops. To provide this ability we presently need to use as the CachingStores are . They need to remain that way for the time being as it is how we can put some limits on memory usage. >

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-25 Thread Damian Guy
Thu, Aug 24, 2017 at 9:47 AM, Bill Bejeck wrote: > > > > > Thanks for the KIP! > > > > > > +1 > > > > > > Thanks, > > > Bill > > > > > > On Thu, Aug 24, 2017 at 12:25 PM, Damian Guy > > wrote: > > >

Re: [DISCUSS] 0.11.0.1 bug fix release

2017-08-29 Thread Damian Guy
We still have 2 outstanding issues to close. They both have patches available and will hopefully be completed shortly. Thanks, Damian On Thu, 24 Aug 2017 at 10:05 Damian Guy wrote: > A quick update. There are 2 remaining issues, that both have patches > available. Hopefully they will be

Re: [VOTE] KIP-138: Change punctuate semantics

2017-08-30 Thread Damian Guy
+1 On Wed, 30 Aug 2017 at 17:49 Guozhang Wang wrote: > Hello Michal and community: > > While working on updating the web docs and java docs for this KIP, I felt > that the term SYSTEM_TIME a bit confusing sometimes from a reader's > perspective as we are actually talking about wall-clock time. I

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-08-31 Thread Damian Guy
tore and Store#persistentSessionStore Please take a look and recast the votes. Thanks for your time, Damian On Fri, 25 Aug 2017 at 17:05 Matthias J. Sax wrote: > Thanks Damian. Great KIP! > > +1 > > > -Matthias > > On 8/25/17 6:45 AM, Damian Guy wrote: > > Hi, >

Re: [DISCUSS] 0.11.0.1 bug fix release

2017-09-01 Thread Damian Guy
Hi all, There is one last outstanding issue in 0.11.0.1. This will hopefully be resolved today. Assuming it is i will cut the first RC tomorrow. Thanks for your patience. Regards, Damian On Tue, 29 Aug 2017 at 10:53 Damian Guy wrote: > We still have 2 outstanding issues to close. They b

[VOTE] 0.11.0.1 RC0

2017-09-05 Thread Damian Guy
Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 0.11.0.1. This is a bug fix release and it includes fixes and improvements from 49 JIRAs (including a few critical bugs). Release notes for the 0.11.0.1 release: http://home.apache.org/~d

Re: [VOTE] 0.11.0.1 RC0

2017-09-06 Thread Damian Guy
Resending as i wasn't part of the kafka-clients mailing list On Tue, 5 Sep 2017 at 21:34 Damian Guy wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for release of Apache Kafka 0.11.0.1. > > This is a bug fix release and i

Re: [VOTE] KIP-182 - Reduce Streams DSL overloads and allow easier use of custom storage engines

2017-09-07 Thread Damian Guy
> > > +1 > > > > On Fri, Sep 1, 2017 at 3:45 PM, Matthias J. Sax > > wrote: > > > > > +1 > > > > > > On 9/1/17 2:53 PM, Bill Bejeck wrote: > > > > +1 > > > > > > > > On Thu, Aug 31, 2017 at 10:20 AM, Dami

<    1   2   3   4   5   6   7   8   >