Re: How distributed countByKey works in KStream ?

2016-09-01 Thread Tommy Q
should create your repartitioning topic you specify in > through() before you start your Kafka Streams application. > > > -Matthias > > > On 08/31/2016 09:07 PM, Guozhang Wang wrote: > > Hello Tommy, > > > > Which version of Kafka are you using? > > > >

Re: How distributed countByKey works in KStream ?

2016-08-31 Thread Tommy Q
gt; a 1 > b 1 > a 2 > b 2 > c 1 > > ? > > On Wed, Aug 31, 2016 at 5:47 AM, Tommy Q wrote: > > > Tried the word count example as discussed, the result in wc-out is wrong: > > > > a 1 > > > b 1 > > > a 1 > > > b 1 > > >

Re: How distributed countByKey works in KStream ?

2016-08-31 Thread Tommy Q
Hi @Matthias & @Michael I pushed the WordCountDemo code to GitHub https://github.com/deeplambda/kstream-debug, can you help to debug the demo ? Thanks, Tommy On Wed, Aug 31, 2016 at 11:47 AM, Tommy Q wrote: > Tried the word count example as discussed, the result in wc-out is wrong:

Re: How distributed countByKey works in KStream ?

2016-08-30 Thread Tommy Q
topics. > > The only explanation might be, that there is no repartitioning step. But > than the question would be, if there is a bug in Kafka Streams, because > between map() and countByKey() repartitioning is required. > > Can you verify that the result is correct? > > -Matth

Re: kafka-streams project compiles using maven but failed using sbt

2016-08-30 Thread Tommy Q
Got it. Thanks Michael. On Tue, Aug 30, 2016 at 2:31 AM, Michael Noll wrote: > Most probably because, in your build.sbt, you didn't enable the > -Xexperimental compiler flag for Scala. This is required when using Scala > 2.11 (as you do) to enable SAM for Java 8 lambda support. Because this >

Re: How distributed countByKey works in KStream ?

2016-08-30 Thread Tommy Q
tps://cwiki.apache.org/confluence/display/KAFKA/ > Kafka+Streams%3A+Internal+Data+Management > > and > > http://www.confluent.io/blog/data-reprocessing-with-kafka- > streams-resetting-a-streams-application > > > -Matthias > > > On 08/30/2016 06:55 AM, Tommy Q wrote:

Re: How distributed countByKey works in KStream ?

2016-08-29 Thread Tommy Q
Michael, Thanks for your help. Take the word count example, I am trying to walk through the code based on your explanation: val textLines: KStream[String, String] = builder.stream("input-topic") val wordCounts: KStream[String, JLong] = textLines .flatMapValues(_.toLowerCase.split("\