Re: comment in addStandbyTasks() method of StreamThread has some typos

2017-06-16 Thread john cheng
ok, I'll open an PR to fix this. 2017-06-17 0:59 GMT+08:00 Matthias J. Sax : > Thanks for reporting this! > > Would you like to open a MINOR PR to fix it? Don't think we need a Jira > for this. > > -Matthias > > On 6/16/17 9:26 AM, john cheng wrote: > > ht

comment in addStandbyTasks() method of StreamThread has some typos

2017-06-16 Thread john cheng
https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java#L1345 This Line: log.info("{} Adding assigned standby tasks {}", logPrefix, partitionAssignor .activeTasks()); The parameter is active task, but the info content is stand

kafka streams first time did not print output

2017-06-12 Thread john cheng
Hi there, I'm testing Kafka Streams's print() method, here is the code: props.put(StreamsConfig.APPLICATION_ID_CONFIG, "dsl-wc1"); props.put(StreamsConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG, 0); KStream source = builder.stream("dsl-input1"); KTable countTable = source

Re: kafka streams checkpoint and restore understanding

2017-06-09 Thread john cheng
cksDB contains all 6 messages and thus there is nothing > to restore. > > Does this make sense? > > -Matthias > > On 6/7/17 7:02 PM, john cheng wrote: > > I have two app instance, input topic has 2 partitions, each instance > config > > one thread and one replica

Re: kafka streams docs about topology picture only one source node

2017-06-08 Thread john cheng
e is only one source node. > > There is no 1-to-1 relationship between input topics and source node, > and thus, the picture is not wrong... > > Do you find that the picture is confusion/miss-leading? > > > -Matthias > > On 6/8/17 5:58 PM, john cheng wrote: > > Kaf

kafka streams docs about topology picture only one source node

2017-06-08 Thread john cheng
Kafka streams topology can define one or many SourceNode. The picture on official document < http://kafka.apache.org/0102/documentation/streams#streams_architecture_tasks > only draw one source node in some place: 1. Stream Partitions and Tasks 2. Threading Model 3. Local StateStore And the topolo

kafka streams checkpoint and restore understanding

2017-06-07 Thread john cheng
I have two app instance, input topic has 2 partitions, each instance config one thread and one replicas. also, instance1's state-store is /tmp/kafka-streams, instance2's state-store is /tmp/kafka-streams2. now I do this experiment to study checkpointin kafka streams (0.10.0.0). 1. start instance1,

Re: kafka streams changelog topic value

2017-06-07 Thread john cheng
001 p:0,o:3,k:msg1,v:0002 p:0,o:4,k:msg3,v:0002 p:1,o:0,k:msg2,v:0001 p:1,o:1,k:msg4,v:0001 p:1,o:2,k:msg2,v:0002 p:1,o:3,k:msg2,v:0003 2017-06-07 18:42 GMT+08:00 john cheng : > I add some log on StoreChangeLog > > for (K k : this.dirty) { > V v = getter.g

Re: kafka streams changelog topic value

2017-06-07 Thread john cheng
ill output kafka: so this may be problem of console-consume. 2017-06-07 18:24 GMT+08:00 john cheng : > I'm running WordCountProcessorDemo with Processor API. and change > something below > 1. config 1 stream-thread and 1 replicas > 2. change inMemory() to persistent() >

kafka streams changelog topic value

2017-06-07 Thread john cheng
I'm running WordCountProcessorDemo with Processor API. and change something below 1. config 1 stream-thread and 1 replicas 2. change inMemory() to persistent() MyKakfa version is 0.10.0.0. After running streaming application, I check msg output by console-consumer ➜ kafka_2.10-0.10.0.0 bin/kafka-c

question about NetworkClient inFlightRequests operation

2016-01-21 Thread john cheng
I'm reading new client design of version 0.9. and I has a question of inFlightRequests in and out. Here is the basic flow : When Sender send a ClientRequest to NetworkClient, it add to inFlightRequests indicator in-flight requests ``` private void doSend(ClientRequest request, long now) {