Re: simple test fails on OSX

Hi Eric, I can reproduce your problem although I have no idea yet what is going wrong. Do you have any updates ? Regards, Manu Zhang On Mon, Aug 8, 2016 at 11:03 AM Eric Newton wrote: > I wrote a little unit test and it works as I expect on linux (ubuntu, > redhat, openjdk 1.8). > &

Re: Kafka Streams change log behavior

ata and restart from scratch. > > We are working on a better re-processing user experience now and once there > is a proposal we will present it to the community. > > > How's everything on the Gearpump side? Looking forward to hear from you for > any news :) > > Guozhan

Re: Kafka Streams change log behavior

ic and can be read by any consumer. > > > -Matthias > > On 05/23/2016 10:48 AM, Manu Zhang wrote: > > Thanks Matthias. Is there a way to allow users to read change logs from a > > previous application ? > > > > On Mon, May 23, 2016 at 3:57 PM Matthias J. Sax > >

Re: Kafka Streams change log behavior

nable per default for those topics. > You still might see some duplicates in your output. Currently, Kafka > Streams guarantees at-least-once processing (exactly-once processing is > on the roadmap though). > > > -Matthias > > On 05/23/2016 06:10 AM, Manu Zhang wrote: > &g

Kafka Streams change log behavior

s required ? Thanks, Manu Zhang

Re: Broker Exception: Attempt to read with a maximum offset less than start offset

; > > > > > I'm asking because there is a Flink user who is using Kafka 0.8.2.1 as > > well > > > who's reporting a very similar issue on SO: > > > > > > > > > http://stackoverflow.com/questions/34982483/flink-streaming-job-switched-

Re: Broker Exception: Attempt to read with a maximum offset less than start offset

Hi, Any suggestions for this issue or do I need to provide more information ? Any links I can refer to would be also very helpful. Thanks, Manu Zhang On Tue, Jan 19, 2016 at 8:41 PM, Manu Zhang wrote: > Hi all, > > Is KAFKA-725 Broker Exception: Attempt to read with a maximum of

Broker Exception: Attempt to read with a maximum offset less than start offset

sumer failed with the fetch exception. This happened only when producers are producing at high throughput. Any ideas would be much appreciated. Thanks, Manu Zhang

Re: Kafka 0.8.1.1

please checkout the kafka integration test under integration.kafka.api package. On Thu, Jun 4, 2015 at 8:31 AM, Vadim Keylis wrote: > We are using kafka 0.8.1.1. I am writing java consumer that will use simple > api and keep offset internally. I would like to write automated tests, that > will u

Re: Kafka Sync Producer threads(ack=0) are blocked

Hi ankit, I think you could find out the stacktrace of the lock "*0x000602358ee8" *that blocks the producer. Thanks, Manu On Fri, Mar 27, 2015 at 3:11 PM ankit tyagi wrote: > any insight would be useful. > > On Tue, Mar 24, 2015 at 11:06 AM, ankit tyagi > wrote: > >> Hi All, >> >> Currentl

Re: Kafka-Storm: troubleshooting low R/W throughput

Hi Emmanuel, You can firstly run a kafka producer perf (bin/kafka-producer-perf-test.sh) test with your storm consumers and kafka consumer perf (bin/ kafka-consumer-perf.test.sh) test with your own producers respectively to see if the bottleneck is really in kafka. Thanks, Manu Zhang On Mon

Re: Error thrown when tear down KafkaServer in tests

Thanks Jun. Hope we could get that in soon On Sat Jan 24 2015 at 上午1:50:57 Jun Rao wrote: > Yes, we have already filed https://issues.apache.org/ > jira/browse/KAFKA-1887 > to track this. > > Thanks, > > Jun > > > On Fri, Jan 23, 2015 at 5:30 AM, Manu Zhang >

Re: kafka topic not exist after creation with api

Jun > > On Thu, Jan 22, 2015 at 9:44 PM, Manu Zhang > wrote: > > > Hi all , > > My application creates kafka topic at runtime with AdminUtils.createTopic > > but that topic is not available when I try to produce to it. > > > > If I run "bin/kafka-cons

Error thrown when tear down KafkaServer in tests

Hi all, I've been using KafkaServerTestHarness to integration test my kafka application. All the tests passed but when the server was teared down the error below was thrown. [ERROR] [01/23/2015 16:34:19.118] [logger] Controller 0 epoch 1 initiated state change for partition [testTopic844927,0] f

kafka topic not exist after creation with api

Hi all , My application creates kafka topic at runtime with AdminUtils.createTopic but that topic is not available when I try to produce to it. If I run "bin/kafka-console-consumer.sh --topic $topic --zookeeper $zkStr --from-beginning" it would throw UnknownTopicOrPartitionException. Also, I fou

Re: can't iterate consumed messages when checking errorCode first

Rao wrote: > Did you get any error code in the response? Also, make sure fetchSize is > larger than the largest single message. > > Thanks, > > Jun > > On Sun, Jan 18, 2015 at 4:54 PM, Manu Zhang > wrote: > > > Hi all, > > > > I'm

can't iterate consumed messages when checking errorCode first

t, soTimeout, soBufferSize, clientId)* * val request = new FetchRequestBuilder()* *.addFetch(topic, partition, offset, fetchSize)* *.build()* * consumer.fetch(request).messageSet(topic, partition).iterator* Any thoughts ? Thanks, Manu Zhang

Re: Partition id of consumed message?

It's a field of MessageAndMetadata actually but there won't be difference from Scala's view. On Thu, Nov 27, 2014 at 9:57 AM, Palur Sandeep wrote: > Thank you. but I dont see that method. I dont know whats wrong > > On Wed, Nov 26, 2014 at 6:31 PM, Jun Rao wrote: > > > Yes, the consumer iterato

Re: benchmark kafka on 10GbE network

N > partitions N threads can all be working at once. > > On Tue, Nov 25, 2014 at 3:03 AM, Manu Zhang > wrote: > > > Thanks for the explanation. > > > > Here are some stats for I/O thread. > > > > *io-ratio 0.155* > > *io-time-ns-avg 16418.5* >

Re: benchmark kafka on 10GbE network

1991 300870 > org.apache.kafka.common.protocol.types.Type$6.write > 11 0.97% 91.26%1961 300789 > org.apache.kafka.clients.producer.internals.RecordAccumulator.ready > 12 0.96% 92.22%1951 300726 > > -Jay > > On Thu, Nov 20, 2014 at 5:42 PM, Manu Zhang >

Re: benchmark kafka on 10GbE network

tell us where the time is going. If you sent that around it may > be informative. > > -Jay > > > On Thu, Nov 20, 2014 at 12:41 AM, Manu Zhang > wrote: > > > Thanks Jay. The producer metrics from jconsole is quite helpful. > > > > I've switched to t

Re: benchmark kafka on 10GbE network

and see if that improves throughput (if so it > is probably a client bottleneck). > > -Jay > > On Tue, Nov 18, 2014 at 4:44 PM, Manu Zhang > wrote: > > > Thanks Jay for the quick response. > > > > Yes, it's a single producer and consumer both configured with mul

Re: benchmark kafka on 10GbE network

he clients > or the brokers. You basically have to look at top, iostat, and the jmx > metrics for clues. The producer has good metrics that explain whether it is > spending most of its time waiting or sending data. Not sure if there is a > similar diagnostic for the consumer. > >

benchmark kafka on 10GbE network

Hi all, I have been trying out kafka benchmarks described in Jay's benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines . I'm able to get similar results on a 4-node GbE