kafka service delete the topic directory unexpectedly

2013-06-17 Thread panfei
our message content as below : {"dataType":"exception","appId":"1001","channelId":"appStore","version":"1.9.3","extra":"","ts":"2013-06-17 15:23:53"," deviceId":"c2605371b2b566958aa1e341cc60657dda3ccb44","stacktrace":"=异常崩溃报告=&&name:&&CALayerInvalidGeomet ry&&reason:&&CALay

Re: Kafka stats description

2013-06-17 Thread Hanish Bansal
Hi Jun As per my understanding: TotalFetchRequestMs - Total time of fetch requests in *microseconds* AvgFetchRequestMs - Average time of all fetch requests in *microseconds* MaxFetchRequestMs - Maximum time of a fetch request in *microseconds* ** Here I am taking Ms as Microsecond. Am i right

Kafka 0.8 producer: Failed to send messages after 3 tries

2013-06-17 Thread Markus Roder
Hi together, I'm collecting currently my first expirience with kafka for a whole no project in our company. We already had a first alpha version of our new system running with kafka version 7.2. But as the kafka version 0.8 will support replication, we thought about already testing this version

Re: problems with one of my tests and the latest 0.8 branch

2013-06-17 Thread Jason Rosenberg
Ok, So it seems the issue is related somehow to how I've wrapped the server (using a container app, using the maven pom from "./sbt make-pom"). If I start the server using the script "kafka-server-start.sh", it works fine. Still looking (but haven't been able to see anything obvious so far). J

Re: problems with one of my tests and the latest 0.8 branch

2013-06-17 Thread Jason Rosenberg
Looking at the convenience script for running kafka server, it builds a classpath by looking in several directories (looking at kafka-run-class.sh). This script includes a jar under perf/target, should I care about that, when launching a mavenized version of the server? Also, it includes jars und

Re: problems with one of my tests and the latest 0.8 branch

2013-06-17 Thread Joe Stein
you can use this to build the binary distributable ./sbt release-tar On Mon, Jun 17, 2013 at 8:17 PM, Jason Rosenberg wrote: > Looking at the convenience script for running kafka server, it builds a > classpath by looking in several directories (looking at > kafka-run-class.sh). > > This scr

Re: problems with one of my tests and the latest 0.8 branch

2013-06-17 Thread Jason Rosenberg
Another thing I notice, is that the sample 'config/server.properties' in many cases seems to have no correlation to the default config properties. In some cases, it offers much greater allocation (e.g. for socket.send.buffer.bytes, and for others much less. Have all the sample settings in config/

Re: problems with one of my tests and the latest 0.8 branch

2013-06-17 Thread Jason Rosenberg
Joe, I there also a way to generate a sources jar via sbt? Thanks, Jason On Mon, Jun 17, 2013 at 5:28 PM, Jason Rosenberg wrote: > Another thing I notice, is that the sample 'config/server.properties' in > many cases seems to have no correlation to the default config properties. > In some c

Re: problems with one of my tests and the latest 0.8 branch

2013-06-17 Thread Joe Stein
Yup, depending on your Scala version ./sbt "++2.9.2 package" 2.8.0, 2.8.2, 2.9.1, 2.9.2 are available /* Joe Stein, Chief Architect http://www.medialets.com Twitter: @allthingshadoop Mobile: 917-597-9771 */ On Jun 17, 2013, at 8:41 PM, Jason Rosenberg wrote: > Joe, > > I there also a way to

Re: problems with one of my tests and the latest 0.8 branch

2013-06-17 Thread Jason Rosenberg
Hmmmthat's not working for me (no *.sources.jar files are generated). do I need to add a flag? On Mon, Jun 17, 2013 at 5:47 PM, Joe Stein wrote: > Yup, depending on your Scala version > > ./sbt "++2.9.2 package" > > 2.8.0, 2.8.2, 2.9.1, 2.9.2 are available > > /* > Joe Stein, Chief Archite

RE: production consumer

2013-06-17 Thread S Ahmed
Greetings, Are there any code samples of consumer that is used in production? I'm looking for a something that is a daemon that has x number of threads and reads messages of kafka. I've looked at the sampleconsumer etc., but those are just one of runs that exit. i'm guessing that you would also

Re: 0.8 backup strategy anyone?

2013-06-17 Thread Jun Rao
We can look into offset preserving mirroring the the future. Note that even with this approach, the offsets in the target cluster will be slightly behind those in the source cluster since the mirroring will be async. Since not all offsets will be preserved. Thanks, Jun On Sun, Jun 16, 2013 at 3

Re: kafka service delete the topic directory unexpectedly

2013-06-17 Thread Jun Rao
Which version of Kafka are you using? Are you on Windows? Thanks, Jun On Mon, Jun 17, 2013 at 12:49 AM, panfei wrote: > our message content as below : > > > {"dataType":"exception","appId":"1001","channelId":"appStore","version":"1.9.3","extra":"","ts":"2013-06-17 > 15:23:53"," > > deviceId":

Re: Kafka stats description

2013-06-17 Thread Jun Rao
Correct, except that they are millisecs. Thanks, Jun On Mon, Jun 17, 2013 at 1:29 AM, Hanish Bansal < hanish.bansal.agar...@gmail.com> wrote: > Hi Jun > > As per my understanding: > > TotalFetchRequestMs - Total time of fetch requests in *microseconds* > > AvgFetchRequestMs - Average time of a

Re: Kafka 0.8 producer: Failed to send messages after 3 tries

2013-06-17 Thread Jun Rao
Are you using the latest 0.8 code? If so, broker.list is now changed to metadata.broker.list (see https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Examplefor an example). Thanks, Jun On Mon, Jun 17, 2013 at 7:36 AM, Markus Roder wrote: > Hi together, > > I'm collecting currentl

Re: production consumer

2013-06-17 Thread Jun Rao
Have you looked at https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example ? Thanks, Jun On Mon, Jun 17, 2013 at 7:43 PM, S Ahmed wrote: > Greetings, > > Are there any code samples of consumer that is used in production? I'm > looking for a something that is a daemon that ha

Upgrade to Zookeeper 3.4.5 stable ?

2013-06-17 Thread Ran RanUser
I'm running Zookeeper 3.4.5 server and using 3.4.5 clients in other code. Will Kafka switch to Zookeeper 3.4.5 soon? Should I expect any potential issues if Kafka's Zookeeper client is 3.3.4 while our server is 3.4.5? Thanks!

Re: kafka service delete the topic directory unexpectedly

2013-06-17 Thread panfei
we are using kafka-0.7.2 and deploy it on CentOS 6.4 Final 64bit. 2013/6/18 Jun Rao > Which version of Kafka are you using? Are you on Windows? > > Thanks, > > Jun > > > On Mon, Jun 17, 2013 at 12:49 AM, panfei wrote: > > > our message content as below : > > > > > > > {"dataType":"exception","

Re: problems with one of my tests and the latest 0.8 branch

2013-06-17 Thread Jason Rosenberg
Ok, Contrary to what I said above, the issue does seem to occur when kafka is launched by the sample startup scripts. I've filed a bug, with a test-case: https://issues.apache.org/jira/browse/KAFKA-945 I'm hoping there's a simple thing to change in the test code that will solve this (but I susp