Re: Programmatic Kafka version detection/extraction?

gt; > >>>>>>>>>>> Here is sample: > >>>>>>>>>>> #Generated by Maven > >>>>>>>>>>> #Mon Oct 10 10:44:31 EDT 2011 > >>>>>>>>>>> version=10.0.1 > >>>>>>>>>>> groupId=com.google.guava > >>>>>>>>>>> artifactId=guava > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> > >>>>>>>>>>> Bhavesh > >>>>>>>>>>> > >>>>>>>>>>> On Tue, Nov 11, 2014 at 10:34 AM, Gwen Shapira < > >>>>> gshap...@cloudera.com > >>>>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> In Sqoop we do the following: > >>>>>>>>>>>> > >>>>>>>>>>>> Maven runs a shell script, passing the version as a > >> parameter. > >>>>>>>>>>>> The shell-script generates a small java class, which is then > >>>>> built > >>>>>>>>>> with a > >>>>>>>>>>>> Maven plugin. > >>>>>>>>>>>> Our code references this generated class when we expose > >>>>>>>> "getVersion()". > >>>>>>>>>>>> > >>>>>>>>>>>> Its complex and ugly, so I'm kind of hoping that there's a > >>>>> better way > >>>>>>>>>> to > >>>>>>>>>>> do > >>>>>>>>>>>> it :) > >>>>>>>>>>>> > >>>>>>>>>>>> Gwen > >>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, Nov 11, 2014 at 9:42 AM, Jun Rao >>> > >>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Currently, the version number is only stored in our build > >>>> config > >>>>>>>>>> file, > >>>>>>>>>>>>> gradle.properties. Not sure how we can automatically > >> extract > >>>> it > >>>>> and > >>>>>>>>>>>> expose > >>>>>>>>>>>>> it in an mbean. How do other projects do this? > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Jun > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, Nov 11, 2014 at 7:05 AM, Otis Gospodnetic < > >>>>>>>>>>>>> otis.gospodne...@gmail.com> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi Jun, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Sounds good. But is the version number stored anywhere > >> from > >>>>> where > >>>>>>>>>> it > >>>>>>>>>>>>> could > >>>>>>>>>>>>>> be gotten? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>> Otis > >>>>>>>>>>>>>> -- > >>>>>>>>>>>>>> Monitoring * Alerting * Anomaly Detection * Centralized > >> Log > >>>>>>>>>>> Management > >>>>>>>>>>>>>> Solr & Elasticsearch Support * http://sematext.com/ > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Tue, Nov 11, 2014 at 12:45 AM, Jun Rao < > >> jun...@gmail.com> > >>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Otis, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> We don't have an api for that now. We can probably expose > >>>> this > >>>>>>>>>> as a > >>>>>>>>>>>> JMX > >>>>>>>>>>>>>> as > >>>>>>>>>>>>>>> part of kafka-1481. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Mon, Nov 10, 2014 at 7:17 PM, Otis Gospodnetic < > >>>>>>>>>>>>>>> otis.gospodne...@gmail.com> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Is there a way to detect which version of Kafka one is > >>>>> running? > >>>>>>>>>>>>>>>> Is there an API for that, or a constant with this > >> value, or > >>>>>>>>>> maybe > >>>>>>>>>>>> an > >>>>>>>>>>>>>>> MBean > >>>>>>>>>>>>>>>> or some other way to get to this info? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Otis > >>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>> Monitoring * Alerting * Anomaly Detection * Centralized > >> Log > >>>>>>>>>>>>> Management > >>>>>>>>>>>>>>>> Solr & Elasticsearch Support * http://sematext.com/ > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> -- Guozhang > >> > -- Jeff Holoman Systems Engineer 678-612-9519

Re: no brokers found when trying to rebalance

328,327 > > > Topic: rawunstruct Partition: 3 Leader: 328 Replicas: 328,329,327 Isr: > > > 328,327 > > > Topic: rawunstruct Partition: 4 Leader: 328 Replicas: 329,327,328 Isr: > > > 328,327 > > > > > > However, when I list /kafka/brokers/ids on any of the ZK servers, I > don't > > > see any brokers ids. > > > > > > I did upgrade from CDH 5.2 to 5.3 but other components in the stack > seem > > to > > > be able to talk to Zookeeper just fine. > > > > > > Any pointers for troubleshooting? > > > > > > Thanks > > > > > > -- Jeff Holoman Systems Engineer 678-612-9519

Re: Routing modifications at runtime

naries are so big that they > > don't > > > fit in memory. Are there any patterns for bookkeeping internal data > > > structures and how route to them? > > > > > > T. > > > > > > > > > -- > > David Morales de Frías :: +34 607 010 411 :: @dmoralesdf > <https://twitter.com/dmoralesdf> > > > <http://www.stratio.com/> > Vía de las dos Castillas, 33, Ática 4, 3ª Planta > 28224 Pozuelo de Alarcón, Madrid > Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd > <https://twitter.com/StratioBD>* > -- Jeff Holoman Systems Engineer 678-612-9519

Re: LinkedIn Engineering Blog Post - Current and Future

> > Here's an overview of what LinkedIn plans to concentrate on in the > upcoming year. > > > https://engineering.linkedin.com/kafka/kafka-linkedin-%E2%80%93-current-and-future > > -Jon > -- Jeff Holoman Systems Engineer 678-612-9519

Re: Explicit control over flushing the messages

t; > Please let me know if there is any API or specific configuration to do > this. > > Thanks in advance, > Ponmani > -- Jeff Holoman Systems Engineer

Re: Kafka Poll: Version You Use?

t; > > otis.gospodne...@gmail.com> wrote: > > > > > > > > > Hi, > > > > > > > > > > With 0.8.2 out I thought it might be useful for everyone to see > which > > > > > version(s) of Kafka people are using. > > > > > > > > > > Here's a quick poll: > > > > > http://blog.sematext.com/2015/02/23/kafka-poll-version-you-use/ > > > > > > > > > > We'll publish the results next week. > > > > > > > > > > Thanks, > > > > > Otis > > > > > -- > > > > > Monitoring * Alerting * Anomaly Detection * Centralized Log > > Management > > > > > Solr & Elasticsearch Support * http://sematext.com/ > > > > > > > > > > > > > > > > > > > > > > > -- > Thanks, > Neha > -- Jeff Holoman Systems Engineer

Re: Best Practices for Java Consumers

it's stops consuming. > > > > Also if this doesn't exist I would be willing to start this document. I > > would think it would be near this page > > > https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example > > > > Thanks, > > Tom > -- Jeff Holoman Systems Engineer

Re: [DISCUSSION] Kafka 0.8.2.2 release?

ache.org/jira/browse/KAFKA-2189). Basically, if > > > they > > > > > use > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase. > The > > > > issue > > > > > > has since been fixed in trunk (just a snappy jar upgrade). Since > > > 0.8.3 > > > > is > > > > > > still a few months away, it may make sense to do an 0.8.2.2 > release > > > > just > > > > > to > > > > > > fix this issue. Any objections? > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Grant Henke > > > Software Engineer | Cloudera > > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke > > > > > > -- Jeff Holoman Systems Engineer

Re: Error handling in New AsyncProducer

r cluster. The callback is > > only triggered when the cluster is reachable . > > > > --SK > > > > > > -- > Thanks and Regards, > Madhukar Bharti > Mob: 7845755539 > -- Jeff Holoman Systems Engineer

Re: Error handling in New AsyncProducer

I should've been more specific...if the producer loses total access to all brokers...eg. some kind of network issue. On Mon, Aug 17, 2015 at 1:50 PM, Jeff Holoman wrote: > Actually this won't work. The problem is if the producer loses > connectivity to the broker, then messages

Re: open source projects based on kafka

; > > Sent from my iPhone > > > > > > > > > > > > > On Sep 12, 2015, at 10:34 PM, Li Tao > > > wrote: > > > > > > > > > > > > > > Hi Hackers, > > > > > > > > > > > > > > This is Lee, a learner of kafka, i have read the original paper > > on > > > > > kafka, > > > > > > > and walked through the document. > > > > > > > > > > > > > > I think the best way to learn sth is to write and read code > about > > > > it. I > > > > > > am > > > > > > > wondering is there any open source code / system which is based > > on > > > > > kafka > > > > > > so > > > > > > > that i can read or contribute to? Not too complex, not too > > simple. > > > > > > > > > > > > > > Thanks a lot! > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Hisham Mardam-Bey > > > -=[ CTO ]-=-[ Mate1 Inc. ]=- > > > > > > A: Because it messes up the order in which people normally read text. > > > Q: Why is top-posting such a bad thing? > > > A: Top-posting. > > > Q: What is the most annoying thing in e-mail? > > > > > > -=[ Codito Ergo Sum ]=- > > > > > > -- Jeff Holoman Systems Engineer

Re: New and updated producers and consumers

a > >> > >> Please let me know which producer and consumer (both high level and > >> simple) class/object should I be using > >> > >> Thanks a lot, > >> Prabhjot > >> > > > > > > > > -- > > - > > "There are only 10 types of people in the world: Those who understand > > binary, and those who don't" > > > > > > -- > - > "There are only 10 types of people in the world: Those who understand > binary, and those who don't" > -- Jeff Holoman Systems Engineer

Re: New and updated producers and consumers

> Thanks, > Cliff > > On Thu, Nov 5, 2015 at 11:18 AM, Jeff Holoman > wrote: > > > Prabhjot, > > > > The answer changes slightly for the Producer and Consumer and depends on > > your timeline and comfort with using new APIs > > > > Today and in

Re: Debug kafka code in intellij

gt; > > > > > > > > > > > ./gradlew -PscalaVersion=2.11.7 releaseTarGz > > > > > > > > > > > > In some cases, I feel the need for debugging the code within > > > IntelliJ. > > > > > > > > > > > > e.g. I, currently, want to debug the ConsumerOffsetCheker to see > how > > > it > > > > > > communicates with zookeeper. > > > > > > > > > > > > How can I do that with IntelliJ ?? > > > > > > > > > > > > Currently, if I try and debug the file > ConsumerOffsetChecker.scala in > > > > > > IntelliJ, I get this message:- > > > > > > > > > > > > Error:scalac: Output path > > > > > > /Users/pbharaj/Desktop/Dev/OpenSource4/Kafka-0.8.2.1/build is > shared > > > > > > between: Module 'Kafka-0.8.2.1' production, Module > 'Kafka-0.8.2.1' > > > > > > > > > > > > > > > > > > > tests > > > > > > Please configure separate output paths to proceed with the > > > > > > > > > > > > > > > > > > > compilation. > > > > > > TIP: you can use Project Artifacts to combine compiled classes if > > > > > > > > > > > > > needed. > > > > > > > > > > > > > > > > > > Regards, > > > > > > Prabhjot > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Jeff Holoman Systems Engineer

Re: Kafka with Flume

This is probably better posted on the flume dev or users list ( d...@flume.apache.org and u...@flume.apache.org). I suspect you'll get a better response there (or even the Cloudera community forums as there is likely some Kite SDK experience there) I think what you are saying is that you have a mo

Re: [ANNOUNCE] New committer: Grant Henke

Well done Grant! Congrats! On Thu, Jan 12, 2017 at 1:13 PM, Joel Koshy wrote: > Hey Grant - congrats! > > On Thu, Jan 12, 2017 at 10:00 AM, Neha Narkhede wrote: > > > Congratulations, Grant. Well deserved! > > > > On Thu, Jan 12, 2017 at 7:51 AM Grant Henke wrote: > > > > > Thanks everyone! >