Re: Supervisord for Kafka 0.8.1

t;-Xms8g -Xmx8g" > > fi > > > > if [ "abc$KAFKA_JVM_PERFORMANCE_OPTS" == "abc" ]; then > > export KAFKA_JVM_PERFORMANCE_OPTS="-server -XX:+UseG1GC > > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 > > -Djava.awt.he

Supervisord for Kafka 0.8.1

Hi, Am having trouble configuring Kafka server starts with supervisord. Has anyone from this group succeeded in integrating Kafka server start and stop via supervisord? Can you please share the snippet of his out of configured? Thanks Kashyap

Re: Keeping Zookeeper and Kafka Server Up

We use supervisord for this. It ensures that the processes are always up and running. Thanks Kashyap On Wednesday, June 17, 2015, Shayne S wrote: > kafka-server-start.sh has a -daemon option, but I don't think Zookeeper has > it. > > On Tue, Jun 16, 2015 at 11:32 PM, Su She > wrote: > > > It s

Re: Security in 0.8.2 beta

://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > / > > On Wed, Nov 12, 2014 at 2:41 AM, Mathias Herberts < > mathias.herbe...@gmail.com> wrote: > > > Simply encrypt your messages with a PSK between producers

Security in 0.8.2 beta

Hi, Is there a way to secure the topics created in Kafka 0.8.2 beta? The need is to ensure no one is asked to read data from the topic without authorization. Regards Kashyap

Re: Partitions per Machine for a topic

n DDS Site Reliability Engineer > Kafka, Zookeeper, Samza SRE > Mobile: 505.385.1484 > BlueJeans: https://www.bluejeans.com/chaskins > > > chask...@linkedin.com > https://www.linkedin.com/in/clarkhaskins > There is no place like 127.0.0.1 > > > > > On 7/23/14, 2:14 PM,

Partitions per Machine for a topic

HI, Is the maximum no. of partitions for a topic dependent on the no. of machines in a kafka cluster? For e.g., if I have 3 machines in a cluster, can I have 5 partitions with a caveat that one machine can host multiple partitions for a given topic? Regards, Kashyap

Partition reassign Kafka 0.8.1.1

Hi, I was testing out Kafka 0.8.1.1 and found that i get the following exception during partition re-assignment : *./kafka-reassign-partitions.sh --path-to-json-file ritest.json --zookeeper localhost:2181* *Partitions reassignment failed due to Partition reassignment currently in progress for Map

Re: Loss of Leader in Kafka

eeing this problem since delete topic is not supported in Kafka. > Any attempt to delete a topic may leave your cluster in an unstable state. > We plan to release 0.8.2 with delete topic support or you could try the > latest trunk. > > > On Mon, May 12, 2014 at 9:39 AM, Kashyap M

Re: Loss of Leader in Kafka

in trunk. > Could you give it a try and see if you see the same issue? > > Thanks, > > Jun > > > On Mon, May 12, 2014 at 9:39 AM, Kashyap Mhaisekar > > >wrote: > > > Hi, > > I am hitting a strange exception while creating a topic in Kafka - > > Step

Loss of Leader in Kafka

Hi, I am hitting a strange exception while creating a topic in Kafka - Steps to generate this- 1. Created a topic multipartition_test with 2 partitions and 2 replicas 2. Added some data to this topics and verified data is coming up for both partitions 3. Deleted the topic. Checked only the zookeepe

Re: Log Retention in Kafka

pic > > Thanks, > > Jun > > > On Sun, May 4, 2014 at 10:11 PM, Kashyap Mhaisekar >wrote: > > > Is there a way to do this at runtime using some available scripts in > > kafka/bin? If so, any pointers on which script? > > > > Regards, > > K

Re: Log Retention in Kafka

Is there a way to do this at runtime using some available scripts in kafka/bin? If so, any pointers on which script? Regards, Kashyap On Tue, Apr 22, 2014 at 11:11 PM, Kashyap Mhaisekar wrote: > Thanks Joel. Am using version 2.8.0. > > Thanks, > Kashyap > > > On Tue, Ap

Re: Log Retention in Kafka

specify time-based retention using log.retention.minutes which > will apply to all topics. You can override that on per-topic basis - > see further down in the above page under "topic-level configuration" > > On Tue, Apr 22, 2014 at 02:34:24PM -0500, Kashyap Mhaisekar wrote: > &

Log Retention in Kafka

Hi, I wanted to set the message expiry for a message on a kafka topic. Is there anything like this in kafka? I came across a property - *log.retention.hours* and *topic.log.retention.hours* Had some queries around it.And it was mentioned that topic.log.retention.hours is per topic configuration. Ha

Re: Leader: -1 on Kafka

e-change.log for that topic/partition? > > On Mon, Apr 21, 2014 at 03:36:52PM -0500, Kashyap Mhaisekar wrote: > > Yes. > > I have 3 kafka brokers and I have created one topic with 1 partition and > 1 > > replication (default options). All params are default. I was testing

Re: Leader: -1 on Kafka

partition are available, but cannot comment further without details. > > On Mon, Apr 21, 2014 at 10:54:59AM -0500, Kashyap Mhaisekar wrote: > > Hi, > > At times, some of kafka topics end up showing the leader as -1. After > this, > > the messages dont get added to the topic nor

Leader: -1 on Kafka

Hi, At times, some of kafka topics end up showing the leader as -1. After this, the messages dont get added to the topic nor consumed. I tried digging into why the leader turns -1 *(leader: -1)* Is there a reason why this happens and how it can be resolved? Regards, kashyap