Re: KafkaSpout

2014-09-15 Thread Harsha
-storm-starter/blob/develop/src/main/scala/com/miguno/kafkastorm/storm/KafkaStormDemo.scala -Harsha On Mon, Sep 15, 2014, at 04:37 AM, siddharth ubale wrote: > Hi all, > > I am using a kafka spout for reading data from a producer into my storm > topology. On the kafkaconfig : >

Re: KafkaSpout

2014-09-16 Thread Harsha
Are you running this using storm LocalCluster and you want to use external zookeeper for LocalCluster?. The latest code has those changes you can pass params to LocalCluster(localhost,2181). -Harsha On Mon, Sep 15, 2014, at 10:30 PM, siddharth ubale wrote: > Hi harsha, > > Yes i did c

Re: Automatic preferred replica election

2014-09-26 Thread Harsha
Neha, I am interested in picking it up. Assigining JIRA to myself. Thanks, Harsha On Fri, Sep 26, 2014, at 02:05 PM, Neha Narkhede wrote: > +1 on having someone pick up the JIRA. I'm happy to mentor a contributor > through that JIRA towards a checkin. > > On Fri, Sep 26,

Re: Installation and Java Examples

2014-10-10 Thread Harsha
le. hope that helps. -Harsha On Fri, Oct 10, 2014, at 12:32 PM, Mohit Anchlia wrote: > I am new to Kafka and very little familiarity with Scala. I see that the > build requires "sbt" tool, but do I also need to install Scala > separately? > Is there a detailed documentation on sof

Re: Monitoring connection with kafka client

2014-10-17 Thread Harsha
se nagios alert on those stats, If the consumed messages drops below certain threshold . -Harsha On Fri, Oct 17, 2014, at 01:08 AM, Alex Objelean wrote: > @Neha Narkede > > Though monitoring the health of Kafka & Zookeeper clusters directly is > useful, it might not be enoug

Re: Kafka producer iOS and android

2014-10-20 Thread Harsha
://github.com/mozilla-metrics/bagheera . This project used kafka 0.7 but you can see how its implemented. Hope that helps. -Harsha On Mon, Oct 20, 2014, at 08:45 AM, Josh J wrote: > hi > > Is it possible to have iOS and android to run the code needed for Kafka > producers ? I

Re: Clean Kafka Queue

2014-10-21 Thread Harsha
you can use log.retention.hours or log.retention.bytes to prune the log more info on that config here https://kafka.apache.org/08/configuration.html if you want to delete a message after the consumer processed a message there is no api for it. -Harsha On Tue, Oct 21, 2014, at 08:00 AM, Eduardo

Re: Errors after reboot on single node setup

2014-10-22 Thread Harsha
reconn ect (org.apache.zookeeper.ClientCnxn) java.net.ConnectException: Connection refused) kill -9 kafka-broker restart zookeeper and than kafka-broker leads into the the error posted by Ciprian. Ciprian, Can you open a jira for this. Thanks, Harsha On Wed, Oct 22, 2014, at 10:03 AM, Neha

Re: Subscibe

2014-11-13 Thread Harsha
Rohit, Please send a mail to users-subscr...@kafka.apache.org. more info here http://kafka.apache.org/contact.html. -Harsha On Wed, Nov 12, 2014, at 09:20 PM, Rohit Pujari wrote: > Hello there: > > I'd like to be added to the mailing list > > Thanks, > -- >

Re: kafka application logdir

2014-11-17 Thread Harsha
you can configure it under /opt/kafka/config/log4j.properties and look for kafka.log.dir On Mon, Nov 17, 2014, at 11:11 AM, Jimmy John wrote: > How do I configure the application kafka log dir? > > Right now the default is /var/log/upstart/kafka.log . I want to point it > to > a different mount d

Re: Partition Key Cannot be Send Out by Producer

2014-11-20 Thread Harsha
Hi Haoming, Take a look at the code here https://github.com/stealthly/scala-kafka/blob/master/src/main/scala/KafkaProducer.scala for your partKey it should be string and when you converting it into byte array you can use partKey.getBytes("UTF8") -Harsha On Thu, No

Re: Partition Key Cannot be Send Out by Producer

2014-11-20 Thread Harsha
also the (key: Key, value: Val, topic: Option[String]) "value" should be a string converted to a byte array. Can you send a example of your key and value data. On Thu, Nov 20, 2014, at 04:53 PM, Haoming Zhang wrote: > Hi Harsha, > > Thanks for suggestion! > > I have

Re: Kafka restart takes a long time

2014-11-22 Thread Harsha
Rajiv , which version of kafka are you using and do you see any errors when the server goes down after sending few messages. -Harsha On Sat, Nov 22, 2014, at 01:05 PM, Rajiv Kurian wrote: > The brokers also seem unavailable while this is going on. Each of these > log me

Re: Kafka restart takes a long time

2014-11-22 Thread Harsha
It might logs check your kafka logs dir (server logs) . Kafka can produce lot of logs in a quick time make sure thats whats in play here. -Harsha On Sat, Nov 22, 2014, at 01:37 PM, Rajiv Kurian wrote: > Actually see a bunch of errors. One of the brokers is out of space and > this >

Re: High level Consumer API doesnt receive 10MB messages?

2014-12-01 Thread Harsha
have you set fetch.message.max.bytes to 10mb or more in your consumer config. -Harsha On Mon, Dec 1, 2014, at 07:27 PM, Palur Sandeep wrote: > Hi all, > > Consumer doesnt receive message if it is big: When the producer sends > 256kb messages to broker, consumer is able to retrieve

Re: High level Consumer API doesnt receive 10MB messages?

2014-12-02 Thread Harsha
=10485800 > replica.fetch.max.bytes=104858000 > replica.fetch.wait.max.ms=2000 > > > On Mon, Dec 1, 2014 at 10:03 PM, Harsha wrote: > > > have you set fetch.message.max.bytes to 10mb or more in your consumer > > config. > > -Harsha > > > > On Mon, De

Re: Producer connection unsucessfull

2014-12-05 Thread Harsha
I think the default port for kafka running there is 6667. Can you check server.properties to see whats the port number -Harsha On Fri, Dec 5, 2014, at 06:10 AM, Marco wrote: > Yes, it's online and version -0.8.1.2.2.0.0-1084. jps lists it also > > 2014-12-05 14:56 GMT+01:00 s

Re: Producer connection unsucessfull

2014-12-05 Thread Harsha
from my > > host...don't know if this can intefere? > > > > > > 2014-12-05 15:14 GMT+01:00 Harsha : > >> I think the default port for kafka running there is 6667. Can you check > >> server.properties to see whats the port number > >> -Harsha &

Re: kafka deleted old logs but not released

2015-01-07 Thread Harsha
Yonghui, Which OS you are running. -Harsha On Wed, Jan 7, 2015, at 01:38 AM, Yonghui Zhao wrote: > Yes and I found the reason rename in deletion is failed. > In rename progress the files is deleted? and then exception blocks file > closed in kafka. > But I don't kn

Re: Create topic, then fetch offset

2015-01-13 Thread Harsha
It looks like you are not adding a ZkStringSerializer to your zkClient. ZkClient zkClient = new ZkClient(ZK_CONN_STRING); in Kafka TopicCommand uses "new ZkClient(opts.options.valueOf(opts.zkConnectOpt), 3, 3, ZKStringSerializer)" because of this although your topic is getting created and

Re: Create topic, then fetch offset

2015-01-13 Thread Harsha
You can import ZKStringSerializer from kafka.utils.ZkClient or write your own similar string serializer like this https://gist.github.com/harshach/7b5447c39168eb6062e0 On Tue, Jan 13, 2015, at 04:31 PM, Harsha wrote: > It looks like you are not adding a ZkStringSerializer to your zkCli

Re: Create topic, then fetch offset

2015-01-13 Thread Harsha
Internally producer sends a TopicMetadataRequest which creates the topic if auto.create.topics.enable is true. On Tue, Jan 13, 2015, at 04:49 PM, Harsha wrote: > > You can import ZKStringSerializer from kafka.utils.ZkClient or write > your own similar string serializer like thi

Re: kafka brokers going down within 24 hrs

2015-01-14 Thread Harsha
Tousif, Do you see any other errors in server.log -Harsha On Wed, Jan 14, 2015, at 01:51 AM, Tousif wrote: > Hello, > > I have configured kafka nodes to run via supervisord and see following > exceptions > and eventually brokers going out of memory. i have given enou

Re: kafka brokers going down within 24 hrs

2015-01-15 Thread Harsha
s to see why its going down for the first time. -Harsha On Wed, Jan 14, 2015, at 10:50 PM, Tousif wrote: > Hello Chia-Chun Shih, > > There are multiple issues, > First thing is i don't see out of memory error every time and OOM happens > after supervisord keep retrying to

Re: kafka brokers going down within 24 hrs

2015-01-16 Thread Harsha
Tousif, I meant to say if kafka broker is going down often its better to analyze whats the root of cause of the crash. Using supervisord to monitor kafka broker is fine, sorry about the confusion. -Harsha On Fri, Jan 16, 2015, at 11:25 AM, Gwen Shapira wrote: > Th

Re: kafka topic not exist after creation with api

2015-01-23 Thread Harsha
Manu, Are you passing ZkStringSerializer to zkClient . ZkClient zkClient = new ZkClient(ZK_CONN_STRING,3, 3, new ZkStringSerializer()); AdminUtils.createTopic(zkClient, topic, 1, 1, props); -Harsha On Thu, Jan 22, 2015, at 09:44 PM, Manu Zhang wrote: > Hi all , > My appli

Re: Delete topic API in 0.8.2

2015-01-23 Thread Harsha
Sumit, You can use AdminUtils.deleteTopic(zkClient, topicName) . This will initiate the deleteTopic process by zookeeper notification to KafkaController.deleteTopicManager. It deletes log files along with zookeeper topic path as Timothy mentioned. -Harsha On Fri, Jan

Re: unable to delete topic with 0.8.2 rc2

2015-01-26 Thread Harsha
topic will be recreated. -Harsha On Sun, Jan 25, 2015, at 11:26 PM, Jason Rosenberg wrote: > cversion did change (incremented by 2) when I issue the delete command. > > From the logs on the conroller broker (also the leader for the topic), it > looks like the delete proceeds, and then th

Re: unable to delete topic with 0.8.2 rc2

2015-01-26 Thread Harsha
Jun, I made an attempt at fixing that issue as part of this JIRA https://issues.apache.org/jira/browse/KAFKA-1507 . As Jay pointed out there should be admin api if there is more info on this api I am interested in adding/fixing this issue. Thanks, Harsha On Mon, Jan 26, 2015, at 07

Re: create topic does not really executed successfully

2015-02-02 Thread Harsha
. -Harsha On Mon, Feb 2, 2015, at 07:03 PM, Xinyi Su wrote: > Hi, > > -bash-4.1$ bin/kafka-topics.sh --zookeeper :2181 --create > --topic > zerg.hydra --partitions 3 --replication-factor 2 > Created topic "zerg.hydra". > > -bash-4.1$ ls -lrt /tmp/kafka-logs/zerg.hydr

Re: create topic does not really executed successfully

2015-02-02 Thread Harsha
. -Harsha On Mon, Feb 2, 2015, at 07:03 PM, Xinyi Su wrote: > Hi, > > -bash-4.1$ bin/kafka-topics.sh --zookeeper :2181 --create > --topic > zerg.hydra --partitions 3 --replication-factor 2 > Created topic "zerg.hydra". > > -bash-4.1$ ls -lrt /tmp/kafka-logs/zerg.hydr

Re: Issue with topic deletion

2015-02-03 Thread Harsha
Sumit, lets say you are deleting a older topic "test1" do you have any consumers running simultaneously for the topic "test1" while deletion of topic going on. -Harsha On Tue, Feb 3, 2015, at 06:17 PM, Joel Koshy wrote: > Thanks for the logs - wi

Re: Issue with topic deletion

2015-02-03 Thread Harsha
you try stopping the consumer first and issue the topic delete. -Harsha On Tue, Feb 3, 2015, at 08:37 PM, Sumit Rangwala wrote: > On Tue, Feb 3, 2015 at 6:48 PM, Harsha wrote: > > > Sumit, > >lets say you are deleting a older topic "test1" do you have any

Re: Issue with topic deletion

2015-02-03 Thread Harsha
AX1-GRIFFIN-r45-142388317" which looks to be getting deleted properly. Do you see same issues with the above topic i.e /admin/delete_topics/LAX1-GRIFFIN-r45-142388317 still exists. If you can post the logs from 23:00 onwards that will be helpful. -Harsha On Tue, Feb 3, 2015, at 09:19 PM,

Re: Issue with topic deletion

2015-02-04 Thread Harsha
on" forever. However, there were many topics (at least 10 of > > them) > > > > that were perennially in "marked for deletion" state. > > > > > > > > > > > I have the setup to recreate the issue in case the logs are not > > sufficient

[DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-02-11 Thread Harsha
the work is in progress. I'll update the thread with a initial prototype patch. Thanks, Harsha

Fwd: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-02-11 Thread Harsha
the work is in progress. I'll update the thread with a initial prototype patch. Thanks, Harsha

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-02-11 Thread Harsha
Thanks Joe. It will be part of KafkaServer and will run on its own thread. Since each kafka server will run with a keytab we should make sure they are all getting renewed. On Wed, Feb 11, 2015, at 10:00 AM, Joe Stein wrote: > Thanks Harsha, looks good so far. How were you thinking of runn

Re: [DISCUSS] KIP-12 - Kafka Sasl/Kerberos implementation

2015-02-12 Thread Harsha
Thanks for the review Gwen. I'll keep in mind about java 6 support. -Harsha On Wed, Feb 11, 2015, at 03:28 PM, Gwen Shapira wrote: > Looks good. Thanks for working on this. > > One note, the Channel implementation from SSL only works on Java7 and up. > Since we are still suppo

Re: MetadataRequest vs Zookeeper

2015-02-13 Thread Harsha
Paul, There is ongoing work to move to Kafka API instead of making calls to zookeeper. Here is the JIRA https://issues.apache.org/jira/browse/STORM-650 . -Harsha On Fri, Feb 13, 2015, at 01:02 PM, Paul Mackles wrote: > I noticed that the standard Kafka storm spout gets to

Re: Kakfa question about starting kafka with same broker id

2015-02-18 Thread Harsha
n the path /brokers/ids/0. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering." -Harsha On Wed, Feb 18, 2015, at 03:16 PM, Dee

Re: Broker Server Crash with HW failure. Broker throwing java.lang.NumberFormatException and will not restart without removing all partitions

2015-02-24 Thread Harsha
Hi Gene, Looks like you might be running into this https://issues.apache.org/jira/browse/KAFKA-1758 . -Harsha On Tue, Feb 24, 2015, at 07:17 AM, Gene Robichaux wrote: > About a week ago one of our brokers crashed with a hardware failure. When > the server restarted the

Re: Batching with new Producer API

2015-02-26 Thread Harsha
Akshat, Produce.batch_size is in bytes and if your messages avg size is 310 bytes and your current number of messages per batch is 46 you are getting close to the max batch size 16384. Did you try increasing the producer batch_size bytes? -Harsha On Thu, Feb 26, 2015

Re: How replicas catch up the leader

2015-02-28 Thread Harsha
you can increase num.replica.fetchers by default its 1 and also try increasing replica.fetch.max.bytes -Harsha On Fri, Feb 27, 2015, at 11:15 PM, tao xiao wrote: > Hi team, > > I had a replica node that was shutdown improperly due to no disk space > left. I managed to clean up

Re: Delete Topic in 0.8.2

2015-03-01 Thread Harsha
Hi Hema, Can you attach controller.log and state-change.log. Image is not showing up at least for me. Can you also give us details on how big the cluster is and topic's partitions and replication-factor and any steps on reproducing this. Thanks, Harsha On Sun, Mar 1, 2015, at 12:40 PM,

Re: 0.7 design doc?

2015-03-02 Thread Harsha
These docs might help https://kafka.apache.org/08/design.html http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf -Harsha On Sun, Mar 1, 2015, at 09:42 PM, Philip O'Toole wrote: > Thanks Guozhang -- no this isn't quite it. The doc

Re: Problem deleting topics in 0.8.2?

2015-03-04 Thread Harsha
stopping any of your consumers or producers run the delete topic command again. -Harsha On Wed, Mar 4, 2015, at 10:28 AM, Jeff Schroeder wrote: > So I've got 3 kafka brokers that were started with delete.topic.enable > set > to true. When they start, I can see in the logs that th

Re: kafka topic information

2015-03-09 Thread Harsha
Yuheng,           kafka keeps cluster metadata in zookeeper along with topic metadata as well. You can use zookeeper-shell.sh or zkCli.sh to check zk nodes, /brokers/topics will give you the list of topics . --  Harsha On March 9, 2015 at 8:20:59 AM, Yuheng Du (yuheng.du.h...@gmail.com) wrote

Re: kafka topic information

2015-03-09 Thread Harsha
machine failure and incase of 5 there should be at least 3 nodes to be up and running.   For more info on zookeeper you can look under here  http://zookeeper.apache.org/doc/r3.4.6/ http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html --  Harsha On March 9, 2015 at 8:39:00 AM, Yuheng Du

Re: Kafka-Storm: troubleshooting low R/W throughput

2015-03-22 Thread Harsha
Hi Emmanuel,        Can you post your kafka server.properties and in your producer are your distributing your messages into all kafka topic partitions. --  Harsha On March 20, 2015 at 12:33:02 PM, Emmanuel (ele...@msn.com) wrote: Kafka on test cluster:  2 Kafka nodes, 2GB, 2CPUs 3 Zookeeper

Re: Check topic exists after deleting it.

2015-03-23 Thread Harsha
node gets deleted as well. Also make sure you don’t have any producers or consumers are running while the topic deleting is going on. --  Harsha On March 23, 2015 at 1:29:50 AM, anthony musyoki (anthony.musy...@gmail.com) wrote: On deleting a topic via TopicCommand.deleteTopic() I get "

Re: Check topic exists after deleting it.

2015-03-23 Thread Harsha
. --  Harsha On March 23, 2015 at 9:57:53 AM, Grant Henke (ghe...@cloudera.com) wrote: What happens when producers or consumers are running while the topic deleting is going on? On Mon, Mar 23, 2015 at 10:02 AM, Harsha wrote: > DeleteTopic makes a node in zookeeper to let controller k

Re: Check topic exists after deleting it.

2015-03-23 Thread Harsha
Just to be clear, one needs to stop producers and consumers that writing/reading from a topic “test” if they are trying to delete that specific topic “test”. Not all producers and clients. --  Harsha On March 23, 2015 at 10:13:47 AM, Harsha (harsh...@fastmail.fm) wrote: Currently we have

Re: KafkaSpout forceFromStart Issue

2015-03-23 Thread Harsha
Hi Francois,            Looks like this belong storm mailing lists. Can you please send this question on storm mailing lists. Thanks, Harsha On March 23, 2015 at 11:17:47 AM, François Méthot (fmetho...@gmail.com) wrote: Hi, We have a storm topology that uses Kafka to read a topic with 6

Re: lost messages -?

2015-03-26 Thread Harsha
Victor,         Its under kaka.tools.DumpLogSegments you can use kafka-run-class to execute it. --  Harsha On March 26, 2015 at 5:29:32 AM, Victor L (vlyamt...@gmail.com) wrote: Where's this tool (DumpLogSegments) in Kafka distro? Is it Java class in kafka jar, or is it third party b

Re: Kafka Consumer

2015-03-31 Thread Harsha
o one consumer instance within each subscribing consumer group. Consumer instances can be in separate processes or on separate machines.” More info on this page http://kafka.apache.org/documentation.html look for "consumer group”.  --  Harsha On March 31, 2015 at 6:10:59 AM, James King

Re: Per-topic retention.bytes uses kilobytes not bytes?

2015-04-02 Thread Harsha
log size to 200 bytes. --  Harsha On April 2, 2015 at 2:29:45 PM, Willy Hoang (w...@knewton.com) wrote: Hello, I’ve been having trouble using the retention.bytes per-topic configuration (using Kafka version 0.8.2.1). I had the same issue that users described in these two threads where logs

Re: Increasing replication factor of existing topics

2015-04-07 Thread Harsha
Hi Navneet,           Any reason that you are looking to modify the zk nodes directly to increase the topic partition. If you are looking for an api to do this there is AdminUtils.addPartitions .  --  Harsha On April 7, 2015 at 6:45:40 AM, Navneet Gupta (Tech - BLR) (navneet.gu

Re: Failure in Leader Election on broker shutdown

2015-06-19 Thread Harsha
Sandeep,         You need to have multiple replicas. Having single replica means you've one copy of the data and if that machine goes down there isn't another replica who can take over and be the leader for that partition.-Harsha  _ From: Sandeep Bis

EOL JDK 1.6 for Kafka

2015-07-01 Thread Harsha
potential MITM. This api doesn't exist in Java 1.6. Are there any users still want 1.6 support or can we stop supporting 1.6 from next release on wards. Thanks, Harsha

Re: Kafka Broker process: Connection refused

2015-10-04 Thread Harsha
Mark,       Can you try running the /usr/hdp/current/kafka- broker/bin/kafka start manually as user Kafka and also do you see jars under kafka-broker/libs/kafka_*.jar . Can you try asking the question here as well http://hortonworks.com/community/forums/forum/kafka/ Thanks, Harsha On Sun, Oct 4

Re: kafka 0.8.2 build problem

2015-11-15 Thread Harsha
Can you try using the 0.8.2.2 source here https://www.apache.org/dyn/closer.cgi?path=/kafka/0.8.2.2/kafka-0.8.2.2-src.tgz https://kafka.apache.org/downloads.html On Sun, Nov 15, 2015, at 05:35 AM, jinxing wrote: > Hi all, I am newbie for kafka, and I try to build kafka 0.8.2 from source > code as

Re: The JIRA Awakens [KAFKA-1841]

2015-12-24 Thread Harsha
kafka 0.9.0 + additional patches . We are making sure on our side not to miss any compatibility patches like these with 3rd party developers and have tests to ensure that. Thanks, Harsha On Wed, Dec 23, 2015, at 04:11 PM, Dana Powers wrote: > Hi all, > > I've been helping debug

Re: Consumer - Failed to find leader

2015-12-29 Thread Harsha
nnection to Zookeeper server without SASL authentication, if Zookeeper" did you try kinit with that keytab at the command line. -Harsha On Mon, Dec 28, 2015, at 04:07 AM, prabhu v wrote: > Thanks for the input Ismael. > > I will try and let you know. > > Also need your v

Re: Consumer - Failed to find leader

2015-12-30 Thread Harsha
/keytabs/kafka1.keytab" principal="kafka/kafka1.witzend@witzend.com"; }; and KafkaClient KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true serviceName="kafka"; }; On Wed, Dec 30, 2015, at 03:10 AM, prabhu v wrote: > Hi Harsha,

Re: behavior of two live brokers with the same id?

2016-02-04 Thread Harsha
You won't be able to start and register to brokers with same id in zookeeper. On Thu, Feb 4, 2016, at 06:26 AM, Carlo Cabanilla wrote: > Hello, > > How does a kafka cluster behave if there are two live brokers with the > same > broker id, particularly if that broker is a leader? Is it determinis

Re: 答复: NoAuth for /controller

2016-02-16 Thread Harsha
Kafka doesn't have security enabled for 0.8.2.2 so make sure zookeeper root that you're using doesn't have any acls set -Harsha On Sun, Feb 14, 2016, at 06:51 PM, 赵景波 wrote: > Can you help me? > > ___ > J

Re: Java Client connection errors with Kafka 0.9.0.0 when SSL is enabled

2016-02-18 Thread Harsha
Did you try what Adam is suggesting in the earlier email. Also to quickly check you can try remove keystore and key.password configs from client side. -Harsha On Thu, Feb 18, 2016, at 02:49 PM, Srikrishna Alla wrote: > Hi, > > We are getting the below error when trying to use a Java new

Re: Kerberized Kafka setup issues

2016-02-23 Thread Harsha
Oleg, Can you post your jaas configs. Its important that serviceName must match the principal name with which zookeeper is running. Whats the principal name zookeeper service is running with. -Harsha On Tue, Feb 23, 2016, at 11:01 AM, Oleg Zhurakousky wrote: > Hey g

Re: Kerberized Kafka setup issues

2016-02-23 Thread Harsha
ssuming Zookeeper is started as ‘zookeeper’ > and Kafka as ‘kafka’ > > Oleg > > > On Feb 23, 2016, at 2:22 PM, Oleg Zhurakousky > > wrote: > > > > Harsha > > > > Thanks for following up. Here is is

Re: Kerberized Kafka setup issues

2016-02-23 Thread Harsha
principal="kafka/ubuntu.oleg@oleg.com"; }; -Harsha On Tue, Feb 23, 2016, at 11:37 AM, Harsha wrote: > can you try adding "serviceName=zookeeper" to KafkaServer section like > KafkaServer { > com.sun.security.auth.module.Krb5LoginModule required >

Re: Kerberized Kafka setup issues

2016-02-23 Thread Harsha
whats your zookeeper.connect in server.properties looks like. Did you use the hostname or localhost -Harsha On Tue, Feb 23, 2016, at 12:01 PM, Oleg Zhurakousky wrote: > Still digging, but here is more info that may help > > 2016-02-23 14:59:24,240] INFO zookeeper state changed (Sync

Re: Kafka 0.9 Spout

2016-03-01 Thread Harsha
hanks, Harsha On Tue, Mar 1, 2016, at 12:56 PM, Ankit Jain wrote: > Hi All, > > We would need to use the SSL feature of Kafka and that would require the > Kafka Spout upgrade from version 0.8.x to 0.9.x as the SSL is only > supported by new consumer API. > > Please

Re: Kafka Error while doing SSL authentication on 0.9.0

2016-03-15 Thread Harsha
/documentation.html#security_ssl -Harsha On Thu, Mar 10, 2016, at 12:41 AM, Ranjan Baral wrote: > i getting below warning while doing produce from client side which is > connecting to server side which contains SSL based authentication. > > *[2016-03-10 07:09:13,018] WARN The c

Re: Kafka Newbie question

2016-04-10 Thread Harsha
Pradeep, How about https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#seekToBeginning%28org.apache.kafka.common.TopicPartition...%29 -Harsha On Sat, Apr 9, 2016, at 09:48 PM, Pradeep Bhattiprolu wrote: > Liquan , thanks for

Re: Reg slack channel

2016-04-26 Thread Harsha
We use apache JIRA and mailing lists for any discussion. Thanks, Harsha On Tue, Apr 26, 2016, at 06:20 PM, Kanagha wrote: > Hi, > > Is there a slack channel for discussing issues related to Kafka? I would > like to get an invite. Thanks! > > > -- > Kanagha

Re: Not able to monitor consumer group lag with new consumer and kerberos

2016-06-07 Thread Harsha
Hi Pierre, Do you see any errors in the server.log when this command ran. Can you please open a thread here https://community.hortonworks.com/answers/index.html . Thanks, Harsha On Tue, Jun 7, 2016, at 09:22 AM, Pierre Labiausse wrote: > Hi, >

Re: Questions on Kafka Security

2016-06-08 Thread Harsha
ions? openSSL is not supported yet. Also dropping the encryption in SSL channel is not possible yet. Any reason for not use kerberos for this since we support non-encrypted channel for kerberos. Thanks, harsha On Wed, Jun 8, 2016, at 02:06 PM, Samir Shah wrote: > Hello, > > Few question

Re: [DISCUSS] Java 8 as a minimum requirement

2016-06-16 Thread Harsha
the dates . Thanks, Harsha On Thu, Jun 16, 2016, at 03:08 PM, Ismael Juma wrote: > Hi Jan, > > That's interesting. Do you have some references you can share on this? It > would be good to know which Java 8 versions have been tested and whether > it > is something

Re: [DISCUSS] Java 8 as a minimum requirement

2016-06-17 Thread Harsha
ersion. Removing Java 7 support when the release is minor in general not a good idea to users. -Harsha "Also note that Kafka bug fixes go to 0.10.0.1, not 0.10.1 and > 0.10.0.x would still be available for users using Java 7." On Fri, Jun 17, 2016, at 12:18 AM, Ismael Juma wrote: &

Re: [DISCUSS] Java 8 as a minimum requirement

2016-06-19 Thread Harsha
ntain lot of features . If we can make this as part of 0.11 and cutting 0.10.1 features moving to 0.11 and giving rough timeline when that would be released would be ideal. Thanks, Harsha On Fri, Jun 17, 2016, at 11:13 AM, Ismael Juma wrote: > H

Re: SSL support for command line tools

2016-06-22 Thread Harsha
Radu, Please follow the instructions here http://kafka.apache.org/documentation.html#security_ssl . At the end of the SSL section we've an example for produce and consumer command line tools to pass in ssl configs. Thanks, Harsha On Wed, Jun 22, 2016, at

Re: ZkUtils creating parent path with different ACL

2016-07-01 Thread Harsha
If i remember correctly it was because we wanted to allow non-secure client still get into child consumers node and create their zookeeper nodes to keep track of offset. If we add the acl at the parent path they won't be able to write to the child nodes. Thanks, Harsha On Fri, Jul 1, 2016,

Re: [VOTE] 1.1.1 RC1

2018-06-22 Thread Harsha
+1 .   1. Ran unit tests.   2. Ran few tests on 3-node cluster Thanks, Harsha On Fri, Jun 22nd, 2018 at 2:41 PM Jakob Homan wrote: > > > > +1 (binding) > > * verified sigs, NOTICE, LICENSE > * ran unit tests > * spot checked headers > > -Jakob > > &

Re: [VOTE] 0.11.0.3 RC0

2018-06-24 Thread Harsha
+1 * Ran Unit tests * 3 node cluster . Ran simple tests. Thanks, Harsha On Sat, Jun 23rd, 2018 at 9:7 AM, Ted Yu wrote: > > > > +1 > > Checked signatures. > > Ran unit test suite. > > On Fri, Jun 22, 2018 at 4:56 PM, Vahid S Hashemian < > vahidhas

Re: [kafka-clients] [VOTE] 1.0.2 RC1

2018-07-02 Thread Harsha
+1.      1) Ran unit tests 2) 3 node cluster , tested basic operations. Thanks, Harsha On Mon, Jul 2nd, 2018 at 11:57 AM, Jun Rao wrote: > > > > Hi, Matthias, > > Thanks for the running the release. Verified quickstart on scala 2.12 > binary. +1 > > Jun > &

Re: [VOTE] 2.0.0 RC1

2018-07-02 Thread Harsha
+1.  1) Ran unit tests  2) 3 node cluster , tested basic operations.  Thanks, Harsha On Mon, Jul 2nd, 2018 at 11:13 AM, "Vahid S Hashemian" wrote: > > > > +1 (non-binding) > > Built from source and ran quickstart successfully on Ubuntu (with Java 8). >

Re: [VOTE] 1.1.1 RC3

2018-07-09 Thread Harsha
+1. * Ran unit tests * Installed in a cluster and ran simple tests Thanks, Harsha On Mon, Jul 9th, 2018 at 6:38 AM, Ted Yu wrote: > > > > +1 > > Ran test suite. > > Checked signatures. > > > > On Sun, Jul 8, 2018 at 3:36 PM Dong Lin < lindon..

Re: Restrict access on kafka with multiple listener

2018-07-17 Thread Harsha
read/writes on the secure topics and it will reject any request on PLAINTEXT port for these topics as AuthorizationException and rest of the topics you can continue access through both the ports. -Harsha On Tue, Jul 17, 2018, at 5:09 PM, Matt L wrote: > Hi, > > I have an existing Kafk

Re: Performance Impact with Apache Kafka Security

2018-08-23 Thread Harsha
Hi, Which Kafka version and Java version are you using? Did you try this with Java 9 which has 2.5x perf improvements over Java 8 for SSL? Can you try using a slightly weaker cipher suite to improve the performance? -Harsha On Wed, Aug 22, 2018, at 1:11 PM, Sri Harsha Chavali wrote: >

Re: [DISCUSS] KIP-369 Alternative Partitioner to Support "Always Round-Robin" Selection

2018-08-30 Thread Harsha
a#L28 . Use your custom partitioner to be configured in your producer clients. Thanks, Harsha On Thu, Aug 30, 2018, at 1:45 AM, M. Manna wrote: > Hello, > > I opened a very simple KIP and there exists a JIRA for it. > > I would be grateful if any comments are available for action. > > Regards,

Re: plaintext connection attempts to SSL secured broker

2019-04-04 Thread Harsha
Hi, Yes, this needs to be handled more elegantly. Can you please file a JIRA here https://issues.apache.org/jira/projects/KAFKA/issues Thanks, Harsha On Mon, Apr 1, 2019, at 1:52 AM, jorg.heym...@gmail.com wrote: > Hi, > > We have our brokers secured with these standard p

Re: [ANNOUNCE] New Kafka PMC member: Sriharsh Chintalapan

2019-04-19 Thread Harsha
Thanks, Everyone. -Harsha On Fri, Apr 19, 2019, at 2:39 AM, Satish Duggana wrote: > Congrats Harsha! > > On Fri, Apr 19, 2019 at 2:58 PM Mickael Maison > wrote: > > > Congratulations Harsha! > > > > > > On Fri, Apr 19, 2019 at 5:49 AM Manikumar

Re: Question about Kafka TLS

2019-05-16 Thread Harsha
certificate to enable TLS. JKS stores are for doing it manually. You can check out https://github.com/spiffe/java-spiffe which talks spiffee agent to get a certificate and pass it to Kafka's SSL context. Thanks, Harsha On Thu, May 16, 2019, at 3:57 PM, Zhou, Thomas wrote: > Hi, >

Re: [VOTE] 2.2.1 RC1

2019-05-23 Thread Harsha
+1 (binding) 1. Ran unit tests 2. System tests 3. 3 node cluster with few manual tests. Thanks, Harsha On Wed, May 22, 2019, at 8:09 PM, Vahid Hashemian wrote: > Bumping this thread to get some more votes, especially from committers, so > we can hopefully make a decision on this RC by t

Re: [ANNOUNCE] Apache Kafka 2.2.1

2019-06-03 Thread Harsha
Thanks Vahid. -Harsha On Mon, Jun 3, 2019, at 9:21 AM, Jonathan Santilli wrote: > That's fantastic! thanks a lot Vahid for managing the release. > > -- > Jonathan > > > > > On Mon, Jun 3, 2019 at 5:18 PM Mickael Maison > wrote: > > > Thank you

Re: TLS Communication in With Zookeeper Cluster

2019-07-29 Thread Harsha
Here is the guide https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide you need zookeeper 3.5 or higher for TLS. On Mon, Jul 29, 2019, at 1:21 AM, Nayak, Soumya R. wrote: > Hi Team, > > Is there any way mutual TLS communication set up can be done with > zookeeper. If a

Re: Securing Multi-Node single broker kafka instance

2017-03-01 Thread Harsha
factor and see if you can produce & consume messages Before stepping into security make sure your non-secure Kafka cluster works ok. Once you’ve a stable & working cluster follow instructions in the doc to enable SSL. -Harsha On Mar 1, 2017, 1:08 PM -0800, IT Consultant <0binarybudd..

Re: confirm subscribe to users@kafka.apache.org

2014-04-17 Thread Harsha
that you would like > >ka...@harsha.io > > added to the users mailing list, please send > a short reply to this address: > >users-sc.1397781463.cnligeeapcohnilolgla-kafka=harsha...@kafka.apache.org > > Usually, this happens when you just hit the "reply" b

Re: Changing the Replication Factor

2014-07-31 Thread Harsha
Hi Sagar, Its not there in release versions or trunk . Here is the jira for it https://issues.apache.org/jira/browse/KAFKA-1543. -Harsha On Thu, Jul 31, 2014, at 05:22 AM, Sagar Khanna wrote: > Hi, > > >>> re: > >>> http://grokbase.com/t/kafka

  1   2   >