moving brokers to different servers
Hi My brokers are currently installed in servers that's end of life. What is the recommended way of migrating them over to new servers? Thanks Printing e-mails wastes valuable natural resources. Please don't print this message unless it is absolutely necessary. Thank you for thinking green! Sent from my iPhone
accessing Kafka broker 0.9.0.1 with Kafka-client 1.0.0
Hi, I am trying to access my Kafka brokers using springbok 2.0 which is using Kafka-client 1.0.0 and it seems like running into timeout exceptions waiting for metadata update. Not sure if we can use 1.0.0 client for my 0.9.0.1 broker - anyone has thoughts on this? Thanks.
kafka 0.9.x : creating topic with delete.retention.ms is faling
with error below: UnrecognizedOptionException: 'delete.retention.ms' is not a recognized option I looked at the documentation for topics and seems like a valid option. I am trying to override this setting on the topic level. Appreciate some clarifications on this. Thanks, Nomar
dynamically changing peer role from voting to observer and vice versa
Is there any way in the current API to achieve this? I am trying to set up some screen to be able to promote an observer to voting member in case of a loss of quorum for instance. Appreciate your help. thanks, Nomar
Re: dynamically changing peer role from voting to observer and vice versa
Anyone? Printing e-mails wastes valuable natural resources. Please don't print this message unless it is absolutely necessary. Thank you for thinking green! Sent from my iPhone > On Jun 8, 2016, at 2:31 PM, Nomar Morado wrote: > > Is there any way in the current API to achieve this? > > I am trying to set up some screen to be able to promote an observer to voting > member in case of a loss of quorum for instance. > > Appreciate your help. > > > thanks, > Nomar
using ZK 3.5.x in 0.9.0.1?
Hi - does anyone know if this is compatible? we like to use ZK's dynamic config but i think it is not available until 3.5.x Thanks.
Intermittent runtime exception: broker already registered
I am using Kafka 0.9.0.1 with ZK 3.5.0-alpha I am seeing this error intermittently which goes away after several reboots. Any ideas? Sent from my iPad
partition running out of disk space
Hi What is kaka's behavior if the partition it is running runs out of disk space? Do producers get error? Does the entire kafka stop running? etc... Appreciate your help. Thanks.
loading 15M messages to kafka
Hi I am trying to load 15 million rows/records/messages to Kafka and looking for the most optimal way of accomplishing this. I can go through client API but was wondering if there's a more efficient way of doing this. Thanks. Nomar
automatically creating a new partition for topic
Hi Is there a way to send message to broker to a new partition with the assumption that the broker will create new partition automatically? (almost like auto.create.topics.enable property) thanks
kafka client on cloud
kafka consumer/producer currently require path to keystore/truststore. my client runs in cloud and won't have access to some actual path to my jks. any ideas on the best way to handle this? thanks.
a broker is already registered on path /brokers/ids/1
we would get this occasionally after a weekend reboot/restart. we tried restarting a couple of times all to naught. we had to delete dk's directory to get his going again. any ideas what might cause this issue and suggestions on how to resolve this? thanks.
Re: a broker is already registered on path /brokers/ids/1
we are using kafka 0.9.0.1 and zk 3.5.0-alpha On Mon, Aug 29, 2016 at 8:12 AM, Nomar Morado wrote: > we would get this occasionally after a weekend reboot/restart. > > we tried restarting a couple of times all to naught. > > we had to delete dk's directory to get his going again. > > any ideas what might cause this issue and suggestions on how to resolve > this? > > > thanks. > -- Regards, Nomar Morado
Re: a broker is already registered on path /brokers/ids/1
Kafka broker in my case
Printing e-mails wastes valuable natural resources. Please don't print this
message unless it is absolutely necessary. Thank you for thinking green!
Sent from my iPhone
> On Aug 30, 2016, at 1:19 PM, Flavio Junqueira wrote:
>
> Is the zookeeper process you're referring to the server or the client (broker
> in the case of Kafka)?
>
> If you're referring to the server, then the ensemble recovers from the disk
> state of the servers. If it is a follower, then there is nothing to recover,
> the leader already has all the necessary data. If it is the leader, then it
> will get the ephemeral/session data from disk.
>
> If you're referring to the Kafka broker, then you should be seeing a message
> like this in your log in the case the broker finds a znode upon registration:
>
> case e: ZkNodeExistsException =>
> throw new RuntimeException("A broker is already registered on the path " +
> brokerIdPath
> + ". This probably " + "indicates that you either have configured a
> brokerid that is already in use, or "
> + "else you have shutdown this broker and restarted it faster than
> the zookeeper "
> + "timeout so it appears to be re-registering.")
>
>
> Does it help?
>
> -Flavio
>
>
>> On 30 Aug 2016, at 14:35, J316 Services wrote:
>>
>> Setup is 3 participants + 1 observer.
>>
>> We used 3.5.0 for dynamic configs.
>>
>> What happens when a system forced killed the zookeeper process and it did
>> not have time to clean up ephemeral? Does it recover in that scenario?
>>
>>
>> Thanks.
>>
>> Sent from my iPad
>>
>>> On Aug 30, 2016, at 8:53 AM, Flavio Junqueira wrote:
>>>
>>> I think you're saying that the session isn't expiring that and the
>>> ephemeral isn't getting deleted. Or maybe the session is expiring but the
>>> ephemeral isn't being deleted? It'd be great if you could check that the
>>> broker session is expiring eventually.
>>>
>>> Since you're on the 3.5 branch, you may want to use 3.5.2-alpha or simply
>>> try 3.4.8 as Harsha suggested.
>>>
>>> -Flavio
>>>
>>>> On 29 Aug 2016, at 18:44, Harsha Chintalapani wrote:
>>>>
>>>> how many brokers you've in this cluster. Do you try using a stable
>>>> zookeeper release like 3.4.8?
>>>> -Harhsa
>>>>
>>>>> On Mon, Aug 29, 2016 at 5:21 AM Nomar Morado
>>>>> wrote:
>>>>>
>>>>> we are using kafka 0.9.0.1 and zk 3.5.0-alpha
>>>>>
>>>>> On Mon, Aug 29, 2016 at 8:12 AM, Nomar Morado
>>>>> wrote:
>>>>>
>>>>>> we would get this occasionally after a weekend reboot/restart.
>>>>>>
>>>>>> we tried restarting a couple of times all to naught.
>>>>>>
>>>>>> we had to delete dk's directory to get his going again.
>>>>>>
>>>>>> any ideas what might cause this issue and suggestions on how to resolve
>>>>>> this?
>>>>>>
>>>>>>
>>>>>> thanks.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Nomar Morado
>>>
>
Kafka broker went down with "No space left on device" when there is a lot more
Two of my brokers went down today with the same error - see attachment for details. The device though is 55% free which is about over 100GB in space. The entire kafka logs is only 1.3GB. Any thoughts on what might be tripping this one? I am using kafka 0.9.0.1. Thanks Nomar
