Hi Jun,
Please ignore the earlier mail.
I have a Kafka Cluster running on AWS machines i.e. 3 Zookeeper and 2
Brokers. I am trying to scale the brokers while producing and consuming
messages.
I am sending 1000 messages by batch producer every second for 200 seconds
for the following topics:
topics : Topic1(20 partitions) Topic2(40 partitions)
Replication Factor.
Add 1 broker:
Produced Consumed
Topic1 200000 199950
Topic2 200000 199975
Remove 1 broker:
Produced Consumed
Topic1 200000 199950
Topic2 200000 199825
I run a script which reassigns the partitions. And as it can be seen from
the above numbers that we see data loss at the consumer end.
Let me know if you need any further details.
Regards,
Vinayak
On 29 May 2017 at 13:03, Sanket Dhopeshwarkar <san...@amagi.com> wrote:
> +Vinayak
>
> On 26 May 2017 at 21:44, Jun Rao <j...@confluent.io> wrote:
>
>> Hi, Vinayak,
>>
>> I am not sure that KAFKA-1561 as reported is an issue. Could you describe
>> the scenario in your test in that jira in a bit more detail?
>>
>> Thanks,
>>
>> Jun
>>
>> On Fri, May 26, 2017 at 5:58 AM, Vinayak Sharma <vinaya...@amagi.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I am experiencing loss of messages while scaling brokers up and down in
>>> kafka cluster.
>>>
>>> I came across this jira issue(link
>>> <https://issues.apache.org/jira/browse/KAFKA-1561>). Can you confirm if
>>> this is actually an issue in kafka or can the problem be solved by changing
>>> some configuration parameters.
>>>
>>> I am enclosing the server.properties that I am using.
>>>
>>> Please get to back to me.
>>>
>>> Regards,
>>> Vinayak.
>>>
>>> <http://www.amagi.com>
>>
>>
>>
>
--
<http://www.amagi.com>
