Lots of warns about LogCleaningPaused during partition reassignment

2017-11-28 Thread BGCH

Hello!

We tried to migrate data from 0.10.2.1 cluster to 0.11.0.2. Firstly we 
spread topics to both clusters. There were lots of problems and restarts 
of some nodes of both clusters (we probably shouldn't do that). All this 
ended up with a state when we had lots of exceptions from 2 nodes of 
0.10 cluster:
/java.lang.IllegalStateException: Compaction for partition topic_name-7 
cannot be aborted and paused since it is in LogCleaningPaused state./


and the whole reassignment process stuck.

I looked through the source code of LogManager and found KAFKA-3123 
 which may be the 
cause. I restarted that 2 nodes and reassignment proceeded, but now I have /
[ReplicaFetcherThread-0-1028], Error for partition [topic_name,33] to 
broker 
1028:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: 
This server does not host this topic-partition./


My guess is that Kafka could not roll out new segments of logs due to 
LogCleaningPaused exception and after restart some segments lost. Is 
this correct?


Is there any other possible cause of that /LogCleaningPaused /exception? 
And the main question is how to prevent all this?

//


--
hivehome.com 



Hive | London | Cambridge | Houston | Toronto
The information contained in or attached to this email is confidential and 
intended only for the use of the individual(s) to which it is addressed. It 
may contain information which is confidential and/or covered by legal 
professional or other privilege. The views expressed in this email are not 
necessarily the views of Centrica plc, and the company, its directors, 
officers or employees make no representation or accept any liability for 
their accuracy or completeness unless expressly stated to the contrary. 
Centrica Connected Home Limited (company no: 5782908), registered in 
England and Wales with its registered office at Millstream, Maidenhead 
Road, Windsor, Berkshire SL4 5GD.


Kafka client and ELB (round-robin DNS)

2017-03-03 Thread BGCH

Hi all,

briefly about the problem, if bootstrap.servers is a domain name and 
that domain name eventually starts pointing to different IP, kafka 
client will never try to re-resolve the new IP.


In our project we use java-based Kafka client 0.9. We configure 
bootstrap.servers  with the domain name, say kafka.ourvpc.io, which is 
managed by ELB (in our case it's just a round-robin DNS).
If we have two Kafka clusters and the first of them is killed after 
application is started, the application never reconnects to the second. 
Kafka client instantly tries to connect to the first IP despite the fact 
that DNS points to another IP.


Is this a know issue?  Are there any workarounds?

Thanks.


--
hivehome.com 



Hive | London | Cambridge | Houston | Toronto
The information contained in or attached to this email is confidential and 
intended only for the use of the individual(s) to which it is addressed. It 
may contain information which is confidential and/or covered by legal 
professional or other privilege. The views expressed in this email are not 
necessarily the views of Centrica plc, and the company, its directors, 
officers or employees make no representation or accept any liability for 
their accuracy or completeness unless expressly stated to the contrary. 
Hive is the trading name of Centrica Connected Home Limited (company no: 
5782908), registered in England and Wales with its registered office at 
Millstream, Maidenhead Road, Windsor, Berkshire SL4 5GD.