[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14506527#comment-14506527 ] Dmitry Bugaychenko commented on KAFKA-2029: --- The problem is that the state is di

[jira] [Issue Comment Deleted] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Comment: was deleted (was: The problem is that the state is distributed - not only contr

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14506528#comment-14506528 ] Dmitry Bugaychenko commented on KAFKA-2029: --- The problem is that the state is di

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14481477#comment-14481477 ] Dmitry Bugaychenko commented on KAFKA-2029: --- Rebased on trunk. > Improving cont

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14481465#comment-14481465 ] Dmitry Bugaychenko commented on KAFKA-2029: --- Created reviewboard https://reviews

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Attachment: KAFKA-2029.patch > Improving controlled shutdown for rolling updates > -

[jira] [Issue Comment Deleted] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Comment: was deleted (was: Created reviewboard against branch 0.8.1) > Improving contr

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Attachment: (was: KAFKA-2029.patch) > Improving controlled shutdown for rolling upda

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381538#comment-14381538 ] Dmitry Bugaychenko commented on KAFKA-2029: --- Created reviewboard against branch

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Attachment: KAFKA-2029.patch > Improving controlled shutdown for rolling updates > -

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381531#comment-14381531 ] Dmitry Bugaychenko commented on KAFKA-2029: --- Created reviewboard against branch

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Attachment: KAFKA-2029.patch > Improving controlled shutdown for rolling updates > -

[jira] [Comment Edited] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378433#comment-14378433 ] Dmitry Bugaychenko edited comment on KAFKA-2029 at 3/24/15 7:35 PM:

[jira] [Comment Edited] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378433#comment-14378433 ] Dmitry Bugaychenko edited comment on KAFKA-2029 at 3/24/15 7:34 PM:

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378433#comment-14378433 ] Dmitry Bugaychenko commented on KAFKA-2029: --- We tried prioritization of controll

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371844#comment-14371844 ] Dmitry Bugaychenko commented on KAFKA-2029: --- You right about timeout - it was a

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Description: Controlled shutdown as implemented currently can cause numerous problems:

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Description: Controlled shutdown as implemented currently can cause numerous problems:

[jira] [Created] (KAFKA-2029) Improving controlled shutdown for rolling updates

Dmitry Bugaychenko created KAFKA-2029: - Summary: Improving controlled shutdown for rolling updates Key: KAFKA-2029 URL: https://issues.apache.org/jira/browse/KAFKA-2029 Project: Kafka Iss

[jira] [Commented] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363129#comment-14363129 ] Dmitry Bugaychenko commented on KAFKA-1305: --- Even with a fast dedicated channel

[jira] [Commented] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361619#comment-14361619 ] Dmitry Bugaychenko commented on KAFKA-1305: --- Retry itself is in the cahnnel mana

[jira] [Commented] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361155#comment-14361155 ] Dmitry Bugaychenko commented on KAFKA-1305: --- Yes, data loss is tolerated to some

[jira] [Commented] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14360253#comment-14360253 ] Dmitry Bugaychenko commented on KAFKA-1305: --- With a large controller queues size

[jira] [Commented] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

[ https://issues.apache.org/jira/browse/KAFKA-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313765#comment-14313765 ] Dmitry Bugaychenko commented on KAFKA-1408: --- When topic deletion is disabled the

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074119#comment-14074119 ] Dmitry Bugaychenko commented on KAFKA-1414: --- I'm not a scala expert, but the thr

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070642#comment-14070642 ] Dmitry Bugaychenko commented on KAFKA-1414: --- "data dir" and "log dir" seems more

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070555#comment-14070555 ] Dmitry Bugaychenko commented on KAFKA-1414: --- We would prefer the second approach

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070274#comment-14070274 ] Dmitry Bugaychenko commented on KAFKA-1539: --- With fileOutputStream.getFD.sync()

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069899#comment-14069899 ] Dmitry Bugaychenko commented on KAFKA-1539: --- Going to test power failure again l

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068333#comment-14068333 ] Dmitry Bugaychenko commented on KAFKA-1414: --- According our experience in operati

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068020#comment-14068020 ] Dmitry Bugaychenko commented on KAFKA-1414: --- Separate thread pool per each dir w

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068015#comment-14068015 ] Dmitry Bugaychenko commented on KAFKA-1414: --- That is why in my initial proposal

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068013#comment-14068013 ] Dmitry Bugaychenko commented on KAFKA-1414: --- In our use case inter-directory par

[jira] [Comment Edited] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068010#comment-14068010 ] Dmitry Bugaychenko edited comment on KAFKA-1539 at 7/20/14 6:30 PM:

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068010#comment-14068010 ] Dmitry Bugaychenko commented on KAFKA-1539: --- Digged the proble a bit more. It lo

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061731#comment-14061731 ] Dmitry Bugaychenko commented on KAFKA-1414: --- Any plans regarding parallel shutdo

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061724#comment-14061724 ] Dmitry Bugaychenko commented on KAFKA-1539: --- It looks like even after flush data

[jira] [Comment Edited] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060456#comment-14060456 ] Dmitry Bugaychenko edited comment on KAFKA-1539 at 7/14/14 8:58 AM:

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060456#comment-14060456 ] Dmitry Bugaychenko commented on KAFKA-1539: --- This is not about log files themsel

[jira] [Created] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

Dmitry Bugaychenko created KAFKA-1539: - Summary: Due to OS caching Kafka might loose offset files which causes full reset of data Key: KAFKA-1539 URL: https://issues.apache.org/jira/browse/KAFKA-1539

[jira] [Commented] (KAFKA-1429) Yet another deadlock in controller shutdown

[ https://issues.apache.org/jira/browse/KAFKA-1429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985300#comment-13985300 ] Dmitry Bugaychenko commented on KAFKA-1429: --- Yes, we have auto.leader.rebalance.

[jira] [Commented] (KAFKA-1429) Yet another deadlock in controller shutdown

[ https://issues.apache.org/jira/browse/KAFKA-1429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13984295#comment-13984295 ] Dmitry Bugaychenko commented on KAFKA-1429: --- Updated libs to 0.8.1.1, will see h

[jira] [Created] (KAFKA-1429) Yet another deadlock in controller shutdown

Dmitry Bugaychenko created KAFKA-1429: - Summary: Yet another deadlock in controller shutdown Key: KAFKA-1429 URL: https://issues.apache.org/jira/browse/KAFKA-1429 Project: Kafka Issue Typ

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13981025#comment-13981025 ] Dmitry Bugaychenko commented on KAFKA-1414: --- Actually similar improvement is nee

[jira] [Created] (KAFKA-1414) Speedup broker startup after hard reset

Dmitry Bugaychenko created KAFKA-1414: - Summary: Speedup broker startup after hard reset Key: KAFKA-1414 URL: https://issues.apache.org/jira/browse/KAFKA-1414 Project: Kafka Issue Type: I

[jira] [Commented] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

[ https://issues.apache.org/jira/browse/KAFKA-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976890#comment-13976890 ] Dmitry Bugaychenko commented on KAFKA-1408: --- Found this patch in the reviosion h

[jira] [Commented] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

[ https://issues.apache.org/jira/browse/KAFKA-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976605#comment-13976605 ] Dmitry Bugaychenko commented on KAFKA-1408: --- We are using artefact from maven:

[jira] [Commented] (KAFKA-1407) Broker can not return to ISR because of BadVersionException

[ https://issues.apache.org/jira/browse/KAFKA-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976597#comment-13976597 ] Dmitry Bugaychenko commented on KAFKA-1407: --- Checked out the logs in more detail

[jira] [Updated] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

[ https://issues.apache.org/jira/browse/KAFKA-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-1408: -- Description: After getting to inconsistence state due to short netwrok failure broker c

[jira] [Created] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

Dmitry Bugaychenko created KAFKA-1408: - Summary: Kafk broker can not stop itself normaly after problems with connection to ZK Key: KAFKA-1408 URL: https://issues.apache.org/jira/browse/KAFKA-1408

[jira] [Created] (KAFKA-1407) Broker can not return to ISR because of BadVersionException

Dmitry Bugaychenko created KAFKA-1407: - Summary: Broker can not return to ISR because of BadVersionException Key: KAFKA-1407 URL: https://issues.apache.org/jira/browse/KAFKA-1407 Project: Kafka