[
https://issues.apache.org/jira/browse/KAFKA-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825708#comment-13825708
]
Jay Kreps commented on KAFKA-1112:
----------------------------------
Yeah at a high-level there are a couple of things we could do:
1. Non-incremental
a. Harden the current approach (what the attached patches do)
b. Use the clean shutdown file
2. Implement incremental recovery (what Jun is proposing)
All of these are good. 1a is implemented, but is arguably gross. I am open to
1b or 2 or a short-term/long-term thing.
For 2 I think the details to figure out would be
1. OffsetCheckpoint is shared so adding the position to that file will impact
other use cases how will that be handled?
2. I suspect that if we want to move to positions we should do something like
(file, log_position, index_position) rather than a mixture of logical and
physical.
3. We need to ensure that log compaction is thought through. This could cause
the physical position to change. That could be fine but we need to reason
through it.
4. We need to ensure that we handle truncation which implies that a position X
could be stable, then deleted, then rewritten differently without flush. This
may be fine we just have to think it through.
> broker can not start itself after kafka is killed with -9
> ---------------------------------------------------------
>
> Key: KAFKA-1112
> URL: https://issues.apache.org/jira/browse/KAFKA-1112
> Project: Kafka
> Issue Type: Bug
> Components: log
> Affects Versions: 0.8, 0.8.1
> Reporter: Kane Kim
> Assignee: Jay Kreps
> Priority: Critical
> Attachments: KAFKA-1112-v1.patch, KAFKA-1112-v2.patch,
> KAFKA-1112-v3.patch, KAFKA-1112.out
>
>
> When I kill kafka with -9, broker cannot start itself because of corrupted
> index logs. I think kafka should try to delete/rebuild indexes itself without
> manual intervention.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
