Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2240803280 ## metadata/src/main/java/org/apache/kafka/image/TopicsImage.java: ## @@ -119,4 +127,40 @@ public Map topicIdToNameView() { public String toString() { re

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2240795050 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## Review Comment: Moved -- This is an automated message from the Apache Git Service. To respon

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2240767704 ## core/src/main/scala/kafka/server/metadata/BrokerMetadataPublisher.scala: ## @@ -340,7 +339,7 @@ class BrokerMetadataPublisher( // recovery-from-unclean-shutd

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

ijuma commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2214892949 ## checkstyle/import-control.xml: ## @@ -486,6 +486,7 @@ + Review Comment: `internals` simply means it should not be used outside the `kafka` re

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

jsancio commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2190622591 ## metadata/src/main/java/org/apache/kafka/image/TopicsImage.java: ## @@ -119,4 +127,40 @@ public Map topicIdToNameView() { public String toString() { ret

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

junrao commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2190605910 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,828 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contri

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

jsancio commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2190478910 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,828 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contr

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2183612050 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,816 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

junrao commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2183189820 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,816 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contri

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2182486542 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,816 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2182464150 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,816 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2182458792 ## raft/src/test/java/org/apache/kafka/raft/KafkaRaftLogTest.java: ## @@ -0,0 +1,1190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

junrao commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2181170038 ## metadata/src/test/java/org/apache/kafka/image/TopicsImageTest.java: ## @@ -671,4 +672,54 @@ public void testTopicIdToNameView() { assertThrows(UnsupportedOpe

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2180132437 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,828 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2180136384 ## raft/src/test/java/org/apache/kafka/raft/KafkaRaftLogTest.java: ## @@ -0,0 +1,1181 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2180138344 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,828 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2180102742 ## metadata/src/main/java/org/apache/kafka/image/TopicsImage.java: ## @@ -119,4 +127,40 @@ public Map topicIdToNameView() { public String toString() { re

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2140130098 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2140129567 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2140112064 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2140110837 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r214004 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2140047143 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2139512929 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2139520463 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

brandboat commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2114281965 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * con

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

brandboat commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2114281965 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * con

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

brandboat commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2114281965 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * con

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

ijuma commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2113966539 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contrib

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

ijuma commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2113966539 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contrib

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

dengziming commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2113867108 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * co

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

ijuma commented on code in PR #19762: URL: https://github.com/apache/kafka/pull/19762#discussion_r2104979893 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftLog.java: ## @@ -0,0 +1,843 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contrib

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

ijuma commented on PR #19762: URL: https://github.com/apache/kafka/pull/19762#issuecomment-2905019893 The description I wrote in the ticket was the conclusion after a discussion with a few people including @jsancio and @cmccabe. @jsancio does that approach still make sense to you? -- Thi

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

dengziming commented on PR #19762: URL: https://github.com/apache/kafka/pull/19762#issuecomment-2903389220 @mimaison That makes sense to me, 👍 if we should make the tradeoff, the modularity outweighs the pluggability of the raft module, @jsancio @ijuma, where do you think is better to put `

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on PR #19762: URL: https://github.com/apache/kafka/pull/19762#issuecomment-2900596586 I'm not sure I interpret that statement from KIP-595 as an intent to keep the `raft` module dependency free. As you said the `raft` module is not public API, and I've not seen any discus

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

dengziming commented on PR #19762: URL: https://github.com/apache/kafka/pull/19762#issuecomment-2900099738 > Where has this been discussed? In KIP-595 we wrote: > We are also trying to pave the way for normal partition replication through Raft as well as eventually supporting metad

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison commented on PR #19762: URL: https://github.com/apache/kafka/pull/19762#issuecomment-2898574096 > we are trying to keep it dependency-free to be used as a raft-sdk anywhere Where has this been discussed? The way Kafka implements Raft is via a topic so our implementation of

Re: [PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

dengziming commented on PR #19762: URL: https://github.com/apache/kafka/pull/19762#issuecomment-2896341933 There are some concerns regarding this change, though we haven't made `kraft` a public API, we are trying to keep it dependency-free to be used as a raft-sdk anywhere. For example, if

[PR] KAFKA-15599: Move KafkaMetadataLog to the raft module [kafka]

mimaison opened a new pull request, #19762: URL: https://github.com/apache/kafka/pull/19762 - Rename `KafkaMetadataLog` to `KafkaRaftLog` - Make `raft` depend on `storage` - Remove `storage` dependency on `metadata` as this otherwise create a cyclic dependency -- This is an auto