Re: [DISCUSSION] KIP-619: Add internal topic creation support

2020-08-19 Thread Cheng Tan
Hi David, Thanks for the feedback. They are really helpful. > Can you clarify a bit more what the difference is between regular topics > and internal topics (excluding __consumer_offsets and > __transaction_state)? Reading your last message, if internal topics > (excluding the two) can be creat

Re: [DISCUSSION] KIP-619: Add internal topic creation support

2020-08-14 Thread David Arthur
Cheng, Can you clarify a bit more what the difference is between regular topics and internal topics (excluding __consumer_offsets and __transaction_state)? Reading your last message, if internal topics (excluding the two) can be created, deleted, produced to, consumed from, added to transactions,

Re: [DISCUSSION] KIP-619: Add internal topic creation support

2020-07-07 Thread Cheng Tan
Hi Colin, Thanks for the comments. I’ve modified the KIP accordingly. > I think we need to understand which of these limitations we will carry > forward and which we will not. We also have the option of putting > limitations just on consumer offsets, but not on other internal topics. In the

Re: [DISCUSSION] KIP-619: Add internal topic creation support

2020-06-15 Thread Colin McCabe
Hi Cheng, The link from the main KIP page is an "edit link" meaning that it drops you into the editor for the wiki page. I think the link you meant to use is a "view link" that will just take you to view the page. In general I'm not sure what I'm supposed to take away from the large UML diagr

Re: [DISCUSSION] KIP-619: Add internal topic creation support

2020-06-12 Thread David Arthur
Cheng, thanks for the KIP! Can you include some details about how this will work the post-ZK world? For KafkaAdminClient, will we add a new "internal" field to NewTopic, or will we reuse the existing "configs" map. One concern with sticking this new special field in the topic configs is that we c

[DISCUSSION] KIP-619: Add internal topic creation support

2020-05-29 Thread Cheng Tan
Hello developers, I’m proposing KIP-619 to add internal topic creation support. Kafka and its upstream applications treat internal topics differently from non-internal topics. For example: • Kafka handles topic creation response errors differently for internal topics • Intern