Oops. Yes, I meant “replication factor”.
> On Oct 4, 2017, at 7:18 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > Randall: > bq. AdminClient currently allows changing the replication factory. > > By 'replication factory' did you mean 'replication factor' ? > > Cheers > >> On Wed, Oct 4, 2017 at 9:58 AM, Randall Hauch <rha...@gmail.com> wrote: >> >> Currently the KIP's scope is only topics that don't yet exist, and we have >> to cognizant of race conditions between tasks with the same connector. I >> think it is worthwhile to consider whether the KIP's scope should expand to >> also address *existing* partitions, though it may not be appropriate to >> have as much control when changing the topic settings for an existing >> topic. For example, changing the number of partitions (which the KIP >> considers a "topic-specific setting" even though technically it is not) >> shouldn't be done blindly due to the partitioning impacts, and IIRC you >> can't reduce them (which we could verify before applying). Also, I don't >> think the AdminClient currently allows changing the replication factory. I >> think changing the topic configs is less problematic both from what makes >> sense for connectors to verify/change and from what the AdminClient >> supports. >> >> Even if we decide that it's not appropriate to change the settings on an >> existing topic, I do think it's advantageous to at least notify the >> connector (or task) prior to the first record sent to a given topic so that >> the connector can fail or issue a warning if it doesn't meet its >> requirements. >> >> Best regards, >> >> Randall >> >> On Wed, Oct 4, 2017 at 12:52 AM, Stephane Maarek < >> steph...@simplemachines.com.au> wrote: >> >>> Hi Randall, >>> >>> Thanks for the KIP. I like it >>> What happens when the target topic is already created but the configs do >>> not match? >>> i.e. wrong RF, num partitions, or missing / additional configs? Will you >>> attempt to apply the necessary changes or throw an error? >>> >>> Thanks! >>> Stephane >>> >>> >>> On 24/5/17, 5:59 am, "Mathieu Fenniak" <mathieu.fenn...@replicon.com> >>> wrote: >>> >>> Ah, yes, I see you a highlighted part that should've made this clear >>> to me the first read. :-) Much clearer now! >>> >>> By the way, enjoyed your Debezium talk in NYC. >>> >>> Looking forward to this Kafka Connect change; it will allow me to >>> remove a post-deployment tool that I hacked together for the purpose >>> of ensuring auto-created topics have the right config. >>> >>> Mathieu >>> >>> >>> On Tue, May 23, 2017 at 11:38 AM, Randall Hauch <rha...@gmail.com> >>> wrote: >>>> Thanks for the quick feedback, Mathieu. Yes, the first >> configuration >>> rule >>>> whose regex matches will be applied, and no other rules will be >>> used. I've >>>> updated the KIP to try to make this more clear, but let me know if >>> it's >>>> still not clear. >>>> >>>> Best regards, >>>> >>>> Randall >>>> >>>> On Tue, May 23, 2017 at 10:07 AM, Mathieu Fenniak < >>>> mathieu.fenn...@replicon.com> wrote: >>>> >>>>> Hi Randall, >>>>> >>>>> Awesome, very much looking forward to this. >>>>> >>>>> It isn't 100% clear from the KIP how multiple config-based rules >>> would >>>>> be applied; it looks like the first configuration rule whose regex >>>>> matches the topic name will be used, and no other rules will be >>>>> applied. Is that correct? (I wasn't sure if it might cascade >>>>> together multiple matching rules...) >>>>> >>>>> Looks great, >>>>> >>>>> Mathieu >>>>> >>>>> >>>>> On Mon, May 22, 2017 at 1:43 PM, Randall Hauch <rha...@gmail.com> >>> wrote: >>>>>> Hi, all. >>>>>> >>>>>> We recently added the ability for Kafka Connect to create >>> *internal* >>>>> topics >>>>>> using the new AdminClient, but it still would be great if Kafka >>> Connect >>>>>> could do this for new topics that result from source connector >>> records. >>>>>> I've outlined an approach to do this in "KIP-158 Kafka Connect >>> should >>>>> allow >>>>>> source connectors to set topic-specific settings for new >> topics". >>>>>> >>>>>> *https://cwiki.apache.org/confluence/display/KAFKA/KIP- >>>>> 158%3A+Kafka+Connect+should+allow+source+connectors+to+ >>>>> set+topic-specific+settings+for+new+topics >>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP- >>>>> 158%3A+Kafka+Connect+should+allow+source+connectors+to+ >>>>> set+topic-specific+settings+for+new+topics>* >>>>>> >>>>>> Please take a look and provide feedback. Thanks! >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Randall >>>>> >>> >>> >>> >>> >>
