Currently the community is busy with merging all the KIP-500 related work into trunk for the 2.8 release so no committer reply 🤨.
Not very sure, but I find that org.apache.kafka.clients.consumer.internals.RequestFuture Is also similar to CompletableFuture, do we have plan to replace RequestFuture with CompletableFuture > On Feb 10, 2021, at 11:23, Chia-Ping Tsai <chia7...@apache.org> wrote: > > Ping for more discussion :) > > On 2021/01/31 05:39:17, Chia-Ping Tsai <chia7...@apache.org> wrote: >> It seems to me changing the input type might make complicate the migration >> from deprecated send method to new API. >> >> Personally, I prefer to introduce a interface called “SendRecord” to replace >> ProducerRecord. Hence, the new API/classes is shown below. >> >> 1. CompletionStage send(SendRecord) >> 2. class ProducerRecord implement SendRecord >> 3. Introduce builder pattern for SendRecord >> >> That includes following benefit. >> >> 1. Kafka users who don’t use both return type and callback do not need to >> change code even though we remove deprecated send methods. (of course, they >> still need to compile code with new Kafka) >> >> 2. Kafka users who need Future can easily migrate to new API by regex >> replacement. (cast ProduceRecord to SendCast and add toCompletableFuture) >> >> 3. It is easy to support topic id in the future. We can add new method to >> SendRecord builder. For example: >> >> Builder topicName(String) >> Builder topicId(UUID) >> >> 4. builder pattern can make code more readable. Especially, Produce record >> has a lot of fields which can be defined by users. >> — >> Chia-Ping >> >> On 2021/01/30 22:50:36 Ismael Juma wrote: >>> Another thing to think about: the consumer api currently has >>> `subscribe(String|Pattern)` and a number of methods that accept >>> `TopicPartition`. A similar approach could be used for the Consumer to work >>> with topic ids or topic names. The consumer side also has to support >>> regexes so it probably makes sense to have a separate interface. >>> >>> Ismael >>> >>> On Sat, Jan 30, 2021 at 2:40 PM Ismael Juma <ism...@juma.me.uk> wrote: >>> >>>> I think this is a promising idea. I'd personally avoid the overload and >>>> simply have a `Topic` type that implements `SendTarget`. It's a mix of both >>>> proposals: strongly typed, no overloads and general class names that >>>> implement `SendTarget`. >>>> >>>> Ismael >>>> >>>> On Sat, Jan 30, 2021 at 2:22 PM Jason Gustafson <ja...@confluent.io> >>>> wrote: >>>> >>>>> Giving this a little more thought, I imagine sending to a topic is the >>>>> most >>>>> common case, so maybe it's an overload worth having. Also, if `SendTarget` >>>>> is just a marker interface, we could let `TopicPartition` implement it >>>>> directly. Then we have: >>>>> >>>>> interface SendTarget; >>>>> class TopicPartition implements SendTarget; >>>>> >>>>> CompletionStage<RecordMetadata> send(String topic, Record record); >>>>> CompletionStage<RecordMetadata> send(SendTarget target, Record record); >>>>> >>>>> The `SendTarget` would give us a lot of flexibility in the future. It >>>>> would >>>>> give us a couple options for topic ids. We could either have an overload >>>>> of >>>>> `send` which accepts `Uuid`, or we could add a `TopicId` type which >>>>> implements `SendTarget`. >>>>> >>>>> -Jason >>>>> >>>>> >>>>> On Sat, Jan 30, 2021 at 1:11 PM Jason Gustafson <ja...@confluent.io> >>>>> wrote: >>>>> >>>>>> Yeah, good question. I guess we always tend to regret using lower-level >>>>>> types in these APIs. Perhaps there should be some kind of interface: >>>>>> >>>>>> interface SendTarget >>>>>> class TopicIdTarget implements SendTarget >>>>>> class TopicTarget implements SendTarget >>>>>> class TopicPartitionTarget implements SendTarget >>>>>> >>>>>> Then we just have: >>>>>> >>>>>> CompletionStage<RecordMetadata> send(SendTarget target, Record record); >>>>>> >>>>>> Not sure if we could reuse `Record` in the consumer though. We do have >>>>>> some state in `ConsumerRecord` which is not present in `ProducerRecord` >>>>>> (e.g. offset). Perhaps we could provide a `Record` view from >>>>>> `ConsumerRecord` for convenience. That would be useful for use cases >>>>> which >>>>>> involve reading from one topic and writing to another. >>>>>> >>>>>> -Jason >>>>>> >>>>>> On Sat, Jan 30, 2021 at 12:29 PM Ismael Juma <ism...@juma.me.uk> wrote: >>>>>> >>>>>>> Interesting idea. A couple of things to consider: >>>>>>> >>>>>>> 1. Would we introduce the Message concept to the Consumer too? I think >>>>>>> that's what .NET does. >>>>>>> 2. If we eventually allow a send to a topic id instead of topic name, >>>>>>> would >>>>>>> that result in two additional overloads? >>>>>>> >>>>>>> Ismael >>>>>>> >>>>>>> On Sat, Jan 30, 2021 at 11:38 AM Jason Gustafson <ja...@confluent.io> >>>>>>> wrote: >>>>>>> >>>>>>>> For the sake of having another option to shoot down, we could take a >>>>>>> page >>>>>>>> from the .net client and separate the message data from the >>>>> destination >>>>>>>> (i.e. topic or partition). This would get around the need to use a >>>>> new >>>>>>>> verb. For example: >>>>>>>> >>>>>>>> CompletionStage<RecordMetadata> send(String topic, Message message); >>>>>>>> CompletionStage<RecordMetadata> send(TopicPartition topicPartition, >>>>>>> Message >>>>>>>> message); >>>>>>>> >>>>>>>> -Jason >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Jan 30, 2021 at 11:30 AM Jason Gustafson <ja...@confluent.io >>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I think this still makes sense as a separate KIP. For KIP-691, we >>>>> are >>>>>>>> just >>>>>>>>> looking to help define the error contract for the new API. >>>>>>>>> >>>>>>>>> -Jason >>>>>>>>> >>>>>>>>> On Sat, Jan 30, 2021 at 8:39 AM Ismael Juma <ism...@juma.me.uk> >>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Are we saying that we won't pursue this KIP in favor of the other >>>>>>> one? >>>>>>>>>> >>>>>>>>>> Ismael >>>>>>>>>> >>>>>>>>>> On Sat, Jan 30, 2021, 4:15 AM Chia-Ping Tsai <chia7...@apache.org >>>>>> >>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> hi Jason >>>>>>>>>>> >>>>>>>>>>> Thanks for your response. "transmit" is good to me. >>>>>>>>>>> >>>>>>>>>>> As we discussed by email, KIP-706 is going to be merged to >>>>> KIP-691( >>>>>>>>>>> https://cwiki.apache.org/confluence/x/PSfZCQ). Hence, please >>>>> feel >>>>>>>> free >>>>>>>>>> to >>>>>>>>>>> replace "produce" by "transmit" in KIP-691. >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Chia-Ping >>>>>>>>>>> >>>>>>>>>>> On 2021/01/30 00:48:38, Jason Gustafson <ja...@confluent.io> >>>>>>> wrote: >>>>>>>>>>>> Hi Chia-Ping, >>>>>>>>>>>> >>>>>>>>>>>> I think this is a great idea. It is a pity that we cannot >>>>>>> continue >>>>>>>> to >>>>>>>>>> use >>>>>>>>>>>> the `send` verb, but I don't see how we can. I know we >>>>> considered >>>>>>>>>>>> `transmit` as another option which is closer to `send`. That >>>>>>> would >>>>>>>>>> avoid >>>>>>>>>>>> the redundancy when people choose the common "producer" >>>>> variable >>>>>>>> name. >>>>>>>>>>>> >>>>>>>>>>>> producer.transmit >>>>>>>>>>>> >>>>>>>>>>>> instead of >>>>>>>>>>>> >>>>>>>>>>>> producer.produce >>>>>>>>>>>> >>>>>>>>>>>> A couple alternatives might be `write` or `append`. I'm happy >>>>>>> with >>>>>>>>>>>> `produce` as well, but curious if others have thoughts. >>>>>>>>>>>> >>>>>>>>>>>> -Jason >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jan 20, 2021 at 9:37 AM Chia-Ping Tsai < >>>>>>> chia7...@apache.org >>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Dear all, >>>>>>>>>>>>> >>>>>>>>>>>>> I'd like to start the discussion thread for KIP-706: >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100829459 >>>>>>>>>>>>> >>>>>>>>>>>>> KIP-706 is proposing to introduce new API "CompletionStage >>>>>>>>>>>>> produce(record)" to Producer. Kafka users can leverage >>>>>>>>>> CompletionStage >>>>>>>>>>> to >>>>>>>>>>>>> write asynchronous non-blocking code. CompletionStage is >>>>> more >>>>>>>>>> powerful >>>>>>>>>>> than >>>>>>>>>>>>> Future and callback. Also, the code using Future and >>>>> callback >>>>>>> can >>>>>>>> be >>>>>>>>>>> easily >>>>>>>>>>>>> re-written by CompletionStage. >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers, >>>>>>>>>>>>> Chia-Ping >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>
