Hey Jungtaek, Thanks for your interest, sometimes I also think such an API would be a good thing. I don't see any strong reasons neither in KIP-288 nor in KIP-266 why such an API shouldn't be created, so go ahead with it, although you'll need to create a short KIP for this as the KafkaConsumer class considered to be a public API.
Best, Viktor On Wed, Aug 7, 2019 at 9:26 AM Jungtaek Lim <kabh...@gmail.com> wrote: > If we just wanted to remove deprecation and let both co-exist, that would > be also viable, though `poll(0)` is still a hack and it would be ideal to > provide official approach to do so. > > On Wed, Aug 7, 2019 at 4:24 PM Jungtaek Lim <kabh...@gmail.com> wrote: > > > Hi devs, > > > > I'm trying to replace deprecated poll(long) with poll(Duration), and > > realized there's no alternative which behaves exactly same as poll(0), as > > poll(0) has been used as a hack to only update metadata instead of > pulling > > records. poll(Duration.ZERO) wouldn't behave same since even updating > > metadata will be timed-out. So now end users would need to give more > > timeout and even pull some records even they're only interested in > metadata. > > > > I looked back some KIPs which brought the change, and "discarded" KIP > > (KIP-288 [1]) actually proposed a new API which only pulls metadata. > > KIP-266 [2] is picked up instead but it didn't cover all the things what > > KIP-288 proposed. I'm seeing some doc explaining poll(0) hasn't been > > supported officially, but the hack has been widely used and they can't be > > ignored. > > > > Kafka test code itself relies on either deprecated poll(0), > > or updateAssignmentMetadataIfNeeded, which seems to be private API only > for > > testing. > > (Btw, I'd try out replacing poll(0) to updateAssignmentMetadataIfNeeded > as > > avoiding deprecated method - if it works I'll submit a PR.) > > > > I'm feeling that it would be ideal to expose > > `updateAssignmentMetadataIfNeeded` to the public API, maybe with renaming > > as `waitForAssignment` which was proposed in KIP-288 if it feels too > long. > > > > What do you think? If it sounds feasible I'd like to try out contribution > > on this. I'm new to contribute Kafka community, so not sure it would > > require a new KIP or not. > > > > Thanks, > > Jungtaek Lim (HeartSaVioR) > > > > 1. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-288%3A+%5BDISCARDED%5D+Consumer.poll%28%29+timeout+semantic+change+and+new+waitForAssignment+method > > 2. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-266%3A+Fix+consumer+indefinite+blocking+behavior > > > > > > -- > Name : Jungtaek Lim > Blog : http://medium.com/@heartsavior > Twitter : http://twitter.com/heartsavior > LinkedIn : http://www.linkedin.com/in/heartsavior >
