@Jungtaek, thanks for the explanation!
@Colin, please see the attached code in my previous mail for all the
details.
On Sun, Aug 11, 2019 at 1:20 PM Jungtaek Lim wrote:
> Btw, I'd like to ask you to move on thread for KIP discussion, as it will
> make us reaching conclusion faster and have singl
Btw, I'd like to ask you to move on thread for KIP discussion, as it will
make us reaching conclusion faster and have single channel to discuss.
On Sun, Aug 11, 2019 at 8:16 PM Jungtaek Lim wrote:
> So we have some use case which we don't just rely on everything what Kafka
> consumer provides. W
So we have some use case which we don't just rely on everything what Kafka
consumer provides. We want to know current assignment on this consumer, and
to get the latest assignment, we called the hack `poll(0)`.
That said, we don't want to pull any records here, and there's no way to
accomplish thi
Hi Gabor,
What is it that you want to do here? If you just want to check that the
partitions exist, but not fetch any data, you could use
AdminClient#describeTopics for that. If you want to create the topics, you
could use AdminClient#createTopics.
best,
Colin
On Fri, Aug 9, 2019, at 11:23
> Each KafkaConsumer method that returns metadata will already block until
such metadata is available
The old API was waiting infinitely but the new has a timeout which has
effect on the metadata fetch as well.
Spark is interested in only the assigned partitions and/or
latest/earliest/... offsets.
> pull some records even they're only interested in metadata.
Jungtaek, what is the use-case here? Each KafkaConsumer method that returns
metadata will already block until such metadata is available... so why
would you need to apply this "hack" in the first place?
Ryanne
On Wed, Aug 7, 2019 at 2
Thanks Viktor for guiding me through this!
I would initiate new thread to ask edit permission on wiki. Once I got
permission I'll come up with simple KIP page and initiate discussion thread.
Thanks again,
Jungtaek Lim
On Thu, Aug 8, 2019 at 9:42 PM Viktor Somogyi-Vass
wrote:
> Hey Jungtaek,
>
Hey Jungtaek,
Thanks for your interest, sometimes I also think such an API would be a
good thing.
I don't see any strong reasons neither in KIP-288 nor in KIP-266 why such
an API shouldn't be created, so go ahead with it, although you'll need to
create a short KIP for this as the KafkaConsumer cla
If we just wanted to remove deprecation and let both co-exist, that would
be also viable, though `poll(0)` is still a hack and it would be ideal to
provide official approach to do so.
On Wed, Aug 7, 2019 at 4:24 PM Jungtaek Lim wrote:
> Hi devs,
>
> I'm trying to replace deprecated poll(long) wi
Hi devs,
I'm trying to replace deprecated poll(long) with poll(Duration), and
realized there's no alternative which behaves exactly same as poll(0), as
poll(0) has been used as a hack to only update metadata instead of pulling
records. poll(Duration.ZERO) wouldn't behave same since even updating
m
10 matches
Mail list logo
