Hi Ryan,
Have you tried Consumer's pause/resume methods?
Steve
On Wed, Mar 25, 2020, 17:13 Kamal Chandraprakash <
kamal.chandraprak...@gmail.com> wrote:
> With group coordination protocol, you only have to increase the `
> max.poll.interval.ms` / `max.poll.records`.
> Ignore the above messages.
Have you measured the duration between two `poll` invocations and the size
of returned `ConsumrRecords`?
On Wed, Aug 22, 2018, 7:00 PM Shantanu Deshmukh
wrote:
> Ohh sorry, my bad. Kafka version is 0.10.1.0 indeed and so is the client.
>
> On Wed, Aug 22, 2018 at 4:26 PM Steve Tian
22, 2018 at 4:12 PM Steve Tian
> wrote:
>
> > Did you observed any GC-pausing?
> >
> > On Wed, Aug 22, 2018, 6:38 PM Shantanu Deshmukh
> > wrote:
> >
> > > Hi Steve,
> > >
> > > Application is just sending mails. Every record is just a
ail. Currently it is averaging out at 70-80 ms.
>
> On Wed, Aug 22, 2018 at 3:06 PM Steve Tian
> wrote:
>
> > How long did it take to process 50 `ConsumerRecord`s?
> >
> > On Wed, Aug 22, 2018, 5:16 PM Shantanu Deshmukh
> > wrote:
> >
> > > Hell
How long did it take to process 50 `ConsumerRecord`s?
On Wed, Aug 22, 2018, 5:16 PM Shantanu Deshmukh
wrote:
> Hello,
>
> We have Kafka 0.10.0.1 running on a 3 broker cluster. We have an
> application which consumes from a topic having 10 partitions. 10 consumers
> are spawned from this process,
Interesting. Thanks for letting us know.
On Fri, 17 Aug 2018 at 11:22 堅強de泡沫 wrote:
> I modified some consumer configuration items, mainly about the parameters
> of fetch frequency, heartbeat, and session timeout. The problem of long
> time rebalance has not been found in the test environment f
My $0.02:
1. Read the documentation.
2. Help people to understand your problem better: Try to describe your
problem in a gist and see if you can provide some details like: which
version of Kafka you're using on client/server side, your
consumer/producer/server code/configuration that can reproduce
Have you checked the javadoc of KafkaConsumer?
On Mon, Aug 13, 2018, 11:10 PM Kailas Biradar wrote:
> I have issue more time this ConcurrentModificationException because
> KafkaConsumer is not safe for multi-threaded access
>
> --
> kailas
>
You should use ConsumerRebalanceListener to check if there is a rebalance
or not. Seeing same assigned partitions after each poll doesn't mean there
is no rebalance.
Steve
On Wed, Aug 1, 2018, 8:57 AM Manoj Khangaonkar
wrote:
> Hi,
>
> I am implementing flow control by
>
> (1) using the pause(
hed more light on this? What section should I revisit? I
> am using high-level consumer. So I am simply calling consumer.close() when
> I am shutting down the process. Is there any other method to be called
> before calling close()?
>
> On Mon, Jul 9, 2018 at 5:58 PM Steve Tian wr
Please re-read the javadoc of KafkaConsumer, make sure you know how to
wakeup/close consumer properly while shutting down your application. Try
to understand the motivation of KIP-62 and adjust related timeout.
On Mon, Jul 9, 2018, 8:05 PM harish lohar wrote:
> Try reducing below timer
> metada
nd the lag increasing when processing for some of
> your partitions are stuck?
>
> May be the problem is in the producer side.
>
> Regards
> Sab
>
>
> On Mon, 9 Jul 2018, 11:43 am Steve Tian, wrote:
>
> > Are you sure your consumer was still the owner of your p
Are you sure your consumer was still the owner of your partitions?
On Mon, Jul 9, 2018, 12:54 PM dev loper wrote:
> Hi Kafka Users,
>
> I am desperate to find an answer to this issue. I would like to know
> whether my issue is due to a Single Kafka Consumer ? Where should I look
> for answers f
Producer configuration?
On Wed, Sep 20, 2017, 2:50 PM MAHA ALSAYASNEH <
maha.alsayas...@univ-grenoble-alpes.fr> wrote:
> Hello,
>
> Any suggestion regarding this msg:
> " org.apache.kafka.common.errors.TimeoutException: Expiring 61 record(s)
> for due to 30001 ms has passed since batch creat
metadata from
> Kafka brokers but Kafka give them stale metadata near 30-40 sec.
> It try to fetch 3-4 time in between until it get updated metadata.
> This is completely different problem than
> https://github.com/Shopify/sarama/issues/661
>
>
>
> On Thu, Jun 2, 2016 at 6:05
0.0
> go sarama client version: 1.8
>
> On Thu, Jun 2, 2016 at 5:14 PM, Steve Tian
> wrote:
>
> > Client version?
> >
> > On Fri, Jun 3, 2016, 4:44 AM safique ahemad
> wrote:
> >
> > > Hi All,
> > >
> > > We are using Kafka broker
Client version?
On Fri, Jun 3, 2016, 4:44 AM safique ahemad wrote:
> Hi All,
>
> We are using Kafka broker cluster in our data center.
> Recently, It is realized that when a Kafka broker goes down then client try
> to refresh the metadata but it get stale metadata upto near 30 seconds.
>
> After
on my Windows10 environment where is
> on the host environment and worked well on the VM environment... It is
> weird indeed.
>
> Best regards
>
> Kim
>
> 2016-01-20 18:14 GMT+09:00 Steve Tian :
>
>> Your code works in my environment. Are you able to run your producer
e.kafka/kafka_2.11/0.9.0.0
>
> The maven dependency is as below:
>
>
> org.apache.kafka
> kafka_2.11
> 0.9.0.0
>
>
> Are you using this version?
>
> Best regards
>
> Kim
>
> 2016-01-20 18:14 GMT+09:00 Steve Tian :
>
>> Your code works in my environ
rom
> producer, and my broker version, if you mean the kafka version, is 0.9.0.
>
> Best regards
>
> Kim
>
> 2016-01-20 17:28 GMT+09:00 Steve Tian :
>
>> Did you start your consumer before sending message? Broker version?
>>
>> Cheers, Steve
>>
&g
Did you start your consumer before sending message? Broker version?
Cheers, Steve
On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM wrote:
> Hello.
>
> I set up the Kafka testbed environment on my VirtualBox, which simply has a
> Kafka broker.
>
> I tested the simple consumer & producer scripts, aka
Hi Joe,
bootstrap.servers is the configuration for new *java* producer.
Cheers, Steve
On Mon, Jan 18, 2016, 5:41 PM Joe San wrote:
> Dear Kafka Users,
>
> Is there a bug with the producer config?
>
> I have asked a question on Stackoverflow:
>
>
> http://stackoverflow.com/questions/34851412/ap
roblem by at least reducing the amount of time for the request to fail.
> Depending on how adventurous you are, you could try using a version with
> that patch and maybe adjust the setting lower than its default.
>
> -Ewen
>
> On Wed, Sep 2, 2015 at 10:46 AM, Steve Tian
> wrote:
&
Would kafka dev kindly give us some advice on this?
Cheers, Steve
On Tue, Sep 1, 2015, 11:20 PM Steve Tian wrote:
> Thanks, Rahul! In my environment I need to have reconnect.backoff.ms
> longer than OS default tcp timeout so that NetworkClient can give second
> node a try.
>
>
ed earlier -
>
> http://qnalist.com/questions/6002514/new-producer-metadata-update-problem-on-2-node-cluster
>
>
>
> On Mon, Aug 31, 2015 at 11:00 PM, Steve Tian
> wrote:
>
> > Hi everyone,
> >
> > Is there any concerns to have a long reconn
Hi everyone,
Is there any concerns to have a long reconnect.backoff.ms for new java
Kafka producer (0.8.2.0/0.8.2.1)?
Assuming we have bootstrap.servers=host1:port1,host2:port2,host3:port3 and
host1 is *down* in the very beginning. If a newly created Kafka producer
decide to choose host1 as first
26 matches
Mail list logo
