I’m seeing behaviour that I don’t understand when I have Consumers fetching
from multiple Partitions from the same Topic. There are two different
conditions arising:
1. A subset of the Partitions allocated to a given Consumer not being consumed
at all. The Consumer appears healthy, the Thread
, M. Manna
mailto:manme...@gmail.com>> wrote:
Hi James,
3 Consumers in a group means you are having 20 partitions per consumer (as
per your 60 partition and 1 CGroup setup), 5 means 12. There's nothing
special about these numbers as you also noticed.
Have you tried setting fetch.max
4,889 DEBUG
[org.apache.kafka.clients.consumer.internals.Fetcher]
'EE-ManagedThreadFactory-default-Thread-2' [Consumer
clientId=consumer-LedgerService-group-1, groupId=LedgerService-group] Sending
READ_UNCOMMITTED IncrementalFetchRequest(toSend=(Ledger-1), toForget=(),
implied=(Ledg
data available to be read instantly?
Thanks,
Jamie
Sent from AOL Mobile Mail
Get the new AOL app: mail.mobile.aol.com<http://mail.mobile.aol.com/>
On Sunday, 8 March 2020, James Olsen
mailto:ja...@inaseq.com>> wrote:
Using 2.3.1 Brokers makes things worse. There are now 2 fetch.max.wait
?
We can choose 2.2.1 or 2.3.1 for the Broker (AWS recommend 2.2.1 although don’t
state why). Based on the experiences below, I would then go with the
corresponding 2.2.2 or 2.3.1 Client version.
Which combo would people recommend?
> On 9/03/2020, at 12:03 PM, James Olsen wrote:
>
&
P.S. I guess the big question is what is the best way to handle or avoid
UNKNOWN_PRODUCER_ID when running versions that don’t include KAFKA-7190 /
KAFKA-8710 ?
We are using non-transactional idempotent Producers.
> On 9/03/2020, at 12:59 PM, James Olsen wrote:
>
> For completene
Also check your Kafka Client and Server versions. There are serious latency
issues when mixing different client and server versions IF your consumers
handle multiple partitions.
> On 27/03/2020, at 12:59, Chris Larsen wrote:
>
> Hi Vidhya,
>
> How many tasks are you running against the topic
Resolved by downgrading Client to 2.2.2 and implementing an application level
heartbeat on every Producer to avoid he UNKNOWN_PRODUCER_ID issue.
> On 9/03/2020, at 16:08, James Olsen wrote:
>
> P.S. I guess the big question is what is the best way to handle or avoid
> UNKNOWN_PROD
te.fr>> wrote:
There are serious latency issues when mixing different client and server version
Could you be more specific ? Link to any issue ?
Thanks by advance !
Christophe
____
De : James Olsen mailto:ja...@inaseq.com>>
Envoyé : vendredi 27 mars 202
org.apache.kafka.clients.producer.ProducerConfig
org.apache.kafka.clients.consumer.ConsumerConfig
> On 3/04/2020, at 04:30, 一直以来 <279377...@qq.com> wrote:
>
> Properties props = new Properties(); props.put("bootstrap.servers",
> "localhost:9092"); props.put("acks", "all"); props.put("key.seri
We are using AWS MSK with Kafka 2.4.1 (and same client version), 3 Brokers. We
are seeing fairly frequent consumer offset commit fails as shown in the example
logs below. Things continue working as they are all retriable, however I would
like to improve this situation.
The issue occurs most o
against the MSK service since some of these networking issues has to do with
one of the brokers being unavailable -- something that is not supposed to
happen.
Thanks,
-- Ricardo
On 6/18/20 9:18 PM, James Olsen wrote:
We are using AWS MSK with Kafka 2.4.1 (and same client version), 3 Brokers.
If it's of any value to you, we use the following test to check that we have a
well balanced set of consumer group ids. Note that in the code,
ConsumerGroups.ALL_GROUPS is simply a list of all our consumer group ids.
Spreading the offset commit load across these partitions evenly helps in
lev
We had a 2.5.1 Broker/Client system running for some time with regular rolling
OS upgrades to the Brokers without any problems. A while ago we upgraded both
Broker and Clients to 2.7.1 and now on the first rolling OS upgrade to the
2.7.1 Brokers we encountered some Consumer issues. We have a 3
21 at 10:27 AM James Olsen
mailto:ja...@inaseq.com>> wrote:
We had a 2.5.1 Broker/Client system running for some time with regular rolling
OS upgrades to the Brokers without any problems. A while ago we upgraded both
Broker and Clients to 2.7.1 and now on the first rolling OS upgrade to t
not occur with 2.5.1 and 2.7.0 Clients. This make me
suspect that https://issues.apache.org/jira/browse/KAFKA-10793 introduced this
issue.
Regards, James.
On 24/11/2021, at 14:35, James Olsen
mailto:ja...@inaseq.com>> wrote:
Luke,
We did not upgrade to resolve the issue. We simply
I recently observed the following series of events for a particular partition
(MyTopic-6):
2022-03-18 03:18:28,562 INFO
[org.apache.kafka.clients.consumer.internals.ConsumerCoordinator]
'executor-thread-2' [Consumer clientId=consumer-MyTopicService-group-3,
groupId=MyTopicService-group] Setti
known issue KAFKA-13636
<https://issues.apache.org/jira/browse/KAFKA-13636>, which should be fixed
in the newer version.
Thank you.
Luke
On Mon, Apr 11, 2022 at 9:18 AM James Olsen
mailto:ja...@inaseq.com>> wrote:
I recently observed the following series of events for a particular
pa
1 and v3.2.0 will be coming soon.
Thank you.
Luke
On Fri, Apr 29, 2022 at 8:53 AM James Olsen
mailto:ja...@inaseq.com>> wrote:
Luke,
Do you know if 2.8.2 will be released anytime soon? It appears to be waiting
on https://issues.apache.org/jira/browse/KAFKA-13805 for which fixes are
avai
Prior to KIP-794 it was possible to create a custom Partitioner that could
delegate to the DefaultPartitioner. This has been deprecated so we can now
only delegate to BuiltInPartitioner.partitionForKey which does not handle a
non-keyed message. Hence there is now no mechanism for a custom Part
Looking at
org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(List,
ClientDnsLookup) shows that error message is referring to a singular url from
the list. So the issue is earlier in the chain - the list is not being decoded
from the CSV.
> On 12 Sep 2024, at 03:03, Josep Prat wr
21 matches
Mail list logo
