Re: fluentd with kafka 3 issues

2022-08-12 Thread Raphael Mazelier
Hi, this seems a fluentd issue not a kafka one. maybe you can you try another log shipper to validate this assumption; there are plenty and fluentd is already an old (and buggy and slow one); you can try fluent-bit or vector maybe. -- Raphael Mazelier On 12/08/2022 04:58, pengyh wrote: do

Replication of consumer group offset

2022-08-12 Thread Raphael Mazelier
tion to reset offset to earliest or a time (maybe its sufficient we will see) ; but I would have prefer a tool to create the exact same consumer group in the dest cluster. Is someone have already done that? and know an tool to do it? Any help appreciated :) -- Raphael Mazelier

Re: Replication of consumer group offset

2022-08-16 Thread Raphael Mazelier
Replying to myself I ended by making a small python utility that do the work. On 12/08/2022 12:50, Raphael Mazelier wrote: Hi kafka community, I've been an happy user of kafka since years. Today I faced a problem that begins to make mad... So we have the project to migrate from an

Re: Kafka MirroMaker 2 not considering topics.exclude configuration to exclude topic from replication

2022-09-09 Thread Raphael Mazelier
Hi, Mirromaker2 config is inconsistent.  But docs mentions topics.blacklist. So maybe try something like: topics.blacklist = xxx or active->standby.topics.blacklist = .* or ... On 09/09/2022 09:35, Anand Parmar wrote: Hi, I have two kafka clusters. one is active and other is standby. The

Re: Kafka Consumer Lag Monitoring

2023-05-10 Thread Raphael Mazelier
I'm not sure to follow you: either mine - https://github.com/ut0mt8/yakle or - https://github.com/danielqsj/kafka_exporter or - https://github.com/redpanda-data/kminion export consumer-group lag metrics. All works. best, -- Raphael Mazelier On 10/05/2023 22:47, Akshay Kumar wrote:

Re: Schema Registry - serving multiple Kafka clusters

2025-01-07 Thread Raphael Mazelier
Actually schema registry is backed by topics on one Kafka cluster. Applications using it can be where you want using this particular cluster or not. Or maybe not even using Kafka ;) -- Raphael Mazelier Sent from Proton Mail Android Original Message On 1/7/25 23:25, karan

Re: HashiCorp config provider for Kafka Connect

2025-02-06 Thread Raphael Mazelier
We do this at work but with custom "tooling". It works this way. We create env variables from secret created by external secret operator (read from vault). Then we add an pre-init shell script that made the substitution where needed. Best On 06/02/2025 13:30, Prateek Kohli wrote: > Hi, > > I a

Warn error message after partial upgrade to 3.9 / upgrade catastrophic failure

2024-12-12 Thread Raphael Mazelier
ues appreciated. Best -- Raphael Mazelier

Re: How does Kafka write to the pagecache/pagetable ?

2025-03-06 Thread Raphael Mazelier
> Then it's just a normal file write where Kafka does not have any control. > Am I correct? Yes. > I am just wondering why then put in the documentation: > "Kafka writes to the pagecache" Indeed Kafka makes marketing on top of standard OS features. > I feel that it's misleading as to what it ac

Re: How does Kafka write to the pagecache/pagetable ?

2025-03-11 Thread Raphael Mazelier
Kafka didn't write to the pagecache. Actually no-one can only the underlying OS. -- Raph On 06/03/2025 14:03, Sreyan Chakravarty wrote: > Hi Guang, > > Thanks so much for the quick response. > > However I have one question, a lot of material on the web says Kafka > writes to the pagecache. > > T

Re: Kafka Connect on Kubernetes: Statefulset vs Deployment

2025-06-15 Thread Raphael Mazelier
I created the docker+kube stuff for our kafka-connect at my current job. I use standard deployment. kafka-connect doesn't care of hostname or IP. The sole trick is to inject the connector configuration at runtime (if you want). -- Raph On 14/06/2025 2:12 pm, Prateek Kohli wrote: > Hi All, > >