Re: Mirror Maker tool is not running

Looks like you are hitting : https://issues.apache.org/jira/browse/KAFKA-6947 or https://jira.apache.org/jira/browse/KAFKA-6177 --Senthil On Mon, May 6, 2019 at 6:32 PM ASHOK MACHERLA wrote: > Dear Team Members > > > > Please find these below configurations for mirror maker tool scripts > > >

RE: Mirror Maker tool is not running

Dear Senthil In that links , there is no proper information about to fix this issue. Could you please tell me what configuration required in source and target sides Sent from Mail for Windows 10 From: SenthilKu

Setting log.dir to a tmpfs

Hello, I'm currently running a Kafka Cluster with a retention policy of 30mins and 10min log segments. In practice, this uses less than 20GB of file system space. Thus, I've been thinking about having no persistent storage at all. In the worst case, I do not mind losing the last 30min of historic

Re: Best Practice Scaling Consumers

Hi Morritz - I don’t believe the number of Kafka consumers is restricted to the number of partitions. When you create a topic - and indicate both the number of partitions and a key - it causes your key value pairs to be allocated to a specific partition on the basis of a hash function on the

LogCleaner is not removing Transaction Records

Hi, We are using Kafka Streams with exactly-once enabled on a Kafka cluster for a while. Recently we found that the size of __consumer_offsets partitions grew huge. Some partition went over 30G. This caused Kafka to take quite long to load "__consumer_offsets" topic on startup (it loads the topic

Re: Mirror Maker tool is not running

Shallow iteration should be enabled for the consumer, not producer. Ryanne On Mon, May 6, 2019, 8:02 AM ASHOK MACHERLA wrote: > Dear Team Members > > > > Please find these below configurations for mirror maker tool scripts > > > > *source-cluster.config* > > > > bootstrap.servers=192.168.175.12

Confluent Center: KSQL query shows less results than when running from KSQL CLI

Hi! I'm running the same query from KSQL CLI and from the Confluent Control Center (KSQL Development). I set 'auto.offset.reset' = 'latest'; to only use latest events. Once I send 20 test events - I get full (correct) results from this query in KSQL CLI - but only partial results in the Control

KSQL CLI: cannot execute script with SELECT query (but can execute other scripts)

Hi, one more question on KSQL CLI: using "run script /myScript.ksql" commands I can run scripts with "CREATE STREAM ..." and "CREATE TABLE ..." commands - no problem. However, when I try to run a script with a SELECT statement, I'm getting an error: Script: top3_reqtime.ksql SELECT payload->sit

Re: Changing tumbling windows inclusion

Hi Alessandro, Interesting. I agree, messing with the record timestamp to achieve your goal sounds too messy. It should be pretty easy to plug in your own implementation of Windows, instead of using the built-in TimeWindows, if you want slightly different windowing behavior. Does that work for y

Re: Changing tumbling windows inclusion

Hi John, on Slack Matthias suggested to have my own transform to window the data myself, I'll have a look into it and the Windows implementation as you suggested and see what I can do! Thanks for the advice! -- Alessandro Tagliapietra On Tue, May 7, 2019 at 8:45 AM John Roesler wrote: > Hi A

Re: InvalidStateStoreException on KStream join using GlobalKtables

Hello Ishita, Is it consistently reproducing? And which Kafka version are you using? Guozhang On Thu, May 2, 2019 at 5:24 PM Ishita Rakshit wrote: > Hi, > I have a Kafka Streams application where I am joining a KStream that reads > from "topic1" with a GlobalKTable that reads from "topic2" a

Regarding kafka memory not distributed across cluster

Hi all, We are having a query regarding memory consumption on kafka scale out. It would be very helpful if you can give suggestion/solution for the below query. We are running kafka as docker container on kubernetes. Memory limit of 4GiB is configured for Kafka broker POD. With some large load

Topic Creation two different config(s)

Hello Experts , We have a requirement to create topic dynamically with two different config(s). Is this possible in Kafka ? Kafka Version : 2.2.0 Topics with different settings: #1 - Set retention as 24 hours for free tier customers # 2 - Set retention as 72 hours for paid customers Note : Right