Re: Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

Do you have location specific SKU pricing? On Sun, Mar 31, 2019, 7:25 PM Hello! > > > > I ask for your help in connection with the my recent task: > > - Price lists are delivered to 20,000 points of sale with a frequency of > <10 > price lists per day. > > - The order in which the price lists fol

Re: Kafka memory estimation

Hi Rammohan, How are you measuring "Kafka seems to be reserving most of the memory"? Thanks, Steve On Thu, Apr 11, 2019 at 11:53 PM Rammohan Vanteru wrote: > Hi Users, > > As per the article here: > https://docs.confluent.io/current/kafka/deployment.html#memory memory > requirement is roughl

Re: SELECT * from table shows no data until new messages come in

Hi Marina, Try... set 'auto.offset.reset'='earliest'; Thanks, Steve On Mon, May 6, 2019 at 4:40 PM Marina Popova wrote: > Hi, > Trying to understand if I'm missing something... > I'm using Confluent KSQL setup (running all components in Docker > containers). I've created a Table from a Str

Re: SELECT * from table shows no data until new messages come in

Sorry, I hit send too quickly. On Mon, May 6, 2019 at 4:42 PM Steve Howard wrote: > Hi Marina, > > Try... > > set 'auto.offset.reset'='earliest'; > > > Thanks, > > > Steve > > On Mon, May 6, 2019 at 4:40 PM Marina Popova > wr

Re: Can kafka brokers have SASL_SSL, SSL and PLAINTEXT listener at same time?

Hi Shrinivas, Yes, you just have each listen on a different port.and change your clients to point to the one you wish to use. Thanks, Steve On Thu, May 30, 2019, 6:57 AM Shrinivas Kulkarni < shrinivas.n.kulka...@gmail.com wrote: > Hi, > > I want to migrate from SSL to SASL_SSL on the Kafka. Is

Re: Kafka broker server high memory utilization

Hi Parth, Yes, this is by design. See the following URL, paying particularly close attention to the section on memory. https://docs.confluent.io/current/kafka/deployment.html Thanks, Steve On Tue, Jun 18, 2019 at 6:29 AM Parth Gandhi < parth.gan...@excellenceinfonet.com> wrote: > Hi, > > I a

Re: Kafka Topic Partition Consumer Lags

Hi Garvit, Are the slow partitions "hot", i.e., receiving a lot more messages than others? Thanks, Steve On Wed, Jun 26, 2019, 9:56 AM Garvit Sharma Just to add more details, these consumers are processing the Kafka events > and writing to DB(fast write guaranteed). > > On Wed, Jun 26, 2019 at

Re: Kafka Topic Partition Consumer Lags

t; wrote: > > >> > > > > >> > > Can anyone please help me with this. > > >> > > > > >> > > On Wed, Jun 26, 2019 at 8:56 PM Garvit Sharma < > eng.gar...@gmail.com > > > > > >> >

Re: Consumer commit logic

"I'm just saying that a message which processing throws an exception is gone within minutes." Is the message no longer in the topic or is your consumer group current offset just higher than the offset of the message in question? On Wed, Sep 25, 2019 at 7:38 PM Alessandro Tagliapietra < tagliapiet

Re: Schema validation in JDBC Connector

Hi VIshnu, Is there a reason you wouldn't want to allow the connector to handle the schema registration and evolution over time? The assumption in most cases is that the database is the source of truth. If you register the schema yourself in the schema registry, you are introducing a tight coupl

Re: Producer batch size

Hi Dhirenda, As long as buffer.memory (default 32MB) has space, the producer will continue to write here. If that is exhausted, eventually the producer will throw... org.apache.kafka.common.errors.TimeoutException: Failed to allocate memory within the configured max blocking time 6 ms The 6

Re: Producer batch size

can be > greater than batch.size upto buffer.memory if the app is sending data > faster than the producer i/o thread can send to broker. > In this situation buffer.size becomes irrelevant. no ? > > Thanks, > Dhirendra. > > On Wed, Dec 23, 2020 at 11:36 PM Steve Howard > wrote: &

Re: Kafka Consumer Consumption based on TimeStamp-based position

Hi, Yes, you can use the offsetsForTimes() method. See below for a simple example that should get you started... import org.apache.kafka.clients.consumer.*; import org.apache.kafka.common.config.ConfigException; import org.apache.kafka.common.*; import java.io.*; import java.time.Duration; impor

Re: Kafka and micro-batch processing dilemma

tps://urldefense.proofpoint.com/v2/url?u=https-3A__www.confluent.io&d=DwMFaQ&c=A-GX6P9ovB1qTBp7iQve2Q&r=Utd0DOMevK4PhTyYQyXeCfximjd_X7dLCRZ4fCszhKE&m=KhvsB9pYz4VOEVRjuZQ3wAnqMS0DBvLknGtl87MPHqw&s=jJxjUduPXUSJdg-LRvxdNTjalgtutz6vQV_Jvzgtt3k&e=> *Steve Howard* Senior Solutions Engineer

Re: JDBC Sink connector

Hi Charandeep, What is in the connect logs?  That generally has a full call stack. On 10/10/2022 7:11 AM, Singh, Charandeep wrote: Hi whenever I try to create a new connector it gets timed outcurl -s -X POST -H "Accept:application/json" -H "Content-Type:application/json" -d @connector-config.