Re: Kafka 2.7.2 to 3.5.1 upgrade

2023-12-04 Thread Lud Antonie
Hi, Just rerun the upgrade from 2.7.2 to 3.5.1 and got the same under replicated issue: kafka@playground2:~/kafka/bin$ ./kafka-topics.sh --describe --under-replicated-partitions --bootstrap-server pg1:30092 Topic: smo-test Partition: 3 Leader: 1 Replicas: 1,3,2 Isr: 3,2 So it looks like somethin

Re: Kafka 2.7.2 to 3.5.1 upgrade

2023-12-04 Thread Lud Antonie
Hi Megh, No, the number of partitions haven't increased. The upgrade is done in a test environment, the topic is just created just before the upgrade. As a test I performed a rollback to 2.7.2 and did an upgrade to 3.4.1. This upgrade went ok i.e. without under replicated partitions. Best regard

Do we have any mechanism to control requests per second for a Kafka connect sink?

2023-12-04 Thread Yeikel Santana
Hello everyone, Is there any mechanism to force Kafka Connect to ingest at a given rate per second as opposed to tasks? I am operating in a shared environment where the ingestion rate needs to be as low as possible (for example, 5 requests/second as an upper limit), and as far as I can tel

How do you know you are at the end of the partition?

2023-12-04 Thread Jesus Cea
I wonder how I can know I have processed all the messages in a partition?. What I currently do is: 1. Publish a "ping" message with a "nonce", 2. read the partition (for the last know processed offset) replacing the application state every time I get a new state message in the topic, 3. when I

Streams configurations to optimize keeping standby tasks within acceptable recovery lag

2023-12-04 Thread Sabit Nepal
Hello, We recently started observing our streams application is taking hours to days to rebalance following events where streams consumers experience churn, such as code deployments. We increased the acceptable.recovery.lag property from the default of 10,000 to 100,000, however, we are still expe

Re: How to get a X509 broker certificate with "openssl s_client"?

2023-12-04 Thread Jesus Cea
On 9/11/23 3:37, Jesus Cea wrote: I am trying to remotely access to the brokers certificates (for audit purposes, expiration alarms, etc) using this command: """ openssl s_client -showcerts -connect localhost:9092 """ The connection is correctly established, but something is wrong. The TLS se