Re: High CPU on kafka nodes

Consumers don't look too busy. Couple java apps and a spark cluster. Java boxes are not busy, and spark is hard to judge as it does shuffling, so network is high, but CPU is low. But thank you for the hint, will check! Alex On Thu, Nov 29, 2018 at 10:31 PM Johan Rask wrote: > What about CPU an

Re: High CPU on kafka nodes

What about CPU and load on your consumers? We had a similar problem with high cpu and high network traffic on our Kafka cluster and high cpu and load and traffic on our consumer machines. Thiis was caused by la to ow setting on kafka consumer fetch.max.wait.ms which was only 5 ms. Setting this to 1

Re: Please explain Rest API

Hi Sönke, when topic got created so i tried to read the topic data using console consumer but it didn't worked out consumer didn't consumer a single message. i read the log and got this every time. i don't understand where m i making mistake. [\!�qg_z��g_z�(�"{"error":{"name":"Er

Re: High CPU on kafka nodes

+1 We too face similar issue sometime. I am interested to know more about it. On Thursday, November 29, 2018, Alexander Filipchik wrote: > Hello! > > I inherited Kafka cluster which runs on AWS (I3.4xl instances). > Each node ingests around 8k messages per second. > Instance stats: > Network in

Re: kafka-streams batch restore state

`kill -9` will force-stop the application the hard way, thus, it cannot write a checkpoint file that it needs for a clean restart. Just you should use `kill` (SIGTERM, not SIGKILL == -9) to tell Kafka Streams to shutdown gracefully. This will allow Kafka Streams to write a checkpoint file. This wa

Re: kafka-streams batch restore state

https://cwiki.apache.org/confluence/display/KAFKA/KIP-116%3A+Add+State+Store+Checkpoint+Interval+Configuration This kafka KPI helped me. I use ‘kill -9' to stop my application. Thanks 原始邮件 发件人:Matthias J. saxmatth...@confluent.io 收件人:usersus...@kafka.apache.org 发送时间:2018年11月30日(周五) 10:30 主题:Re:

Re: kafka-streams batch restore state

Not sure. Maybe you can provide a larger part of the log? Maybe in DEBUG level? -Matthias On 11/28/18 10:08 PM, meigong.wang wrote: > When I restart my kafka-streams application, sometimes I can see following > log： > > > [Consumer > clientId=market-kline-stream-wmg-ticker-d891eeaf-2932-45b

Re: Please explain Rest API

Hi Satendra, if I understand correctly you are using the RestSourceConnector to pull data into a Kafka topic and that seems to work - at least as far as the topic getting created. But are you saying that you tried to read data with 'cat filename.log' on a file within your kafka data directory? I

Re: Please explain Rest API

Hi Sönke, I have single node kafka cluster, i have setup web server locally, many apps are sending logs to this server. I have Rest API which send/receive client request to/from server. here is my connector configuration : curl -i -X POST -H "Accept:application/json" -H "Content-Type: applicati

Re: Please explain Rest API

Hi Satendra, feel free to share the error and I'll see if I can help. Best regards, Sönke Am Do., 29. Nov. 2018, 14:59 hat Satendra Pratap Singh geschrieben: > Hi Sönke, > > Thanks for your response. The reason for asking about rest bcoz I m stack > while pulling pulling request from rest api

Re: Data loss - Compacted topic behind reduce function

*Update in case it is relevant for someone:* It seems that the problem is related to the "Too many open files" errors we were getting. We changed the open files limitations on our instances and it looks like the problem is gone. Thanks On Wed, Nov 28, 2018 at 2:32 PM Nitay Kufert wrote: > It ju

Re: Please explain Rest API

Hi Sönke, Thanks for your response. The reason for asking about rest bcoz I m stack while pulling pulling request from rest api. I have been using Kafka rest source connect to pull the request, connected is pulling some error instead actual data. Would you like to help me. I will share that e

Re: Will KafkaFuture becomes CompletableFuture ?

That might be the case, yes. I've fixed the title, maybe that saves the next poor soul some trouble :) On Thu, Nov 29, 2018 at 2:12 PM Benoit Maggi | AT Internet < benoit.ma...@atinternet.com> wrote: > Hi Sönke, > > Many thanks, that was exactly what I was looking for 😊 > > I didn't found this i

RE: Will KafkaFuture becomes CompletableFuture ?

Hi Sönke, Many thanks, that was exactly what I was looking for 😊 I didn't found this issue (probably due to the typo in the title) Best Regards, Benoit -Message d'origine- De : Sönke Liebau Envoyé : jeudi 29 novembre 2018 13:02 À : users@kafka.apache.org Objet : Re: Will KafkaFuture

Re: Please explain Rest API

Hi Satendra, in addition to what Ryanne said, maybe a graphical REST tool would better suit your needs, there is a couple of good ones out there: Postman - https://www.getpostman.com/ Insomnia - https://insomnia.rest/ are two that I have used myself, but just googling for "rest gui" should turn u

Got 2 mins for a quick favor?

Hey, can you sign this petition? http://chng.it/rYg8wc9dsC Thanks! - Shashank

Re: Will KafkaFuture becomes CompletableFuture ?

Hi Benoit, have you looked at KAFKA-6987 ? That seems to me to be the one that had the most recent activity around this. You probably found this as well but it wasn't mentioned in your mail, so I thought I'd point it out. Best regards, Sönke On T

Will KafkaFuture becomes CompletableFuture ?

Hi everyone, I have a use case, where a KafkaCompletableFuture would be useful. As far as I can tell KafkaFuture only implements Future < public abstract class KafkaFuture implements Future > [1] I found this KIP suggesting the change to CompletableFuture < When KIP-118