Re: Kafka JMX monitoring

2020-03-19 Thread Gioacchino Vino
Hi, I would like to share my experience with you. Even if I found out that prometheus is the a simple and fast solution, since I'm using InfluxDB (and Telegraf) in my project, I'm using the chain: Jolokia -> Telegraf -> InfluxDB -> Grafana Some useful links: https://jolokia.org/ https://

Running stateful Kafka Stream Application on topic with multiple partitions

2019-11-15 Thread Gioacchino Vino
Hi expert, I don't understand a kafka behavior and I'm here to ask for explanation. My processing task is pretty simple and it's quite similar to a change-log one. The record value contains a key/value pair: if the new value is different respect the stored one, forward to the output topic a

Re: Kafka Streams - Java StreamException - Serdes issue

2019-04-10 Thread Gioacchino Vino
.Double())... (I'm assuming your keys are of type String here) it should solve your issue. HTH, Bill On Tue, Apr 9, 2019 at 1:14 PM Gioacchino Vino wrote: Hi experts, I believe to understand there is the need to set the serde for the Double type after/in the map function for a re-parti

Kafka Streams - Java StreamException - Serdes issue

2019-04-09 Thread Gioacchino Vino
Hi experts, I believe to understand there is the need to set the serde for the Double type after/in the map function for a re-partition task. I can't figure out where to specified. I've already tried to find the answer on documentation and article but I failed. The following code KStream

Re: Configuration guidelines for a specific use-case

2019-01-09 Thread Gioacchino Vino
I would like to improve it just a bit. Do you think using 6 partitions in a 3 broker cluster is a good choice? Gioacchino On 08/01/2019 18:52, Ryanne Dolan wrote: Latency sounds high to me, maybe your JVMs are GC'ing a lot? Ryanne On Tue, Jan 8, 2019, 11:45 AM Gioacchino Vino Hi expert,

Configuration guidelines for a specific use-case

2019-01-08 Thread Gioacchino Vino
Hi expert, I would ask you some guidelines, web-pages or comments regarding my use-case. *Requirements*: - 2000+ producers - input rate 600k messages/s - consumers must write in 3 different databases (so i assume 3 consumer groups) at 600k messages/s overall (200k messages/s/database)