one of bootstrap.servers unavailable causes producer error

2018-04-08 Thread 许志峰
In the Kafka official document you may want more than one, though, in case a server is down But when there are invalid brokers in the bootstrap.servers, the producer may fail! I Set up one broker 10.142.33.51:9092 on CentOS 7, using kafka 0.10.1.0. My Producer is like Properties props = new Pro

Re: join 2 topic streams --> to another topic

2018-04-08 Thread Ted Yu
The blog is an interesting reading material. There was a minor typo: bq. an event arrives for either the left of right input stream 'of' above should be 'or'. Cheers On Sun, Apr 8, 2018 at 2:04 PM, Matthias J. Sax wrote: > Check out this blog post that explain how the different joins work: >

Re: join 2 topic streams --> to another topic

2018-04-08 Thread Matthias J. Sax
Check out this blog post that explain how the different joins work: https://www.confluent.io/blog/crossing-streams-joins-apache-kafka/ It's hard to give a general answer -- it depends on the context of your application. Are keys unique? Do you want to get exactly one result or should a single stoc

join 2 topic streams --> to another topic

2018-04-08 Thread adrien ruffie
Hello all, I have 2 topics streamed by KStream and one KStream I want to merge both object's informations (Stock & Dividend) and send to another topic with for example The key of 2 two topic is the same. I need to use, leftJoin, merge, KTable, ... what is the best solution ? What do you

Re: Scaling up kafka consumer applications(kafka streams)

2018-04-08 Thread Matthias J. Sax
It depend on the concrete program you write... Most likely it's 4, but hard to say without more information. If you read all 4 topics as a single stream (i.e. pattern subscription) it should be 4. If you repartitions data in your applications later, it might be more thought. -Matthias On 4/8/1

Scaling up kafka consumer applications(kafka streams)

2018-04-08 Thread Shivam Sharma
Hi, Lets say I have one consumer group which subscribed to *4 topics* and partitions for each topics are:- 1. First topic => 4 partitions 2. Second topic => 3 partitions 3. Third topic => 2 partitions 4. Fourth topic => 1 partitions Total number of *partitions = 10*. So total how man