Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

2019-03-31 Thread akuterin
Hello! I ask for your help in connection with the my recent task: - Price lists are delivered to 20,000 points of sale with a frequency of <10 price lists per day. - The order in which the price lists follow is important. It is also important that the price lists are delivered to the point of

Re: Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

2019-03-31 Thread Hans Jespersen
Doesn’t every one of the 20,000 POS terminals want to get the same price list messages? If so then there is no need for 20,000 partitions. -hans > On Mar 31, 2019, at 7:24 PM, wrote: > > Hello! > > > > I ask for your help in connection with the my recent task: > > - Price lists are delive

Re: Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

2019-03-31 Thread Peter Bukowinski
I don’t want to be a downer, but because kafka is relatively new, the reference material you seek probably doesn’t exist. Kafka is flexible and can be made to work in many different scenarios — not all of the ideal. It sounds like you’ve already reached a conclusion that kafka is the wrong solu

Re: Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

2019-03-31 Thread Steve Howard
Do you have location specific SKU pricing? On Sun, Mar 31, 2019, 7:25 PM Hello! > > > > I ask for your help in connection with the my recent task: > > - Price lists are delivered to 20,000 points of sale with a frequency of > <10 > price lists per day. > > - The order in which the price lists fol

Re: Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

2019-03-31 Thread Alexander Kuterin
Yes, we have location specific SKU pricing. пн, 1 апр. 2019 г., 4:08 Steve Howard : > Do you have location specific SKU pricing? > > On Sun, Mar 31, 2019, 7:25 PM > > Hello! > > > > > > > > I ask for your help in connection with the my recent task: > > > > - Price lists are delivered to 20,000 p

Re: Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

2019-03-31 Thread Alexander Kuterin
Thanks, Hans! We use location specific SKU pricing and send specific price lists to the specific POS terminal. пн, 1 апр. 2019 г., 3:01 Hans Jespersen : > Doesn’t every one of the 20,000 POS terminals want to get the same price > list messages? If so then there is no need for 20,000 partitions. >

Re: Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

2019-03-31 Thread Hans Jespersen
Yes but you have more than 1 POS terminal per location so you still don't need 20,000 partitions. Just one per location. How many locations do you have? In doesn’t matter anyway since you can build a Kafka cluster with up to 200,000 partitions if you use the latest versions of Kafka. https://bl