Thanks for sharing Radek, great article. Michael
> On 17 Sep 2016, at 21:13, Radoslaw Gruchalski <ra...@gruchalski.com> wrote: > > Please read this article: > https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying > > – > Best regards, > Radek Gruchalski > ra...@gruchalski.com > > > On September 17, 2016 at 9:49:43 PM, kant kodali (kanth...@gmail.com) wrote: > > Still it should be possible to implement using reactive streams right. > Could you please enlighten me on what are the some major differences you > see > between a commit log and a message queue? I see them being different only > in the > implementation but not functionality wise so I would be glad to hear your > thoughts. > > > > > > > On Sat, Sep 17, 2016 12:39 PM, Radoslaw Gruchalski ra...@gruchalski.com > wrote: > Kafka is not a queue. It’s a distributed commit log. > > > > > – > > Best regards, > > Radek Gruchalski > > ra...@gruchalski.com > > > > > > > > On September 17, 2016 at 9:23:09 PM, kant kodali (kanth...@gmail.com) > wrote: > > > > > Hmm...Looks like Kafka is written in Scala. There is this thing called > > reactive > > streams where a slow consumer can apply back pressure if they are consuming > > slow. Even with Java this is possible with a Library called RxJava and > > these > > ideas will be incorporated in Java 9 as well. > > I still don't see why they would pick poll just to solve this one problem > > and > > compensating on others. Poll just don't sound realtime. I heard from some > > people > > that they would set poll to 100ms. Well 1) that is a lot of time. 2) > > Financial > > applications requires micro second latency. Kafka from what I understand > > looks > > like has a very high latency and here is the article. > > http://bravenewgeek.com/dissecting-message-queues/ I usually don't go by > > articles but I ran my own experiments on different queues and my numbers > > are > > very close to this article so I would say whoever wrote this article has > > done a > > good Job. 3) poll does generate unnecessary traffic in case if the data > > isn't > > available. > > Finally still not sure why they would pick poll() ? or do they plan on > > introducing reactive streams?Thanks,kant > > > > > > > > > > > > > > > > > > > > On Sat, Sep 17, 2016 5:14 AM, Radoslaw Gruchalski ra...@gruchalski.com > > wrote: > > I'm only guessing here regarding if this is the reason: > > > > > Pull is much more sensible when a lot of data is pushed through. It allows > > consumers consuming at their own pace, slow consumers do not slow the > > complete > > system down. > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Rad > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 17, 2016 at 11:18 AM +0200, "kant kodali" <kanth...@gmail.com> > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > why did Kafka choose pull instead of push for a consumer? push sounds like > > it > > > > > is more realtime to me than poll and also wouldn't poll just keeps polling > > even > > > > > when they are no messages in the broker causing more traffic? please > > enlighten > > > > > me
