Re: Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-07 Thread Andrew Stevenson
Hi Khurrum, Is ready now. https://github.com/Landoop/stream-reactor Regards Andrew From: Khurrum Nasim Sent: Thursday, 7 December, 08:36 Subject: Re: Comparing Pulsar and Kafka: unified queuing and streaming To: dev@kafka.apache.org Cc: us...@kafka.apache.org Andrew, Thank you! Is there any

Re: Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-06 Thread Khurrum Nasim
nctionality to be > added to > > > Kafka - like transactional messages, more complex acks, centralized > > > consumer management, etc. > > > > > > If you really need those feature - there are other systems that are > > > designed for that.

Re: Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-06 Thread Andrew Stevenson
ded - but maybe those that require > > major changes to the Kafka's core principles should go into separate > > frameworks, plug-ing (like Connectors) or something in that line, rather > > that packing it all into the Core Kafka. > > > > Just my

Re: Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-05 Thread Khurrum Nasim
nsumer management, etc. > > > > > > If you really need those feature - there are other systems that are > > > designed for that. > > > > > > I truly worry that if all those changes are added to Core Kafka - it > will > > > become just another

Re: Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-05 Thread Jason Gustafson
gt; increased > > complexity (and, thus, management and possibility of bugs). Sure, there > has > > to be innovation and new features added - but maybe those that require > > major changes to the Kafka's core principles should go into separate > > frameworks, plug-ing

Re: Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-05 Thread Khurrum Nasim
te > frameworks, plug-ing (like Connectors) or something in that line, rather > that packing it all into the Core Kafka. > > Just my 2 cents :) > > Marina > > Sent with [ProtonMail](https://protonmail.com) Secure Email. > > > Original Message

Re: Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-05 Thread Marina Popova
g (like Connectors) or something in that line, rather that packing it all into the Core Kafka. Just my 2 cents :) Marina Sent with [ProtonMail](https://protonmail.com) Secure Email. > Original Message -------- > Subject: Re: Comparing Pulsar and Kafka: unified queuing and streami

Re: Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-04 Thread Khurrum Nasim
Thank you, Jason. Nice to see inputs from you about messaging retention. I have the same feeling that a centralized place for managing offsets (what Pulsar calls cursors) is actually much better than current kafka model. Would love to hear what is the plan for kafka community heading towards on thi

Re: Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-04 Thread Jason Gustafson
Hi Khurrum, Thanks for sharing the article. I think one interesting aspect of Pulsar that stands out to me is its notion of a subscription and how it impacts message retention. In Kafka, consumers are more loosely coupled and retention is enforced independently of consumption. There are some scena

Comparing Pulsar and Kafka: unified queuing and streaming

2017-12-03 Thread Khurrum Nasim
Dear Kafka Community, I happened to read this blog post comparing the messaging model between Apache Pulsar and Apache Kafka. It sounds interesting. Apache Pulsar claims to unify streaming (kafka) and queuing (rabbitmq) in one unified API. Pulsar also seems to support Kafka API. Have anyone taken