Ha ha, yes, exactly, you need a database. Kafka is a wonderful tool, but
not the right one for a job like that.
On Sun, Oct 6, 2013 at 7:03 PM, Ravindranath Akila <
ravindranathak...@gmail.com> wrote:
> Actually, we need a broker. But a more stateful one. Hence the decision to
> use TTL on hbase
Actually, we need a broker. But a more stateful one. Hence the decision to
use TTL on hbase.
On 7 Oct 2013 08:38, "Benjamin Black" wrote:
> What you are discovering is that Kafka is a message broker, not a database.
>
>
> On Sun, Oct 6, 2013 at 5:34 PM, Ravindranath Akila <
> ravindranathak...@gm
What you are discovering is that Kafka is a message broker, not a database.
On Sun, Oct 6, 2013 at 5:34 PM, Ravindranath Akila <
ravindranathak...@gmail.com> wrote:
> Thanks a lot Neha!
>
> Actually, using keyed messages(with Simple Consumers) was the approach we
> took. But it seems we can't ma
Thanks a lot Neha!
Actually, using keyed messages(with Simple Consumers) was the approach we
took. But it seems we can't map each user to a new partition due to
Zookeeper limitations. Rather, we will have to map a "group" of users on
one partition. Then we can't fetch the messages for only one use
Kafka is designed to have of the order of few thousands of partitions
roughly less than 10,000. And the main bottleneck is zookeeper. A better
way to design such a system is to have fewer partitions and use keyed
messages to distribute the data over a fixed set of partitions.
Thanks,
Neha
On Oct 5
