Dears,
I'm looking for a way to get real-time updates using my service, I believe
kafka is the way to go but I still have an issue on how to use it.
My system gets data from devices using GPRS, I then read this data and analyze
it to check what action I should do afterwards. I need the analyzing step to be
as fast as possible. I was thinking of two options:
The first option is to gather all the data sent from all the devices into one
huge topic and then getting all the data from this topic and analyzing it. The
downside of this option is that the data analysis step is delaying my work
since I was to loop through the topic data, on the other hand the advantage is
that I have a manageable number of topics ( only 1 topic).
The other option is to divide the data I'm gathering into several small topics
by allowing each device to have its own topic, take into consideration that the
number of devices is large, I'm talking about more that 5000 devices. The
downside of this option is that I have thousands of topics, where the advantage
is that each topic will have a manageable amount of data allowing me to get my
analysis done in much more reasonable time.
Can you advise on what option is better and whether there is a third option
that I'm not considering,
Best Regards
Ola Bissani
Developer Manager
Easysoft
Mobile Lebanon : +961 3 61 16 90
Office Lebanon :+961 1 33 55 15/17
E mail: ola.biss...@easysoft.com.lb
web site:www.easysoft.com.lb
"Tailored to Perfection"
The information transmitted is intended only for the person or entity to which
it is addressed and it may contain proprietary, business-confidential, and/or
legally privileged information. If you are not the intended recipient of this
email you are hereby notified that any use, review, retransmission,
dissemination, distribution, reproduction or any other action taken in reliance
upon this email is strictly prohibited. If you have received this email in
error, please contact the sender and delete this email and its contents from
any computer. Any views expressed in this email are those of the individual
sender and may not necessarily reflect the views of the company.
Please consider the environmet before
printing this email.
-----Original Message-----
From: Wes Peng <pent...@gmail.com>
Sent: Thursday, December 23, 2021 10:11 PM
To: users@kafka.apache.org
Subject: Re: Kafka Topics
That depends on your resources such as ram, disk etc. General speaking there is
no problem.
Regards
> Dears,
>
> I'm new to using Kafka, and I was wondering up to how many topics can
> Kafka Handle. I'm trying to use Kafka but using it I will be obliged
> to create thousands of topics to keep up with my data. Will Kafka on
> my server handle this kind of data?
>
> Thank you,
>
>
> Best Regards
>
> Ola Bissani
>
> Developer Manager
>
> Easysoft
>
> Mobile Lebanon : +961 3 61 16 90
>
> Office Lebanon :+961 1 33 55 15/17
>
> E mail: ola.biss...@easysoft.com.lb
>
> web site:www.easysoft.com.lb
>
> "Tailored to Perfection"
>
>
>
>
> The information transmitted is intended only for the person or entity
> to which it is addressed and it may contain proprietary,
> business-confidential, and/or legally privileged information. If you
> are not the intended recipient of this email you are hereby notified
> that any use, review, retransmission, dissemination, distribution,
> reproduction or any other action taken in reliance upon this email is
> strictly prohibited.
> If you have received this email in error, please contact the sender
> and delete this email and its contents from any computer. Any views
> expressed in this email are those of the individual sender and may not
> necessarily reflect the views of the company.
>
> Please consider the environmet before printing this email.
>
>
