Hi Achintya,
1. Kafka can be used as a message store.
2. What is the message arrival rate per second?
3. What is the SLA for the messages to be processed?
4. If your messages arrive faster than they are consumed, you will get
a backlog of messages. In that case, you
Hi Shekar,
Alternatively, you could make each stage of your pipeline to write to a
Cassandra (or other DB) and your API will read from it. With Cassandra TTL, the
row will be deleted after TTL is passed. No manual cleanup is required.
Best regards / Mit freundlichen Grüßen / Sincères sal
Hi,
You can use Storm also, Here you have the option of rotating the file.
You can also write to Hive directly.
Best regards / Mit freundlichen Grüßen / Sincères salutations
M. Lohith Samaga
-Original Message-
From: Mudit Kumar [mailto:mudit.ku...@askme.in]
Sent: Wednesday, J
Hi
Using Storm would be another way. This will scale as well.
Spark streaming would fit as well
It all depends on the complexity of the filter and any additional processing
required.
HTH
Lohith
Sent from my Sony Xperia™ smartphone
Gerard Klijs wrote
Using kafka streams is one way
t; / Consumer groups have different purpose. If I need to setup three
> kafka instances each for App ?
>
> On Fri, Apr 22, 2016 at 12:30 PM, Lohith Samaga M <
> lohith.sam...@mphasis.com
> > wrote:
>
> > Hi,
> > It is better NOT to share topics among
Hi,
It is better NOT to share topics among applications. You may have a
wrapper application reading from the queue/topic and routing it to the correct
application, but it is simpler for each application to read from its own topic.
Best regards / Mit freundlichen Grüßen / Sincères salutat
Dear All,
After a system restart, the new consumer is working as expected.
Best regards / Mit freundlichen Grüßen / Sincères salutations
M. Lohith Samaga
-Original Message-
From: Lohith Samaga M [mailto:lohith.sam...@mphasis.com]
Sent: Tuesday, April 12, 2016 17.00
To: users
-
org.apache.kafka.clients.consumer.internals.SendFailedException
So, is it advisable to use the old consumer on Kafka 0.9.0.1?
Please help.
Thanks in advance.
Best regards / Mit freundlichen Grüßen / Sincères salutations
M. Lohith Samaga
-Original Message-
From: Lohith Samaga M
org.apache.kafka.clients.consumer.internals.SendFailedException in
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.RequestFutureCompletionHandler.
After that, the consumer goes into a loop.
Best regards / Mit freundlichen Grüßen / Sincères salutations
M. Lohith Samaga
-Original Message-
From: Lohith Samaga M
compacted topics on Windows and the new consumer uses a compacted topic to
store offsets.
Ismael
On Mon, Apr 4, 2016 at 12:20 PM, Lohith Samaga M
wrote:
> Dear All,
> The error seems to be NOT_COORDINATOR_FOR_GROUP.
> The exception
Thanks Niko!
I think I missed an
org.apache.kafka.clients.consumer.internals.SendFailedException exception at
the very beginning (or atleast it is giving an exception today).
Even after using a new install of Kafka, I get the same errors. Strangely, all
topics are re-created in the logs. I can
threads;
- If my producer producing, then why the consumer couldn't consume? it
stuck @ poll()
- Consumer thread is waiting forever, not returning any objects
I think new APIs are recommended.
On 4 April 2016 at 16:37, Lohith Samaga M wrote:
> Thanks for letting me know.
>
essage-
> From: Ratha v [mailto:vijayara...@gmail.com]
> Sent: Monday, April 04, 2016 11.27
> To: users@kafka.apache.org
> Subject: Re: New consumer API waits indefinitely
>
> I too face same issue:(
>
> On 4 April 2016 at 15:51, Lohith Samaga M
> wrote:
>
> > HI,
, 2016 11.27
To: users@kafka.apache.org
Subject: Re: New consumer API waits indefinitely
I too face same issue:(
On 4 April 2016 at 15:51, Lohith Samaga M wrote:
> HI,
> Good morning.
>
> I am new to Kafka. So, please bear with me.
> I a
HI,
Good morning.
I am new to Kafka. So, please bear with me.
I am using the new Producer and Consumer API with Kafka 0.9.0.1
running on Windows 7 laptop with zookeeper.
I was able to send messages using the new Producer API. I can
15 matches
Mail list logo
