I have an application where I will be getting some Time Series data which I
am feeding to Kafka and Kafka in turn is giving data to Storm for doing
some real time processing.
Now one of my use case is that there might be certain lag in my data. For
an example: I might not get all the data for 2:00
How can I make multiple producers to write data? I have written a producer
that produces some data for 15 seconds on a single machine setup. Now when
I run another instance of same producer it says the port is in use (which
is natural as I think the first producer is sending data using TCP). So it
Millions of messages per day (with each message being few bytes) is not
really 'Big Data'. Kafka has been tested for a million message per second.
The answer to all your question IMO is "It depends".
You can start with a single instance (Single machine installation). Let
your producer send messag
but may apply to the producers too.
>
> [2013-06-28 14:07:19,653] ERROR Connection attempt to 0:0 failed, next
> attempt in 1 ms (kafka.producer.SyncProducer)
> java.net.ConnectException: Connection refused
>
> Thanks,
>
> Jun
>
>
> On Sat, Jul 6, 2013 at 3:30 PM
lection.immutable.Stream.foreach(Stream.scala:254)
> > at
> >
>
kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:70)
> > at
> > kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:41)
Regards,
Yavar
On Thu, Jul 4, 2013 a
e out the data? What
directories do I need to visit?
Thanks,
Yavar
On Mon, Jul 1, 2013 at 9:13 PM, Jun Rao wrote:
> You need to wipe out both the ZK data and the Kafka data from 0.8, in order
> to try 0.7.
>
> Thanks,
>
> Jun
>
>
> On Sun, Jun 30, 2013 at 11:28 PM, Ya
Kafka 0.8 works great. I am able to use CLI as well as write my own
producers/consumers!
Checking Zookeeper... and I see all the topics and partitions created
successfully for 0.8.
Kafka 0.7 does not work!
Why Kafka 0.7? I am using Kafka Spout from Storm which is made for Kafka
0.7.
First I jus
