How to integrate storm into a web service?

Hi all, I would like to implement a topology where the spout receives http request. Is there a code sample that can be a good starting point for my implementation? Is kafka designed for this use case? Thanks for your help. D. Debarbieux --- L'absence de virus dans ce courrier électronique a ét

Re: How to integrate storm into a web service?

Denis, I suggest its better to have your http requests going to kafka and than use Storm's KafkaSpout to process. This allow you to not loose any events as KafkaSpout can do replays of the message incase if there is a failure in your topology. -Harsha On Mon, Jan 26

Kafka Spout Drops messages when kafka broker fails, does not failover to kafka replicas

Hello: I have a three node kafka cluster with a single topic and a topic replication factor of 3. I ran a test where I inserted a few hundred messages into kafka. While the topology was reading these messages, I killed one of the brokers. My hope was that the kafka spout would simply use one of t

trident partitionBy dramatically low down the thoughput？

As I tested in local mode，with or without partitionBy, the throughputs are quit different, I saw partitionBy dramatically low down the throughput. That surprised me, as I thought partitionBy may cause higher latency but should do nothing with throughput. Is that normal or I should check my test

Re: Kafka Spout Drops messages when kafka broker fails, does not failover to kafka replicas

three node kafka cluster with replication factor of 3 is bad design. It should always be less than the cluster size. Please change it to 2 and try again. -Manoj On Mon, Jan 26, 2015 at 7:19 AM, Milad Fatenejad wrote: > Hello: > > I have a three node kafka cluster with a single topic and a t

Re: Kafka Spout Drops messages when kafka broker fails, does not failover to kafka replicas

Hello: I reran my test with a replication factor of 2 but encountered the same issue...any other suggestions? Thanks Milad On Mon, Jan 26, 2015 at 1:06 PM, Manoj Jaiswal wrote: > three node kafka cluster with replication factor of 3 is bad design. > > It should always be less than the cluster

Re: Kafka Spout Drops messages when kafka broker fails, does not failover to kafka replicas

Milad, Can you share your kafkaSpout config. -Harsha On Mon, Jan 26, 2015, at 01:31 PM, Milad Fatenejad wrote: > Hello: > > I reran my test with a replication factor of 2 but encountered the > same issue...any other suggestions? > > Thanks Milad > > On Mon, Jan 26, 2015 at 1:06 PM, Manoj Jaiswal

Re: Kafka Spout Drops messages when kafka broker fails, does not failover to kafka replicas

Hello Harsha: I am mainly just using the default settings... BrokerHosts zk = new ZkHosts(zkConnect); SpoutConfig spoutConfig = new SpoutConfig(zk, kafkaTopic, "/kafkaStorm", spoutComponentId); spoutConfig.scheme = new SchemeAsMultiScheme(new StringScheme()); Kafka

Storm-deploy fail due to ganglia setup error

Hello storm-users, I had deployed successfully couple times to AWS ec2 instance. But since last friday, when I try to setup a new cluster, I keeping failing due to Ganglia setup problem. I’ve looked https://groups.google.com/forum/#!topic/storm-user/xRB1zMwT-fY this report. But Ben’s github

worker connection to zookeeper suspended

Hi, i have a single node zookeeper and 2 node storm having 1 worker on each node. one of the worker stops and start on other node so at this point i ll be having two workers on same node and rebalancing happens only if i manually do it from storm UI. Here are the logs 2015-01-24 06:34:59 o.a.z.

What happpens when a Task fails in storm?

Hi all, Can someone please clarify what action does supervisor takes when a certain tasks fails within a worker process? Also, if some other task is emitting tuples to the failed task how does the emitting task handle the failure of receiving task? Is there a way to capture and handler such case