Re: How to increase supervisor slots in apache storm 0.9.2

2014-10-08 Thread Jan Sipke van der Veen
Hello Sridhar, I think adding that to your storm.yaml should suffice. Have you restarted the supervisor afterwards? How much memory does your machine have? Each slot is essentially a new Java virtual machine which takes up memory. Best regards, Jan Sipke van der Veen On Oct 8, 2014 8:56 AM, "Srid

Re: How to increase supervisor slots in apache storm 0.9.2

2014-10-08 Thread Dima Dragan
Hi Sridhar, I think you should make rebalancing after changing storm.yaml. Check please: http://www.michael-noll.com/tutorials/running-multi-node-storm-cluster/#add-another-worker-node Best regards, Dmytro Dragan On Oct 8, 2014 9:56 AM, "Sridhar G" wrote: > Hi All, > > > > We are unable to inc

RE: How to increase supervisor slots in apache storm 0.9.2

2014-10-08 Thread Sridhar G
Hi All, Getting following error : in 'reader', line 41, column 1: supervisor.slots.ports: ^ at org.yaml.snakeyaml.parser.ParserImpl$ParseDocumentStart.produce(ParserImpl.java:225) at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:158)

Re: Is storm a good choice for our architecture?

2014-10-08 Thread Albert Vila
Now we are using Gearman, don't know if anyone succeed using Gearman to populate the Spouts. And one of my questions is if it's possible to isolate the execution of some bolts on a specific machine, so all keyword extraction for document X is done by the same machine it crawled it. Or maybe I shou

Re: How to increase supervisor slots in apache storm 0.9.2

2014-10-08 Thread Yuri Kostin
You might have an extra characters there, spaces, tabs, etc. Try http://yamllint.com On Oct 8, 2014, at 2:06 AM, Sridhar G wrote: > Hi All, > > Getting following error : > > in 'reader', line 41, column 1: > supervisor.slots.ports: > ^ > > at > org.yaml.snakeyaml.parser.P

Re: Is storm a good choice for our architecture?

2014-10-08 Thread Onur Ünlü
Hi Albert, You can use "local or shuffle grouping". https://storm.incubator.apache.org/documentation/Concepts.html (stream groupings) Onur On Wed, Oct 8, 2014 at 10:11 AM, Albert Vila wrote: > Now we are using Gearman, don't know if anyone succeed using Gearman to > populate the Spouts. > >

RE: How to increase supervisor slots in apache storm 0.9.2

2014-10-08 Thread Sridhar G
I get following error when do storm rebalance Exception in thread "main" java.lang.IllegalArgumentException: Field supervisor.slots.ports must be an Iterable of java.lang.Number at backtype.storm.ConfigValidation$1.validateField(ConfigValidation.java:63 ) at sun.reflect.Na

Confuse on setting spout name and get topology name in Trident topology

2014-10-08 Thread vlhf刘海峰
Hi, I tried to use name() method to name the nodes in a trident topology, and I saw the generated bolts then has a name like b{N}-{name} in storm UI, but the spout was still shown as spout0. How to name the spout node? Another question, in trident topology, via context.getStormId I can get the

Re: Is storm a good choice for our architecture?

2014-10-08 Thread Richards Peter
You can use custom scheduler to execute specific tasks on a particular machine. Please check the following mail thread: https://groups.google.com/forum/m/#!topic/storm-user/n_-HZWSdcmM Regards, Richards Peter.

Multiple nimbus configuration possible?

2014-10-08 Thread An Tran
Hi, I am wondering if it's possible to set up storm with multiple nimbus in its configuration or not. If possible, is it as simple as adding it to the storm.yml file? If this feature is not supported yet is there plans to support it in the near future? Thanks, An

Re: Multiple nimbus configuration possible?

2014-10-08 Thread Harsha
An, Its not possible to have multiple nimbus running at this point. Here is the JIRA for nimbus HA https://issues.apache.org/jira/browse/STORM-166 which is tracking the work to enable it. Thanks, Harsha On Wed, Oct 8, 2014, at 03:37 AM, An Tran wrote: > Hi, > > I am wonde

Re: Multiple nimbus configuration possible?

2014-10-08 Thread Harsha
An, Its not possible to have multiple nimbus running at this point. Here is the JIRA for nimbus HA https://issues.apache.org/jira/browse/STORM-166 which is tracking the work to enable it. Thanks, Harsha On Wed, Oct 8, 2014, at 03:37 AM, An Tran wrote: > Hi, > > I am wonde

java.lang.NullPointerException, at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor

2014-10-08 Thread Stefano Pugnetti
Hi all! I am trying to use an HDFS state in a Trident topology, but I always get the exception I report below. I tried the following config values topology.max.spout.pending=1 topology.trident.batch.emit.interval.millis=(500, then 1500, then 5000) but nothing changed. If I remove the partitio

Kryo

2014-10-08 Thread Klausen Schaefersinho
Hi, Every time I want to test a simple topology I get the following error which seems to be caused by different versions of the Kryo dependency. I have a dependency on my code on kryo (2.17), which I thought is the same version as Storm. But obviously not... Here is the exception: b.s.d.worker

Re: Kryo

2014-10-08 Thread Nathan Leung
0.9.2 release upgrades to kryo 2.21. On Wed, Oct 8, 2014 at 12:13 PM, Klausen Schaefersinho < klaus.schaef...@gmail.com> wrote: > Hi, > > Every time I want to test a simple topology I get the following error > which seems to be caused by different versions of the Kryo dependency. I > have a depen

Re: Kryo

2014-10-08 Thread Curtis Allen
You might find this interesting http://stackoverflow.com/questions/24244689/kryo-compatibility-in-storm-0-9-1and-titan-graph-database-0-4-2-with-apache-cass/24246845#24246845 On Wed, Oct 8, 2014 at 10:16 AM, Nathan Leung wrote: > 0.9.2 release upgrades to kryo 2.21. > > On Wed, Oct 8, 2014 at 12