That's interesting. When I match the number of executors to the number of workers, I always get exactly one executor per worker, at least with versions 0.9.4-0.9.6.
--John Sent from my iPhone > On Apr 21, 2016, at 12:19 AM, Vijay Patil <[email protected]> wrote: > > If your topology is the only topology running on 20 node cluster, then I > think you can reduce slots per supervisor to "1" by setting > "supervisor.slots.ports" (mention just 1 port number there) in storm.yaml. If > you are running multiple topologies on this 20 node cluster then this > solution may not work, need to think of something else like writing our own > meta-data aware custom scheduler by implementing > backtype.storm.scheduler.IScheduler. > > But I think there can be some scope for refactoring that HishTensionBolt in > order to reduce memory usage. All the memory used by that bolt remains static > for every tuple? Or it's "execute()" method which consumes that much memory > each time and discards it once tuple processing is done? > >> On 21 April 2016 at 06:47, <[email protected]> wrote: >> >> >> Hi, >> >> >> >> I have setup 4 workers on each machine. In my Topology, there is one bolt >> which needs a lot of memory, so ideally, I don’t want it to schedule more >> than 1 of that on any machines. In terms of computation it is pretty fast so >> it can manage good throughput when running. Lets call it, BoltHighTension. >> >> But, my other bolts are very light weight and I can have a lot of >> parallelism on that. >> >> >> >> How do I ensure that if I have 20 Supervisors, I don’t have more than 1 >> ‘BoltHighTension’ on each machine? I want to give parallelism hint of 20 to >> this bolt. >> >> But, I notice that sometimes, more than 1 such instance gets allocated on >> same machine. (Machine can handle 2, but the performance due to paging >> becomes a problem). >> >> >> >> Thanks for your help/advice/hints. >> >> >> >> Thanks >> >> -Abhishek >> >> >> >> Sent from Mail for Windows 10 >> >
