If you look at your streaming app UI you should see how many tasks are executed
each batch and on how many executors. This is dependent on the batch duration
and block interval, which defaults to 200ms. So every block interval a
partition will be generated. You can control the parallelism by adj
Any suggestions please ..!!
How to know that In stream Processing over the cluster of 8 machines
all the machines/woker nodes are being used (my cluster have 8 slaves )
.
I am submitting job from master itself over the ec-2 cluster crated by the
ec-2 scripts available with spark. But i am
How to know that In stream Processing over the cluster of 8 machines
all the machines/woker nodes are being used (my cluster have 8 slaves )
.
--
Thanks & Regards,
Anshu Shukla
