Re: Slow performance in spark streaming

2014-11-21 Thread Prannoy
e a batch duration of 110.000 milliseconds in order to receive all > the data in one batch. I believed that, even locally, the spark is very > fast. In this case, it takes about 3.5sec to execute the rule (between > "RULE STARTS" and "RULE ENDS"). Am I doing something wron

Slow performance in spark streaming

2014-11-20 Thread Blackeye
} } case class Data (symbol: String, time: Int)/ I have a batch duration of 110.000 milliseconds in order to receive all the data in one batch. I believed that, even locally, the spark is very fast. In this case, it takes about 3.5sec to execute the rule (between "RULE STARTS" and "