Oh I should add I've tried a range of batch durations and reduce by window durations to no effect. I'm not too sure how to choose these?
<br/><br/> Currently today I've been testing with batch duration of 1 minute - 10 minute and reduce window duration of 10 minute or 20 minutes. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-unable-to-handle-production-Kafka-load-tp15077p15080.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
