Why spark streaming does not support groupingsets

2020-06-10 Thread ashok.blend
We have scenario where we wanted to generate groupy on specific set of dimensions. Grouping sets solves our purpose, but its not supported in spark streaming. We cannot go for cube as it generates lot of combination which will be useless for us. So wanted to know if there is any specific reason wh

JVM Crashes after running example for long time

2017-10-17 Thread ashok.blend
hi We have integrated spark with yarn cluster. To test for long running cluster, we ran below script to launch application, an example programme as given at bottom. #Script which runs example infinitely /while true do echo "start $times" /opt/ficlient/Spark/spark/bin/spark-submit --class com.examp