Process data in micro batch On 18-Oct-2017 10:36 AM, "Chetan Khatri" <chetan.opensou...@gmail.com> wrote:
> Your hard drive don't have much space > On 18-Oct-2017 10:35 AM, "Mina Aslani" <aslanim...@gmail.com> wrote: > >> Hi, >> >> I get "No space left on device" error in my spark worker: >> >> Error writing stream to file /usr/spark-2.2.0/work/app-......./0/stderr >> java.io.IOException: No space left on device >> >> In my spark cluster, I have one worker and one master. >> My program consumes stream of data from kafka and publishes the result >> into kafka. I set my RDD = null after I finish working, so that >> intermediate shuffle files are removed quickly. >> >> How can I avoid "No space left on device"? >> >> Best regards, >> Mina >> >
