Re: Memory grows exponentially

2016-07-08 Thread Cody Koeninger
Just as an offhand guess, are you doing something like updateStateByKey without expiring old keys? On Fri, Jul 8, 2016 at 2:44 AM, Jörn Franke wrote: > Memory fragmentation? Quiet common with in-memory systems. > >> On 08 Jul 2016, at 08:56, aasish.kumar wrote: >> >> Hello everyone: >> >> I have

Re: Memory grows exponentially

2016-07-08 Thread Jörn Franke
Memory fragmentation? Quiet common with in-memory systems. > On 08 Jul 2016, at 08:56, aasish.kumar wrote: > > Hello everyone: > > I have been facing a problem associated spark streaming memory. > > I have been running two Spark Streaming jobs concurrently. The jobs read > data from Kafka with