Spark Streaming - Shared hashmaps

2014-03-26 Thread Bryan Bryan
Hi there, I have read about the two fundamental shared features in spark (broadcasting variables and accumulators), but this is what i need. I'm using spark streaming in order to get requests from Kafka, these requests may launch long-running tasks, and i need to control them: 1) Keep them in a

Re: Spark Streaming - Shared hashmaps

2014-03-26 Thread Bryan Bryan
long running tasks (assuming 1 task corresponding to each > request from Kafka). > > TD > > > On Wed, Mar 26, 2014 at 1:19 AM, Bryan Bryan wrote: > >> Hi there, >> >> I have read about the two fundamental shared features in spark >> (broadcasting variabl