Looking at the source code for StateMap[1], which is used by JavaPairDStream#mapWithState(), it looks like state keys are serialized using an ObjectOutputStream. I couldn't find a reference to this restriction in the documentation. Did I miss that?
Unless I'm mistaken, I'm guessing there isn't a way to use Kryo for this serialization? Thanks! -Joey [1] https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/util/StateMap.scala#L251 --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
