Spark-1.6.0-preview2 trackStateByKey exception restoring state

gContext.getOrCreate to actually read the checkpointed state when the job is started, leading to the exception below. Just to make sure: using StreamingContext.getOrCreate should still be compatible with using the trackStateByKey API? Thanx, Jan 15/11/23 10:55:07 ERROR StreamingContext: Error

Spark streaming 1.6.0-RC4 NullPointerException using mapWithState

nario has been tested successfully when: - step 3 is excluded, so restoring state from a checkpoint is successful when no messages are added when the job is stopped - after step 2, the checkpoints are deleted Any clues? Am I doing something wrong here, or is there still a problem with the mapWithSta

Re: Spark streaming 1.6.0-RC4 NullPointerException using mapWithState

ssue SPARK-12591 ( https://issues.apache.org/jira/browse/SPARK-12591). It contains the procedure to reproduce the error with the testcase which is again a modified version of the StatefulNetworkWordCount Spark streaming example ( https://gist.github.com/juyttenh/9b4a4103699a7d5f698f). Best regards, J

Contributing to managed memory, Tungsten..

But I have questions, about current situation, future direction and what would be the best way to contribute. Perhaps if some developer would spare a 30 minutes for a chat, and assign me some Issues for start. [1] https://github.com/jankotek/mapdb-hz-offheap[1] Thanks, Jan Kotek Ma

Re: Contributing to managed memory, Tungsten..

Hi Amit, I am slowly getting into it, so I will contact in a few weeks. Jan On Friday, March 11, 2016 09:22:27 Amit Chavan wrote: Hi Jan, Welcome to the group. I have used mapdb on some personal project and really enjoyed working with it. I am also willing to contribute to the spark

Task failures and other problems

Hello! This might not be the perfect list for the issue, but I tried user@ previously with the same issue, but with a bit less information to no avail. So I'm hoping someone here can point me into the right direction. We're using Spark 2.2 on CDH 5.13 (Hadoop 2.6 with patches) and a lot of our

Re: Task failures and other problems

Jörn Franke writes: > Maybe contact Oracle support? Something like that would be the last option I guess, university money is usually hard to come by for such things. > Do you have maybe accidentally configured some firewall rules? Routing > issues? Maybe only one of the nodes... All systems

Apache Spark chat channel

Hello! There was a request on scala-debate [0] to create a Spark centric chat room under the scala namespace on Gitter with a focus on Scala related questions. This is just a heads up to the Apache Spark "management" to give them a chance to get involved. It might be better to create a dedicated

Stackoverflow in createDataFrame.

org/apache/spark/sql/SQLContext.scala#L441> The function will just call itself over and over. If you consider this a bug as well I'll gladly open an JIRA issue :). Cheers Jan

Re: Stackoverflow in createDataFrame.

Sorry, missed that issue :) > On 24 Apr 2015, at 15:17, yash datta wrote: > > This is already reported : > > https://issues.apache.org/jira/browse/SPARK-6999 > <https://issues.apache.org/jira/browse/SPARK-6999> > On 24 Apr 2015 18:11, "Jan-Paul Bultmann&