Re: There is no space for new record

Hi , I am facing the following when running on EMR Caused by: java.lang.IllegalStateException: There is no space for new record at org.apache.spark.util.collection.unsafe.sort. UnsafeInMemorySorter.insertRecord(UnsafeInMemorySorter.java:226) at org.apache.spark.sql.execution.Unsa

Re: There is no space for new record

This has been reported before: http://apache-spark-developers-list.1001551.n3.nabble.com/java-lang-IllegalStateException-There-is-no-space-for-new-record-tc20108.html I think we may have a real bug here, but we need a reproduce. Can you provide one? thanks! On Fri, Feb 9, 2018 at 5:59 PM, SNEHASI

Structured Streaming Broadcasts

Hi all, I'm working on a Structured Streaming job that uses a Spark ML algorithm. The algorithm includes the broadcast of a dataframe. The idea is that the Structured Streaming job will stay running, and process data as it comes in from a Kafka bus. On start up, I have specified that I want at lea

Re: There is no space for new record

It should be fixed by https://github.com/apache/spark/pull/20561 soon. On Fri, Feb 9, 2018 at 6:16 PM, Wenchen Fan wrote: > This has been reported before: http://apache-spark- > developers-list.1001551.n3.nabble.com/java-lang- > IllegalStateException-There-is-no-space-for-new-record-tc20108.html