Re: [spark executor error] container running from bad node-> exit code 134

That error could mean different things, most of the time is that the JVM crashed . If you are running yarn check the yarn logs or the stderr of your spark job to see if there is any more details of the cause On Fri, 19 Nov 2021 at 15:25, Joris Billen wrote: > Hi, > we are seeing this error: > >

[spark executor error] container running from bad node-> exit code 134

Hi, we are seeing this error: Job aborted due to stage failure: Task 0 in stage 1.0 failed 8...Reason: Container from a bad node: container_xxx on host: dev-yyy Exit status: 134 This post suggests this has to do with blacklisted nodes: https://stackoverflow.com/questions/65889696/spark-exit-stat

Re: Spark

You can't set a default, but you can replace NULL values with a value with a simple SQL statement that is applied to the stream. On Fri, Nov 19, 2021 at 8:28 AM Xiao, Alton wrote: > Hello, > > I am struggling with a task that should be super simple: > > I define a structType to load json d

Spark

Hello, I am struggling with a task that should be super simple: I define a structType to load json data from kafka with spark structed streaming, and some fields may have no value, how can I set a default value for this record? For example: StructType( Array(StructField("a", StringTyp