Re: NotSerializableException in Spark runner

2020-12-01 Thread Ajit Dongre
another node with NotSerializableException. Regards, Ajit Dongre From: Alexey Romanenko Reply to: "user@beam.apache.org" Date: Tuesday, 1 December 2020 at 11:35 PM To: "user@beam.apache.org" Subject: EXT: Re: NotSerializableException in Spark runner Could you make sure

Re: NotSerializableException in Spark runner

2020-12-01 Thread Alexey Romanenko
Could you make sure that the instance of com.walmart.dataplatform.aorta.river.meta.Payload, created from a failed record, is serializable? > On 1 Dec 2020, at 06:06, Ajit Dongre wrote: > > Hi all, > > I have pipeline to read data from kafka & write to file. I am using Beam > 2.12 with spar

NotSerializableException in Spark runner

2020-11-30 Thread Ajit Dongre
Hi all, I have pipeline to read data from kafka & write to file. I am using Beam 2.12 with spark runner in java. While executing I am getting below exception : org.apache.spark.network.client.ChunkFetchFailureException: Failure while fetching StreamChunkId{streamId=1660417857014, chunkIndex=0