Re: Invalid lambda deserialization

2018-01-03 Thread Till Rohrmann
Hi Amit, could this be related [1]? How do you build your job? [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=439889 Cheers, Till On Wed, Jan 3, 2018 at 2:55 PM, Timo Walther wrote: > Hi Amit, > > which of the two lambdas caused the error? I guess it was the mapper after > the parquet inpu

Re: Invalid lambda deserialization

2018-01-03 Thread Timo Walther
Hi Amit, which of the two lambdas caused the error? I guess it was the mapper after the parquet input, right? In both cases this should not happen. Maybe you can open an issue with a small reproducible code example? Thanks. Regards, Timo Am 1/3/18 um 12:15 PM schrieb Amit Jain: Hi Timo,

Re: Invalid lambda deserialization

2018-01-03 Thread Amit Jain
Hi Timo, Thanks a lot! Quick re-look over the code helped me to detect used lambdas. I was using lambdas in two cases which are following. DataSet newMainDataSet = mainDataSet .fullOuterJoin(latestDeltaDataSet, JoinHint.REPARTITION_HASH_SECOND) .where(keySelector).equalTo(keySelector) *

Re: Invalid lambda deserialization

2018-01-03 Thread Timo Walther
Hi Amit, are you using lambdas as parameters of a Flink Function or in a member variable? If yes, can you share an lambda example that fails? Regards, Timo Am 1/3/18 um 11:41 AM schrieb Amit Jain: Hi, I'm writing a job to merge old data with changelogs using DataSet API where I'm reading c