Invalid lambda deserialization

Hi, I'm writing a job to merge old data with changelogs using DataSet API where I'm reading changelog using TextInputFormat and old data using HadoopInputFormat. I can see, job manager has successfully deployed the program flow to worker nodes. However, workers are immediately going to failed sta

Re: Invalid lambda deserialization

Hi Amit, are you using lambdas as parameters of a Flink Function or in a member variable? If yes, can you share an lambda example that fails? Regards, Timo Am 1/3/18 um 11:41 AM schrieb Amit Jain: Hi, I'm writing a job to merge old data with changelogs using DataSet API where I'm reading c

[jira] [Created] (FLINK-8353) Add support for timezones

Timo Walther created FLINK-8353: --- Summary: Add support for timezones Key: FLINK-8353 URL: https://issues.apache.org/jira/browse/FLINK-8353 Project: Flink Issue Type: New Feature Compo

Re: Invalid lambda deserialization

Hi Timo, Thanks a lot! Quick re-look over the code helped me to detect used lambdas. I was using lambdas in two cases which are following. DataSet newMainDataSet = mainDataSet .fullOuterJoin(latestDeltaDataSet, JoinHint.REPARTITION_HASH_SECOND) .where(keySelector).equalTo(keySelector) *

Re: Docker images

Hi Felix, As a brief note, these images are not "official" in that they are owned by the Apache Flink project or the ASF, but they are maintained by the Flink community. The sources currently live in a separate repo to make them much easier to version and maintain independent of Flink itself. Thi

[jira] [Created] (FLINK-8354) Flink Kafka connector ingonre Kafka message headers

Mohammad Abareghi created FLINK-8354: Summary: Flink Kafka connector ingonre Kafka message headers Key: FLINK-8354 URL: https://issues.apache.org/jira/browse/FLINK-8354 Project: Flink I

Re: Invalid lambda deserialization

Hi Amit, which of the two lambdas caused the error? I guess it was the mapper after the parquet input, right? In both cases this should not happen. Maybe you can open an issue with a small reproducible code example? Thanks. Regards, Timo Am 1/3/18 um 12:15 PM schrieb Amit Jain: Hi Timo,

[jira] [Created] (FLINK-8355) DataSet Should not union a NULL row for AGG without GROUP BY clause.

sunjincheng created FLINK-8355: -- Summary: DataSet Should not union a NULL row for AGG without GROUP BY clause. Key: FLINK-8355 URL: https://issues.apache.org/jira/browse/FLINK-8355 Project: Flink

Re: Need to restart flink job on yarn as supervisord does

Hi Shivam, could you elaborate a little bit on the OutOfMemory issue you're observing? Maybe you could provide the logs. Cheers, Till On Tue, Jan 2, 2018 at 2:12 PM, Shivam Sharma <28shivamsha...@gmail.com> wrote: > Hi, > > I am using below restart strategy > > // Retry always > env.setRestartS

Re: Invalid lambda deserialization

Hi Amit, could this be related [1]? How do you build your job? [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=439889 Cheers, Till On Wed, Jan 3, 2018 at 2:55 PM, Timo Walther wrote: > Hi Amit, > > which of the two lambdas caused the error? I guess it was the mapper after > the parquet inpu

[jira] [Created] (FLINK-8356) JDBCAppendTableSink does not work for Hbase Phoenix Driver

Paul Wu created FLINK-8356: -- Summary: JDBCAppendTableSink does not work for Hbase Phoenix Driver Key: FLINK-8356 URL: https://issues.apache.org/jira/browse/FLINK-8356 Project: Flink Issue Type: Bu

Re: [jira] [Created] (FLINK-8356) JDBCAppendTableSink does not work for Hbase Phoenix Driver

I had a similar problem with batch API...the problem is that you have to enable autocommit in the connection URL. Thr jdbc connector should better handle this specific case as well (IMHO). See https://issues.apache.org/jira/browse/FLINK-7605 On 3 Jan 2018 22:25, "Paul Wu (JIRA)" wrote: > Paul W

Re: Flink-Yarn-Kerberos integration

Thanks a lot for the clarification, Eron. That's very helpful. Currently, we are more concerned about 1) data access, but will get to 2) and 3) eventually. I was thinking doing the following: 1) extend the current HadoopModule to use and refresh DTs as suggested on YARN Application Security docs.

[jira] [Created] (FLINK-8357) enable rolling in default log settings

Xu Mingmin created FLINK-8357: - Summary: enable rolling in default log settings Key: FLINK-8357 URL: https://issues.apache.org/jira/browse/FLINK-8357 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-8358) Hostname used by DataDog metric reporter is not configurable

Elias Levy created FLINK-8358: - Summary: Hostname used by DataDog metric reporter is not configurable Key: FLINK-8358 URL: https://issues.apache.org/jira/browse/FLINK-8358 Project: Flink Issue T