Re: Migration to Flink 1.6.0, issues with StreamExecutionEnvironment.registerCachedFile

2018-09-19 Thread Dawid Wysakowicz
Hi Subramanya, I could reproduce this behavior running a job in YARN cluster. This works in standalone cluster just fine. We've changed a little bit how the cache entries are distributed in 1.6.0. I am investigating this problem right now. Would you like to create a JIRA bug for it? Best, Dawid

Re: Migration to Flink 1.6.0, issues with StreamExecutionEnvironment.registerCachedFile

2018-09-18 Thread Subramanya Suresh
Yes it works locally for me as well. We are running on YARN where it fails on 1.6.0 though (works fine with 1.4.2). Regards, On Tue, Sep 18, 2018 at 1:47 AM, Fabian Hueske wrote: > Hi, > > The functionality of the SQL ScalarFunction is backed by Flink's > distributed cache and just passes on th

Re: Migration to Flink 1.6.0, issues with StreamExecutionEnvironment.registerCachedFile

2018-09-18 Thread Fabian Hueske
Hi, The functionality of the SQL ScalarFunction is backed by Flink's distributed cache and just passes on the function call. I tried it locally on my machine and it works for me. What is your setup? Are you running on Yarn? Maybe Chesnay or Dawid (added to CC) can help to track the problem down.

Migration to Flink 1.6.0, issues with StreamExecutionEnvironment.registerCachedFile

2018-09-17 Thread Subramanya Suresh
Hi, We are running into some trouble with StreamExecutionEnvironment.registerCachedFile (works perfectly fine in 1.4.2). - We register some CSV files in HDFS with executionEnvironment.registerCachedFile("hdfs:///myPath/myCsv", myCSV.csv) - In a UDF (ScalarFunction), in the open function