Hi Subramanya, I could reproduce this behavior running a job in YARN cluster. This works in standalone cluster just fine. We've changed a little bit how the cache entries are distributed in 1.6.0. I am investigating this problem right now. Would you like to create a JIRA bug for it?
Best, Dawid On 19/09/18 06:41, Subramanya Suresh wrote: > Yes it works locally for me as well. We are running on YARN where it > fails on 1.6.0 though (works fine with 1.4.2). > > Regards, > > On Tue, Sep 18, 2018 at 1:47 AM, Fabian Hueske <fhue...@gmail.com > <mailto:fhue...@gmail.com>> wrote: > > Hi, > > The functionality of the SQL ScalarFunction is backed by Flink's > distributed cache and just passes on the function call. > I tried it locally on my machine and it works for me. > > What is your setup? Are you running on Yarn? > > Maybe Chesnay or Dawid (added to CC) can help to track the problem > down. > > Best, Fabian > > 2018-09-18 6:10 GMT+02:00 Subramanya Suresh > <ssur...@salesforce.com <mailto:ssur...@salesforce.com>>: > > Hi, > We are running into some trouble with > StreamExecutionEnvironment.registerCachedFile (works perfectly > fine in 1.4.2). > > * We register some CSV files in HDFS with > executionEnvironment.registerCachedFile("hdfs:///myPath/myCsv", > myCSV.csv) > * In a UDF (ScalarFunction), in the open function, we do a > FunctionContext.getCachedFile("myCSV) to load the CSV in a > singleton. > > We are running into a > > java.lang.IllegalArgumentException: File with name 'myCSV.csv' is not > available. Did you forget to register the file? > > Sincerely, > > > -- > > <http://smart.salesforce.com/sig/ssuresh//us_mb/default/link.html> > > > > > > -- > > <http://smart.salesforce.com/sig/ssuresh//us_mb/default/link.html>
signature.asc
Description: OpenPGP digital signature
