Re: Weird error in submitting a flink job to yarn cluster

2017-10-04 Thread Chesnay Schepler
This isn't related to FLink but i might be able to help you out anyway. Does the ParquestFileWriter set the 'overwrite' flag when calling 'FileSystem#create()'? My suspicion is that you create a file for the first batch, write it out, but not delete it. For the next batch, the file cannot be

Weird error in submitting a flink job to yarn cluster

2017-10-02 Thread vipul singh
Hello, I am working on a ParquetSink writer, which will convert a kafka stream to parquet format. I am having some weird issues in deploying this application to a yarn cluster. I am not 100% sure this falls into a flink related error, but I wanted to reach out to folks here incase it might be. I