Re: Re: Implementing a TarInputFormat based on FileInputFormat

2021-01-07 Thread Yun Gao
: android.disableChaining().sinkTo(sink); Best, Yun [1] https://issues.apache.org/jira/browse/FLINK-20888 --Original Mail -- Sender:Billy Bain Send Date:Thu Jan 7 04:02:34 2021 Recipients:Arvid Heise CC:user , Billy Bain Subject:Re: Implementing a TarInputFormat based on

Re: Implementing a TarInputFormat based on FileInputFormat

2021-01-06 Thread Billy Bain
Hi Arvid, Thanks for the response. I have created a sample application with input data and uploaded it to google drive. The sample data is in the archive... thus the large size. (27 mb) https://drive.google.com/file/d/1dxpnDF3hPUPNlPO5p2tBf-88oOVV0qck/view?usp=sharing To run it: flink run -Dexe

Re: Implementing a TarInputFormat based on FileInputFormat

2021-01-05 Thread Arvid Heise
Hi Billy, the exception is happening on the output side. Input side looks fine. Could you maybe post more information about the sink? On Mon, Dec 28, 2020 at 8:11 PM Billy Bain wrote: > I am trying to implement a class that will work similar to AvroFileFormat. > > This tar archive has a very sp

Implementing a TarInputFormat based on FileInputFormat

2020-12-28 Thread Billy Bain
I am trying to implement a class that will work similar to AvroFileFormat. This tar archive has a very specific format. It has only one file inside and that file is line delimited JSON. I get this exception, but all the data is written to the temporary files. I have checked that my code isn't clo