Re: Option for silent failure while reading a list of files.

2019-07-01 Thread Naresh Peshwe
en moved offline, where the conclusion was "you get to > filter, sorry" > > https://issues.apache.org/jira/browse/SPARK-21797 > > > > On Mon, Jul 1, 2019 at 2:52 AM Naresh Peshwe > wrote: > >> Hi All, >> When I try to read a list parquet files from S3,

Option for silent failure while reading a list of files.

2019-06-30 Thread Naresh Peshwe
Hi All, When I try to read a list parquet files from S3, my application errors out if even one of the files are absent. When I searched for solutions most of them suggested filtering the list of files (on presence) before calling read. Shouldn't this be handled by Spark by providing an option for c