Yea, please report the bug on a supported Spark version like 2.4.
On Thu, Apr 23, 2020 at 3:40 PM Dhrubajyoti Hati
wrote:
> FYI we are using Spark 2.2.0. Should the change be present in this spark
> version? Wanted to check before opening a JIRA ticket?
>
>
>
>
> *Regards,Dhrubajyoti Hati.*
>
>
FYI we are using Spark 2.2.0. Should the change be present in this spark
version? Wanted to check before opening a JIRA ticket?
*Regards,Dhrubajyoti Hati.*
On Thu, Apr 23, 2020 at 10:12 AM Wenchen Fan wrote:
> This looks like a bug that path filter doesn't work for hive table
> reading. Can
This looks like a bug that path filter doesn't work for hive table reading.
Can you open a JIRA ticket?
On Thu, Apr 23, 2020 at 3:15 AM Dhrubajyoti Hati
wrote:
> Just wondering if any one could help me out on this.
>
> Thank you!
>
>
>
>
> *Regards,Dhrubajyoti Hati.*
>
>
> On Wed, Apr 22, 2020 a
Just wondering if any one could help me out on this.
Thank you!
*Regards,Dhrubajyoti Hati.*
On Wed, Apr 22, 2020 at 7:15 PM Dhrubajyoti Hati
wrote:
> Hi,
>
> Is there any way to discard files starting with dot(.) or ending with .tmp
> in the hive partition while reading from Hive table usi
Hi,
Is there any way to discard files starting with dot(.) or ending with .tmp
in the hive partition while reading from Hive table using spark.read.table
method.
I tried using PathFilters but they didn't work. I am using spark-submit and
passing my python file(pyspark) containing the source code.
